Skip to main content

ONNX Runtime Module

This module is available in versions released after 20260402
Only supported on iOS 13 and later

The onnxruntime module loads and runs ONNX models directly on device, suitable for text, embeddings, classification, detection, and general-purpose tensor inference workflows.

Load the module​

local ort = require("onnxruntime")

Unlike coreml, this is not a built-in global module; it is loaded on demand.

After require("onnxruntime") succeeds, it also injects two native copy bridges into the built-in coreml APIs:

  • coreml.multi_array_from_ort_tensor(tensor[, data_type])
  • multi_array:to_ort_tensor([data_type])

Both conversions are native-layer copies and do not go through Lua tables.

Module-level functions​

Runtime and basic information​

  • onnxruntime.version()
  • onnxruntime.providers()
  • onnxruntime.configure(opts)

Notes:

  • providers() returns the execution providers actually available in the current ORT runtime
  • configure() sets global runtime defaults and must be called before creating any session

Tensors, images, and numeric helpers​

  • onnxruntime.tensor(type, shape[, data])
  • onnxruntime.tensor_from_bytes(type, shape, bytes)
  • onnxruntime.tensor_from_cv_mat(mat[, opts])
  • onnxruntime.tensor_from_quad(mat, quad[, opts])
  • onnxruntime.tensor_from_quads(mat, quads[, opts])
  • onnxruntime.tensor_from_image(image[, opts])
  • onnxruntime.tensor_from_images(images[, opts])
  • onnxruntime.image_from_tensor(tensor[, opts])
  • onnxruntime.clamp(tensor, min, max)
  • onnxruntime.sigmoid(tensor)
  • onnxruntime.exp(tensor)
  • onnxruntime.where(condition, x, y)
  • onnxruntime.matmul(lhs, rhs)
  • onnxruntime.concat(tensors[, axis])
  • onnxruntime.stack(tensors[, axis])

Notes:

  • clamp(), sigmoid(), exp(), and matmul() are equivalent to the same-named tensor: methods, with the tensor passed as the first argument
  • where() supports mixing scalars, booleans, and tensors, and applies broadcasting rules
  • Image preprocessing, OpenCV bridges, and image_from_tensor() details are documented in the Tensor Module

Detection, decoding, and post-processing helpers​

  • onnxruntime.nms(boxes, scores[, opts])
  • onnxruntime.box_points(rotated_boxes)
  • onnxruntime.xywh_to_xyxy(boxes)
  • onnxruntime.xyxy_to_xywh(boxes)
  • onnxruntime.rotated_iou(lhs_box, rhs_box)
  • onnxruntime.rotated_nms(boxes, scores[, opts])
  • onnxruntime.create_decoder(schema)
  • onnxruntime.decode_yolo(output[, opts])
  • onnxruntime.decode_yolo_obb(output[, opts])
  • onnxruntime.decode_matrix_candidates(output, schema[, opts])
  • onnxruntime.decode_dense_detection(output, opts)
  • onnxruntime.records_from_boxes(boxes, scores, class_ids[, keep_indices])
  • onnxruntime.obb_records_from_rows(rows, scores, class_ids[, angles[, keep_indices[, opts]]])
  • onnxruntime.points_to_records(points[, opts])
  • onnxruntime.threshold_masks(masks, threshold)
  • onnxruntime.crop_masks_by_boxes(masks, boxes)
  • onnxruntime.resize_masks(masks, width, height[, opts])
  • onnxruntime.mask_iou(lhs_mask, rhs_mask)
  • onnxruntime.mask_to_polygon(mask[, opts])
  • onnxruntime.proto_masks(proto, coeffs, boxes, image_width, image_height[, opts])
  • onnxruntime.project_masks(proto, coeffs, boxes, image_width, image_height[, opts])
  • onnxruntime.db_postprocess(score_map[, opts])
  • onnxruntime.tracker([opts])
  • onnxruntime.reshape_keypoints(points[, keypoint_count[, keypoint_dim|opts]])
  • onnxruntime.scale_boxes(boxes, transform)
  • onnxruntime.clip_boxes(boxes, clip_width, clip_height)
  • onnxruntime.scale_points(points, transform[, opts])
  • onnxruntime.scale_keypoints(points, transform[, opts])
  • onnxruntime.clip_keypoints(points, clip_width, clip_height[, opts])
  • onnxruntime.ctc_greedy_decode(logits[, opts])
  • onnxruntime.sample_logits(logits[, opts])

Notes:

  • tensor_from_quad() / tensor_from_quads() require require("image.cv") first and are useful when OCR pipelines need to crop quadrilateral regions directly into tensors
  • box_points() takes a rotated-box tensor shaped [5], [1, 5], or [N, 5]; it does not take five separate scalar arguments
  • create_decoder() returns a decoder object with :decode(), :task(), and :schema()
  • tracker() returns a tracker object with :update(), :reset(), :state(), and :close()
  • records_from_boxes(), obb_records_from_rows(), and points_to_records() reshape tensor outputs into Lua-friendly record tables
  • proto_masks() and project_masks() currently share the same implementation; project_masks() is an alias
  • mask_iou() computes the intersection-over-union between two masks directly, and also accepts a third opts argument with compare_size = true, or explicit width / height
  • db_postprocess() is intended for DB / DBNet-style text-detection post-processing; each returned detection contains score, points, and box
  • decode_dense_detection() requires opts.strides, and it must be a non-empty array of positive integers; it also requires decode_width and decode_height, and currently supports only box_encoding = "grid_center_log_wh"
  • ctc_greedy_decode() supports blank_index, merge_repeated, apply_softmax, return_probabilities, and charset
  • ctc_greedy_decode() always returns indices; text appears only when charset is provided; confidence appears only when apply_softmax or return_probabilities is enabled; probabilities and probability_confidence appear only when return_probabilities is enabled
  • nms() and rotated_nms() return int64 tensors with 1-based indices
  • sample_logits() supports argmax, temperature, top_k, top_p, min_p, and seed
  • sample_logits() returns a scalar index for 1D logits, and an int64 tensor for batched logits

Structured values​

  • onnxruntime.value(value)
  • onnxruntime.optional(value, type_info)
  • onnxruntime.sequence(items)
  • onnxruntime.map(key_type, value_type, pairs)
  • onnxruntime.sparse_tensor(type, dense_shape, indices, values)
  • onnxruntime.sparse_tensor_from_dense(tensor)

These are useful when model inputs or outputs are not plain tensors — for example optionals, sequences, maps, or sparse tensors.

The current behavior can be summarized like this:

  • onnxruntime.value(x) If x is already an ORT tensor / value / sequence / map / sparse tensor, it is returned as-is; if x is a Lua table, it becomes a sequence; otherwise the scalar is wrapped as a tensor
  • onnxruntime.optional(value, type_info) The second argument is required; type_info may be a string or a type-info table such as the result of session:input_info(...) / output_info(...); an empty optional is written as onnxruntime.optional(nil, type_info)
  • onnxruntime.map(key_type, value_type, pairs) At the moment, key_type only supports "string" or "int64"
  • onnxruntime.sparse_tensor(type, dense_shape, indices, values) Currently only numeric / bool sparse tensors are supported, built in COO form; indices may be flat or nested
  • onnxruntime.sparse_tensor_from_dense(tensor) Currently does not support string tensors

Common object methods:

  • value:type() / value:has_value() / value:get()
  • sequence:length() / sequence:get(i) / sequence:items()
  • map:get(key) / map:set(key, value) / map:keys() / map:pairs()
  • sparse_tensor:dense_shape() / sparse_tensor:values() / sparse_tensor:indices() / sparse_tensor:format() / sparse_tensor:to_dense()

Sessions and inference​

  • onnxruntime.session(model_path[, opts])
  • onnxruntime.session_from_bytes(model_bytes[, opts])
  • onnxruntime.run_options([opts])
  • onnxruntime.load_custom_op_library(path)

Session objects handle model loading, input/output inspection, inference execution, and IOBinding. See the Session Module.

Supported element types​

The tensor APIs currently support these type names:

  • "float32" / "float"
  • "float16"
  • "bfloat16"
  • "uint8"
  • "uint16"
  • "uint32"
  • "uint64"
  • "int8"
  • "int16"
  • "int32"
  • "int64"
  • "double" / "float64"
  • "bool"
  • "string"

Notes:

  • tensor_from_bytes() and copy_from_bytes() only support numeric and bool tensors
  • bytes() is not available for string tensors
  • tensor:to("string") currently only supports string -> string

Provider notes​

onnxruntime.providers() returns what the runtime reports as available, but the provider strings currently recognized by the session option parser are:

  • "cpu"
  • "coreml"

Notes:

  • provider / providers also accept aliases such as CPUExecutionProvider and CoreMLExecutionProvider; they are normalized internally to "cpu" and "coreml"
  • If no provider is specified, or the provider list is empty, session creation appends the CPU provider automatically
  • If the provider list contains "coreml" and fallback_to_cpu = true, the implementation may also append CPU as a fallback path
  • If you pass providers = {"coreml", "cpu"}, the session tries CoreML first and CPU second

Working with CoreML​

If you want to reuse the coreml tokenizer or an MLMultiArray preprocessing pipeline, a common pattern is:

local ort = require("onnxruntime")

local tokenizer = assert(coreml.new_text_tokenizer({
    type = "wordpiece",
    vocab_path = XXT_HOME_PATH.."/models/demo/vocab.txt",
    context_length = 52,
}))

local input_ids = assert(tokenizer:encode("hello", {
    output = "ort_tensor",
}))

Or convert an existing MLMultiArray directly into an ORT tensor:

local ort = require("onnxruntime")
local tensor = assert(multi_array:to_ort_tensor("int64"))