ONNX Runtime Tensor Module

This page documents the most commonly used tensor-related functions and object methods in the onnxruntime module.

Creation and conversion

onnxruntime.tensor(type, shape[, data])

tensor, err = onnxruntime.tensor("float32", {1, 3}, {1, 2, 3})

Creates a regular ORT tensor.

• type is the element type name
• shape is the tensor shape
• data is optional; when omitted, an empty tensor is created
• Numeric tensors accept a scalar, which fills the entire tensor with the same value
• string tensors accept a single string, which fills the entire tensor with that same string

onnxruntime.tensor_from_bytes(type, shape, bytes)

tensor, err = onnxruntime.tensor_from_bytes("float32", {1, 3}, raw_bytes)

Creates a tensor from contiguous raw bytes.

• Only numeric and bool types are supported
• The byte length must exactly match the given shape and type

onnxruntime.tensor_from_cv_mat(mat[, opts])

tensor, err = onnxruntime.tensor_from_cv_mat(mat, {
    layout = "hwc",
    channel_order = "rgb",
    type = "uint8",
})

Converts a cv.mat into a tensor.

Notes:

• require("image.cv") first
• opts.type controls the target tensor element type

onnxruntime.tensor_from_quad(mat, quad[, opts])

tensor, err = onnxruntime.tensor_from_quad(mat, {
    {x = 0, y = 0},
    {x = 100, y = 0},
    {x = 100, y = 32},
    {x = 0, y = 32},
}, {
    width = 100,
    height = 32,
    layout = "hwc",
    channel_order = "rgb",
    type = "uint8",
})

Runs a perspective crop from a cv.mat using a quadrilateral region and converts the result directly into an ORT tensor.

• Requires require("image.cv")
• quad may be passed directly as four points, or as a table with a points field
• Common opts mostly match tensor_from_image(), with extra commonly used fields such as content_width, content_height, and border_type
• Useful for OCR rectification plus tensorization of a single text box

onnxruntime.tensor_from_quads(mat, quads[, opts])

batch_tensor, err = onnxruntime.tensor_from_quads(mat, {
    {
        points = quad1,
        content_width = 96,
        content_height = 32,
    },
    {
        points = quad2,
        content_width = 80,
        content_height = 32,
    },
}, {
    width = 96,
    height = 32,
    resize_mode = "top_left_letterbox",
    border_type = "replicate",
})

Runs perspective crops for multiple quadrilaterals and automatically packs the results into a batch tensor.

• Requires require("image.cv")
• quads must be a non-empty array; each item may include points
• Per-item content_width and content_height override the same global fields in opts
• The return value is packed with stack() or concat() depending on tensor rank, which is convenient for OCR batch preprocessing

onnxruntime.tensor_from_image(image[, opts])

tensor, meta = onnxruntime.tensor_from_image(image, {
    width = 224,
    height = 224,
    layout = "nchw",
    channel_order = "rgb",
    data_type = "float32",
    scale = 1 / 255,
    mean = {0.485, 0.456, 0.406},
    std = {0.229, 0.224, 0.225},
    resize_mode = "letterbox",
})

Converts an image object directly into an ONNX Runtime input tensor.

Common options:

• width / height
• layout: "nchw", "nhwc", "chw", or "hwc"
• channel_order: "rgb", "bgr", "gray", "grey", or "grayscale"
• data_type
• scale
• mean
• std
• resize_mode: "stretch", "letterbox", or "center_crop"
• letterbox_mode: supports "top_left"; otherwise letterboxing is centered
• pad_color
• interpolation: "bilinear" or "nearest"
• alpha_mode: "ignore", "white", "black", or "premultiply"
• crop = {x, y, width, height}
• add_batch

On success, the second return value is a preprocessing metadata table. Common fields include:

• src_width / src_height
• crop_x / crop_y / crop_width / crop_height
• dst_width / dst_height
• resized_width / resized_height
• layout
• channel_order
• resize_mode
• scale_x / scale_y / ratio
• offset_x / offset_y
• pad_left / pad_top / pad_right / pad_bottom

onnxruntime.tensor_from_images(images[, opts])

batch_tensor, metas = onnxruntime.tensor_from_images({img1, img2}, {
    width = 640,
    height = 640,
    layout = "nchw",
})

Converts a batch of images into a tensor.

• Source image sizes may differ
• The batch can be formed as long as each image resolves to the same output shape and data_type
• The second return value is a metadata array aligned with the input order

onnxruntime.image_from_tensor(tensor[, opts])

image, err = onnxruntime.image_from_tensor(tensor, {
    layout = "nchw",
    channel_order = "rgb",
    batch_index = 1,
    scale = 1 / 255,
    mean = {0.485, 0.456, 0.406},
    std = {0.229, 0.224, 0.225},
    value_range = "0_1",
})

Converts a 2D / 3D / 4D tensor back into an image object. Useful for debugging model inputs and outputs.

Common options:

• layout
• channel_order
• batch_index: 1-based, default 1
• scale
• mean
• std
• clamp
• value_range: "0_255" or "0_1"

Notes:

• Only 2D / 3D / 4D tensors are supported
• The channel count must be 1 or 3

Tensor object methods

Basic information

• tensor:shape()
• tensor:rank()
• tensor:size()
• tensor:type()
• tensor:to_table()
• tensor:bytes()

Notes:

• to_table() expands the tensor contents into a Lua table
• bytes() only supports numeric and bool tensors

Read, write, and copy

• tensor:get(index1[, index2, ...])
• tensor:set(index1[, index2, ...], value)
• tensor:fill(value_or_table)
• tensor:clone()
• tensor:copy_from_bytes(raw_bytes)
• tensor:to(type)

Notes:

• fill() accepts either a scalar fill value or a Lua table whose element count must match exactly
• copy_from_bytes() only supports numeric and bool tensors, and the byte length must match exactly
• get() / set() use 1-based indexing semantics
• tensor:to("string") currently only supports string -> string

Shape and indexing

• tensor:reshape(shape)
• tensor:transpose([axes])
• tensor:flatten([start_dim[, end_dim]])
• tensor:squeeze([dim])
• tensor:unsqueeze(dim)
• tensor:slice(dim, start, stop[, step])
• tensor:select(dim, index)
• tensor:gather(dim, indices)

Notes:

• slice() uses 1-based start and stop, and stop is inclusive
• slice() requires a positive step
• select() removes the selected dimension
• gather() accepts either a Lua array or a shape-[N] tensor as indices, still using 1-based indexing semantics
• These methods return new tensor objects

Numeric operations

• tensor:add(other)
• tensor:sub(other)
• tensor:mul(other)
• tensor:div(other)
• tensor:clamp(min, max)
• tensor:sigmoid()
• tensor:exp()
• tensor:matmul(other)
• tensor:dot(other)

Notes:

• other can be a scalar or another tensor with a matching shape
• matmul() currently supports rank-1 / rank-2 tensor combinations
• sigmoid(), exp(), and matmul() promote results to a floating-point result type
• These operations do not support string tensors

Reductions, sorting, and probability

• tensor:argmax([axis])
• tensor:sum([axis])
• tensor:mean([axis])
• tensor:max([axis])
• tensor:min([axis])
• tensor:softmax([axis])
• tensor:normalize([axis])
• tensor:sort([axis[, descending]])
• tensor:topk(k[, axis])

Notes:

• argmax() without an axis returns a single 1-based index
• argmax(axis) returns an int64 tensor, also with 1-based indices
• sort() returns { values = tensor, indices = tensor }
• topk() returns { values = tensor, indices = tensor }
• sort() and topk() both return 1-based indices

OpenCV bridge

• tensor:to_cv_mat([opts])

mat, err = tensor:to_cv_mat({
    layout = "hwc",
    channel_order = "rgb",
    coreml_data_type = "uint8",
})

Notes:

• require("image.cv") first
• Some tensor types cannot be mapped to cv.mat directly; in those cases pass coreml_data_type explicitly

Module-level tensor helpers

Basic numeric helpers

• onnxruntime.clamp(tensor, min, max)
• onnxruntime.sigmoid(tensor)
• onnxruntime.exp(tensor)
• onnxruntime.where(condition, x, y)
• onnxruntime.matmul(lhs, rhs)
• onnxruntime.concat(tensors[, axis])
• onnxruntime.stack(tensors[, axis])

Notes:

• clamp(), sigmoid(), exp(), and matmul() share the same implementation as the corresponding tensor: methods
• where() supports scalar / boolean / tensor mixtures and follows broadcasting rules

Additional post-processing helpers

• onnxruntime.mask_iou(lhs_mask, rhs_mask)
• onnxruntime.db_postprocess(score_map[, opts])

Notes:

• mask_iou() computes the intersection-over-union between two masks directly
• mask_iou() also accepts a third opts argument with compare_size = true, or explicit width / height for the comparison size
• db_postprocess() is intended for DB / DBNet-style text-detection post-processing; inputs may be [H, W], [C, H, W], or [N, C, H, W]
• db_postprocess() returns a detection array where each item contains score, points, and box; meta / image_meta can reuse preprocessing metadata returned by image tensorization

onnxruntime.nms(boxes, scores[, opts])

Axis-aligned box NMS.

Common options:

• iou_threshold
• score_threshold
• top_k
• class_aware
• class_ids

Returns an int64 tensor with 1-based indices.

onnxruntime.box_points(rotated_boxes)

Converts rotated boxes [cx, cy, w, h, theta] into four corner points.

• The input may be a tensor shaped [5], [1, 5], or [N, 5]
• A single box returns one Lua point list; multiple boxes return an array of point lists

onnxruntime.xywh_to_xyxy(boxes)

Converts boxes from [cx, cy, w, h] to [x1, y1, x2, y2].

onnxruntime.xyxy_to_xywh(boxes)

Converts boxes from [x1, y1, x2, y2] to [cx, cy, w, h].

onnxruntime.rotated_iou(box1, box2)

Computes IoU between two rotated boxes.

onnxruntime.rotated_nms(boxes, scores[, opts])

Rotated-box NMS. The return value is also an int64 tensor with 1-based indices.

• boxes must be shaped [N, 5]
• scores may be a Lua numeric array or a tensor shaped [N] / [N, 1]

onnxruntime.create_decoder(schema)

Creates a reusable decoder object.

• Decoder objects support :decode(output[, opts]), :task(), and :schema()
• This is useful when a detection / OBB / classification schema should be defined once and reused

onnxruntime.decode_yolo(output[, opts])

Decodes output with the built-in YOLO detection path and returns detection records.

onnxruntime.decode_yolo_obb(output[, opts])

Decodes output with the built-in YOLO OBB path and returns rotated detection records.

onnxruntime.decode_matrix_candidates(output, schema[, opts])

Splits a matrix output into candidate tensors according to a schema. The returned table commonly contains:

• boxes
• scores
• class_ids
• keep_indices
• selected_rows
• angles (for OBB-related schemas)

onnxruntime.decode_dense_detection(output, opts)

Decodes a dense detection head output into:

• boxes
• scores
• labels

All three fields are tensors.

• Inputs may be shaped [R, C] or [N, R, C]
• opts.strides is required, and it must be a non-empty array of positive integers
• decode_width and decode_height are also required
• Currently only box_encoding = "grid_center_log_wh" is supported
• Other common fields include box_offset, score_offset, class_offset, num_classes, and score_threshold
• Batched outputs return a Lua array of per-batch results

onnxruntime.records_from_boxes(boxes, scores, class_ids[, keep_indices])

Turns [N, 4] boxes, scores, and class IDs into Lua record tables. Each record commonly includes:

• box
• score
• class_id
• row_index
• x1 / y1 / x2 / y2
• width / height
• cx / cy

onnxruntime.obb_records_from_rows(rows, scores, class_ids[, angles[, keep_indices[, opts]]])

Turns OBB row tensors into Lua record tables.

• opts supports x_index, y_index, width_index, and height_index

onnxruntime.points_to_records(points[, opts])

Turns [N, P, D] or [N, P*D] point / keypoint tensors into Lua tables.

• opts supports point_count / keypoint_count
• opts supports point_dim / keypoint_dim

Mask helpers

onnxruntime.threshold_masks(masks, threshold)

Thresholds dense mask tensors into Lua mask tables. Each mask contains:

• width
• height
• bits
• pixel_count
• bounds

onnxruntime.crop_masks_by_boxes(masks, boxes)

Crops thresholded mask tables using [N, 4] boxes.

onnxruntime.resize_masks(masks, width, height[, opts])

Resizes mask tables to a target size.

• Only opts.interpolation = "nearest" is currently supported

onnxruntime.mask_to_polygon(mask[, opts])

Converts one binary mask into a polygon point list.

• opts.epsilon / opts.approx_epsilon can be used for polygon simplification

onnxruntime.proto_masks(proto, coeffs, boxes, image_width, image_height[, opts])

Projects prototype masks, mask coefficients, and detection boxes back into the target image size.

• project_masks() is an alias
• The return value is a Lua mask-table list, not a tensor

Keypoint and geometry helpers

• onnxruntime.reshape_keypoints(points[, keypoint_count[, keypoint_dim|opts]])
• onnxruntime.scale_boxes(boxes, transform)
• onnxruntime.clip_boxes(boxes, clip_width, clip_height)
• onnxruntime.scale_points(points, transform[, opts])
• onnxruntime.scale_keypoints(points, transform[, opts])
• onnxruntime.clip_keypoints(points, clip_width, clip_height[, opts])

Notes:

• reshape_keypoints() reshapes between [N, K*D] and [N, K, D]
• scale_points() assumes a generic point layout by default; scale_keypoints() assumes a keypoint layout by default
• Transform tables align with image preprocessing metadata fields such as scale_x, scale_y, pad_left, and pad_top

onnxruntime.tracker([opts])

Creates a reusable tracker object with:

• tracker:update(detections[, timestamp])
• tracker:reset()
• tracker:state()
• tracker:close()

Common config fields:

• iou_threshold
• max_age
• min_hits

onnxruntime.ctc_greedy_decode(logits[, opts])

Returns a table like:

• indices

• text

• confidence

• Inputs may be shaped [T, C] or [N, T, C]

• Supports blank_index, merge_repeated, apply_softmax, return_probabilities, and charset

• Always returns indices

• text appears only when charset is provided

• confidence appears only when apply_softmax or return_probabilities is enabled

• probabilities and probability_confidence appear only when return_probabilities is enabled

• Batched inputs return an array of per-batch decode results

onnxruntime.sample_logits(logits[, opts])

Supports these sampling options:

• argmax
• temperature
• top_k
• top_p
• min_p
• seed

For 1D logits it returns a scalar index. For batched logits it returns an int64 tensor. Indices are 1-based.

Example

local ort = require("onnxruntime")

local tensor = assert(ort.tensor("float32", {2, 3}, {
    1, 9, 3,
    8, 2, 7,
}))

local sliced = assert(tensor:slice(2, 2, 3))
print(sliced:to_table()[1]) -- 9

local topk = assert(tensor:topk(2, 2))
print(topk.values:to_table()[1])  -- 9
print(topk.indices:to_table()[1]) -- 2