Skip to main content

ML multi-array module

MLMultiArray is the most common tensor type in CoreML.
This page summarizes the coreml APIs related to MLMultiArray, including:

  • creating tensors
  • converting Lua / image / OpenCV data into tensors
  • converting between MLMultiArray and ORT tensors
  • common math, reductions, sorting, and concatenation
  • detection / OBB / mask / keypoint / tracker post-processing helpers

If you want to wrap classifiers, embedding models, text models, detection models, or other general-purpose CoreML models in Lua, this page is the main foundation.

This module is available in versions released after 20260319

Creation and conversion​

coreml.new_multi_array(opts) / coreml.tensor(opts)​

array, err = coreml.new_multi_array({
    shape = shape,
    data_type = data_type,
})

Creates an empty CoreML multi-array object that can later be used as model input or an intermediate tensor.

  • shape is the target shape, for example {1, 3, 224, 224}
  • data_type can be "int32", "float32", "float16", or "double"; "float64" is accepted as an alias for "double"
  • coreml.tensor(opts) is an equivalent alias

coreml.multi_array_from_table(data, opts) / coreml.tensor_from_table(data, opts)​

array, err = coreml.multi_array_from_table(data, {
    shape = shape,
    data_type = data_type,
})

Converts Lua table data explicitly into an MLMultiArray.

  • shape must match the total number of elements
  • data_type follows the same rules as new_multi_array()
  • coreml.tensor_from_table(...) is an equivalent alias

coreml.tensor_from_image(image[, opts])​

array, meta = coreml.tensor_from_image(image, opts)

Converts an image object into a CoreML-ready tensor using explicit preprocessing options.

Common options:

  • width / height
  • layout: only "nchw" and "nhwc" are supported
  • channel_order: "rgb", "bgr", "gray", "grey", or "grayscale"
  • data_type
  • scale
  • mean
  • std
  • resize_mode: "stretch", "letterbox", "center_crop"
  • letterbox_mode: supports "center" and "top_left"; "topleft" is treated as "top_left"
  • pad_color
  • interpolation: "bilinear", "nearest"
  • alpha_mode: "ignore", "white", "black", "premultiply"
  • crop = {x, y, width, height}

Notes:

  • This function only performs explicitly configured image tensorization; it does not silently bind itself to one model's preprocessing rules
  • On success, the second return value is a preprocessing metadata table; on failure, it returns nil, err
  • Common metadata fields include: src_width, src_height, crop_x, crop_y, crop_width, crop_height, dst_width, dst_height, resized_width, resized_height, scale_x, scale_y, ratio, pad_left, pad_top, pad_right, pad_bottom, resize_mode, offset_x, offset_y, and letterbox_mode

coreml.tensor_from_images(images[, opts])​

batch_array, batch_meta = coreml.tensor_from_images({img1, img2}, {
    width = 640,
    height = 640,
    layout = "nchw",
})

Converts a batch of images into one MLMultiArray.

  • Source image sizes may differ
  • The batch can be formed as long as each image resolves to the same output shape and data_type
  • The second return value is a metadata array aligned with the input order

coreml.image_from_tensor(tensor[, opts])​

image, err = coreml.image_from_tensor(tensor, opts)

Converts a 2D / 3D / 4D MLMultiArray back into an image object. Useful for debugging model inputs and outputs.

Common options:

  • layout
  • channel_order
  • batch_index: 1-based, default 1
  • scale
  • mean
  • std
  • clamp
  • value_range: "0_255" or "0_1"

Notes:

  • Only 2D / 3D / 4D tensors are supported
  • The channel count must be 1 or 3

coreml.image_to_multi_array(image[, opts])​

This is the old API name. It is now only a compatibility alias of coreml.tensor_from_image(...). New code should prefer tensor_from_image().

coreml.tensor_from_cv_mat(mat[, opts]) / coreml.multi_array_from_cv_mat(mat[, opts])​

Converts a cv.mat into an MLMultiArray.

  • Requires require("image.cv")
  • Both names are equivalent aliases

coreml.tensor_from_quad(mat, quad[, opts]) / coreml.multi_array_from_quad(mat, quad[, opts])​

array, err = coreml.tensor_from_quad(mat, {
    {x = 0, y = 0},
    {x = 100, y = 0},
    {x = 100, y = 32},
    {x = 0, y = 32},
}, {
    width = 100,
    height = 32,
    layout = "hwc",
    channel_order = "rgb",
    data_type = "float32",
})

Runs a perspective crop from a cv.mat using a quadrilateral region and converts the result directly into an MLMultiArray.

  • Requires require("image.cv")
  • quad may be passed directly as four points, or as a table with a points field
  • Common opts mostly match tensor_from_image(), with extra commonly used fields such as content_width, content_height, and border_type
  • Useful for OCR rectification plus tensorization of a single text box

coreml.tensor_from_quads(mat, quads[, opts]) / coreml.multi_array_from_quads(mat, quads[, opts])​

batch_array, err = coreml.tensor_from_quads(mat, {
    {
        points = quad1,
        content_width = 96,
        content_height = 32,
    },
    {
        points = quad2,
        content_width = 80,
        content_height = 32,
    },
}, {
    width = 96,
    height = 32,
    resize_mode = "top_left_letterbox",
    border_type = "replicate",
})

Runs perspective crops for multiple quadrilaterals and automatically packs the results into a batch tensor.

  • Requires require("image.cv")
  • quads must be a non-empty array; each item may include points
  • Per-item content_width and content_height override the same global fields in opts
  • The return value is packed with stack() or concat() depending on tensor rank, which is convenient for OCR batch preprocessing

coreml.multi_array_from_ort_tensor(tensor[, data_type])​

array, err = coreml.multi_array_from_ort_tensor(ort_tensor[, "float32"])

Copies an onnxruntime.tensor into an MLMultiArray using a native bridge.

  • This function does not exist by default; it is injected into the built-in coreml module only after require("onnxruntime")
  • The conversion is a native copy and does not go through a Lua table
  • string tensors cannot be converted into MLMultiArray

Type checks and aliases​

coreml.is_multi_array(value) / coreml.is_tensor(value)​

is_multi_array = coreml.is_multi_array(value)

Checks whether a value is a coreml_multi_array_object. is_tensor() is an equivalent alias.

Module-level helper functions​

Basic tensor helpers​

  • coreml.concat(arrays, axis)
  • coreml.stack(arrays, axis)
  • coreml.gather(array, dim, indices)
  • coreml.take(array, indices[, dim])
  • coreml.gather_rows(array, indices)
  • coreml.clamp(array, min, max)
  • coreml.sigmoid(array)
  • coreml.exp(array)
  • coreml.where(condition, x, y)
  • coreml.matmul(lhs, rhs)

Notes:

  • take() defaults to dimension 1
  • gather_rows() is a convenience alias for take(array, indices, 1)
  • where() supports mixing scalars and MLMultiArray values and applies broadcasting rules
  • matmul() currently supports rank-1 / rank-2 input combinations

Geometry and detection helpers​

  • coreml.nms(boxes, scores[, opts])
  • coreml.box_points(rotated_boxes)
  • coreml.xywh_to_xyxy(boxes)
  • coreml.xyxy_to_xywh(boxes)
  • coreml.rotated_iou(lhs, rhs)
  • coreml.rotated_nms(boxes, scores[, opts])

Notes:

  • nms() expects boxes shaped [N, 4] and scores shaped [N] or [N, C]
  • rotated_nms() expects boxes shaped [N, 5]; its scores argument is currently a Lua number array
  • Both return an MLMultiArray of 1-based kept indices

Decode and record helpers​

  • coreml.create_decoder(schema)
  • coreml.decode_yolo(output[, opts])
  • coreml.decode_yolo_obb(output[, opts])
  • coreml.decode_matrix_candidates(output, schema[, opts])
  • coreml.decode_dense_detection(output, opts)
  • coreml.records_from_boxes(boxes, scores, class_ids[, keep_indices])
  • coreml.obb_records_from_rows(rows, scores, class_ids[, angles[, keep_indices[, opts]]])
  • coreml.points_to_records(points[, opts])

Notes:

  • create_decoder() returns a decoder object that supports :decode(), :task(), and :schema()
  • decode_dense_detection() returns { boxes, scores, labels }; when the input is batched, it returns a batch result array
  • decode_dense_detection() requires opts.strides, and it must be a non-empty array of positive integers
  • decode_dense_detection() also requires decode_width and decode_height, and currently supports only box_encoding = "grid_center_log_wh"
  • records_from_boxes(), obb_records_from_rows(), and points_to_records() turn tensor outputs into record tables that are easier to consume from Lua

Mask, keypoint, and tracking helpers​

  • coreml.threshold_masks(masks, threshold)
  • coreml.crop_masks_by_boxes(masks, boxes)
  • coreml.resize_masks(masks, width, height[, opts])
  • coreml.mask_iou(lhs_mask, rhs_mask)
  • coreml.mask_to_polygon(mask[, opts])
  • coreml.proto_masks(proto, coeffs, boxes, image_width, image_height[, opts])
  • coreml.project_masks(proto, coeffs, boxes, image_width, image_height[, opts])
  • coreml.db_postprocess(score_map[, opts])
  • coreml.tracker([opts])
  • coreml.reshape_keypoints(points[, keypoint_count[, keypoint_dim|opts]])
  • coreml.scale_boxes(boxes, transform)
  • coreml.clip_boxes(boxes, clip_width, clip_height)
  • coreml.scale_points(points, transform[, opts])
  • coreml.scale_keypoints(points, transform[, opts])
  • coreml.clip_keypoints(points, clip_width, clip_height[, opts])
  • coreml.ctc_greedy_decode(logits[, opts])
  • coreml.sample_logits(logits[, opts])

Notes:

  • project_masks() is an alias of proto_masks()
  • mask_iou() computes the intersection-over-union between two masks directly
  • mask_iou() also accepts a third opts argument with compare_size = true, or explicit width / height for the comparison size
  • db_postprocess() is intended for DB / DBNet-style text-detection post-processing; inputs may be [H, W], [C, H, W], or [N, C, H, W]
  • db_postprocess() returns a detection array where each item contains score, points, and box; meta / image_meta can reuse preprocessing metadata returned by image tensorization
  • tracker() returns a tracker object that supports :update(), :reset(), :state(), and :close()
  • ctc_greedy_decode() accepts [T, C] or [N, T, C] logits
  • ctc_greedy_decode() supports blank_index, merge_repeated, apply_softmax, return_probabilities, and charset
  • ctc_greedy_decode() always returns indices; text appears only when charset is provided; confidence appears only when apply_softmax or return_probabilities is enabled; probabilities and probability_confidence appear only when return_probabilities is enabled
  • sample_logits() supports argmax, temperature, top_k, top_p, min_p, and seed
  • sample_logits() returns a single 1-based index for 1D logits; for batched logits it returns an index MLMultiArray

Basic object methods​

Basic queries​

  • array:shape()
  • array:data_type()
  • array:count()
  • array:strides()
  • array:to_table()
  • array:to_cv_mat([opts])

Notes:

  • data_type() returns "int32", "float32", "float16", or "double"
  • to_cv_mat() requires require("image.cv")

ORT bridge​

  • array:to_ort_tensor([data_type])

This method does not exist by default; it is injected into coreml_multi_array_object only after require("onnxruntime").

Type and shape transforms​

  • array:astype(data_type)
  • array:clone()
  • array:reshape(shape)
  • array:transpose(axes)
  • array:slice(dim, start, stop[, step])
  • array:select(dim, index)
  • array:squeeze([dim])
  • array:unsqueeze(dim)
  • array:flatten([start_dim[, end_dim]])

Notes:

  • reshape(), transpose(), squeeze(), unsqueeze(), and flatten() return views by default and do not copy underlying data
  • reshape() / flatten() fail on non-contiguous layouts; call clone() first if needed
  • slice() and select() return new contiguous tensors, not views
  • The indexing semantics of slice() / select() / gather() / take() stay 1-based to match the ONNX docs

Numeric and indexing methods​

  • array:gather(dim, indices)
  • array:take(indices[, dim])
  • array:l2_norm()
  • array:dot(other)
  • array:max([axis])
  • array:min([axis])
  • array:add(other)
  • array:sub(other)
  • array:mul(other)
  • array:div(other)
  • array:clamp(min, max)
  • array:sigmoid()
  • array:exp()
  • array:matmul(other)
  • array:scale(number)

Notes:

  • add/sub/mul/div support scalars and limited broadcasting
  • sigmoid(), exp(), and matmul() promote results to floating-point output

Reductions, ranking, and selection​

  • array:sum([axis])
  • array:mean([axis])
  • array:softmax([axis])
  • array:normalize([axis])
  • array:argmax([axis])
  • array:topk(k[, axis])
  • array:sort([axis[, descending]])

Notes:

  • argmax() without an axis returns the 1-based linear index of the maximum value in the whole array
  • argmax(axis) returns an MLMultiArray of indices
  • topk() returns { values = tensor, indices = tensor }
  • sort() returns a new sorted MLMultiArray; it does not return a separate index table

Geometry and post-processing object methods​

  • array:clip_boxes(clip_width, clip_height)
  • array:xywh_to_xyxy()
  • array:xyxy_to_xywh()
  • array:reshape_keypoints([keypoint_count[, keypoint_dim|opts]])
  • array:scale_points(transform[, opts])
  • array:clip_keypoints(clip_width, clip_height[, opts])

These methods share the same underlying implementation as the module-level helpers; they simply pass the current array as the first argument.

Usage notes​

  • In the newer general-purpose CoreML APIs, MLMultiArray is the default first-class data type; avoid converting to Lua tables prematurely
  • Most bulk operations should stay on tensor objects; use to_table() mainly for debugging, small outputs, or legacy code compatibility
  • Image preprocessing should be configured explicitly through tensor_from_image() options rather than baking one model's preprocessing into a generic pipeline
  • If you need an isolated copy, or want to turn a non-contiguous layout into a contiguous tensor, call clone() explicitly

Example​

local arr = assert(coreml.tensor({
    shape = {2, 3},
    data_type = "float32",
}))

local filled = assert(coreml.tensor_from_table({
    {1, 2, 3},
    {4, 5, 6},
}, {
    shape = {2, 3},
    data_type = "float32",
}))

local merged = assert(coreml.concat({filled, filled}, 1))
print(coreml.is_tensor(merged))
print(merged:shape())