Skip to main content

Run image inference with a CoreML Vision request (:predict)

Declaration​

result, err = vision_request:predict(image)

or

submitted, err = vision_request:predict(image, {
    async = is_async,
    multi_array_output = "table" or "MLMultiArray",
    uses_cpu_only = cpu_only_for_this_call,
})

Parameters​

  • image
    An image object to run inference on.

  • async
    Boolean, optional. When true, submits the request asynchronously. Defaults to false.

  • multi_array_output
    String, optional. When the model returns MLMultiArray features, choose "table" or "MLMultiArray". Defaults to "table".

  • uses_cpu_only
    Boolean, optional. Applies only to this call. When true, this inference runs on CPU only.

Returns​

  • result
    Table.

    Object detection model return structure
    {
        {
            ["y"] = number_value,
            ["x"] = number_value,
            ["w"] = number_value,
            ["h"] = number_value,
            ["confidence"] = number_value(0.0 ~ 100.0),
            ["name"] = string_value,
        },
        ...
    }
    Image classification model return structure
    {
        {
            ["confidence"] = number_value(0.0 ~ 100.0),
            ["name"] = string_value,
        },
        ...
    }
    Other model types
    {
        ...
    }

Notes​

  • Available since app version 1.3.8+.
  • Not supported on iOS versions below 12.
  • Runs inference on the image using the current Vision request and returns the result.
  • If the model returns MLMultiArray feature values, set multi_array_output = "MLMultiArray" to keep them as native tensor objects for further processing.
  • :run() is an alias of :predict(). After async submission, use :is_done() and :results() to read the result.
  • Vision request objects also expose methods such as :metadata(), :compute_units(), :image_crop_and_scale_option(), :class_labels(), and :is_vision_request()

Example​

compiled_model_path = XXT_HOME_PATH..'/models/yolo11m.mlmodelc'

file.remove(compiled_model_path)

if not file.exists(compiled_model_path) then
    local tmp_path, err = coreml.compile_model(XXT_HOME_PATH..'/models/yolo11m.mlpackage')
    if not tmp_path then
        error(err)
    end
    file.move(tmp_path, compiled_model_path, 'mo')
end

vnrequest, err = coreml.new_vision_request(compiled_model_path)
if not vnrequest then
    error(err)
end

rets = vnrequest:predict(screen.image())
nLog(rets)