Skip to main content

Use CoreML predictor to infer an image (:predict)

Declaration​

result = vnreq:predict(image)

Parameters​

Returns​

  • result
    Table.

    Object detection model return structure
    {
        {
            ["y"] = number_value,
            ["x"] = number_value,
            ["w"] = number_value,
            ["h"] = number_value,
            ["confidence"] = number_value(0.0 ~ 100.0),
            ["name"] = string_value,
        },
        ...
    }
    Image classification model return structure
    {
        {
            ["confidence"] = number_value(0.0 ~ 100.0),
            ["name"] = string_value,
        },
        ...
    }
    Other model types
    {
        ...
    }

Notes​

  • Available since app version 1.3.8+.
  • Not supported on iOS versions below 13.
  • Perform inference with the vision request context and return results.

Example​

compiled_model_path = XXT_HOME_PATH..'/models/yolo11m.mlmodelc'

local noexecute = require('no_os_execute')
noexecute.rm_rf(compiled_model_path)

if not file.exists(compiled_model_path) then
    local tmp_path, err = coreml.compile_model(XXT_HOME_PATH..'/models/yolo11m.mlpackage')
    if not tmp_path then
        error(err)
    end
    local noexecute = require('no_os_execute')
    noexecute.cp_r(tmp_path, compiled_model_path)
    noexecute.rm_rf(tmp_path)
end

vnrequest, err = coreml.new_vision_request(compiled_model_path)
if not vnrequest then
    error(err)
end

rets = vnrequest:predict(screen.image())
nLog(rets)