Skip to main content

! OCR Text in Region (screen.ocr_text)

Declaration​

text, details = screen.ocr_text(left, top, right, bottom [, engine_options, binarization_options ])

Parameters​

  • left, top, right, bottom
    Integer. Region on the screen. 0, 0, 0, 0 means full screen.

  • engine_options
    Optional, table. Choose recognition language and engine.

    Structure
    {
        -- If engine = "apple", iOS 13+ uses Vision.framework
        -- You can use image.vision_supported_recognition_languages() to get available OCR models for Vision.framework
        -- If engine = "paddle", uses Paddle-Lite. You can specify lang, e.g. lang = "ppocr_ch" means model at /var/mobile/Media/1ferver/models/ppocr_ch
        -- Paddle-Lite supports Slim models in *.nb format
        engine = "apple" | "paddle" | "tesseract",
        lang = "zh-Hans",
    }
    Built-in Vision.framework OCR models by iOS version
    { -- iOS 13
        [1] = "en-US",
    }

    { -- iOS 14~15
        [1] = "en-US",
        [2] = "fr-FR",
        [3] = "it-IT",
        [4] = "de-DE",
        [5] = "es-ES",
        [6] = "pt-BR",
        [7] = "zh-Hans",
        [8] = "zh-Hant",
    }

    { -- iOS 16
        [ 1] = "en-US",
        [ 2] = "fr-FR",
        [ 3] = "it-IT",
        [ 4] = "de-DE",
        [ 5] = "es-ES",
        [ 6] = "pt-BR",
        [ 7] = "zh-Hans",
        [ 8] = "zh-Hant",
        [ 9] = "yue-Hans",
        [10] = "yue-Hant",
        [11] = "ko-KR",
        [12] = "ja-JP",
        [13] = "ru-RU",
        [14] = "uk-UA",
    }

  • binarization_options
    Number, threshold. See Image auto binarization
    Table, custom color-bias binarization. See Image manual binarization
    String, custom color-bias binarization. See Image manual binarization

Returns​

  • text
    String. Recognized text.

  • details
    Table.

    OCR result detail structure
    {
        {
            ["y"] = number_value,
            ["x"] = number_value,
            ["w"] = number_value,
            ["h"] = number_value,
            ["confidence"] = number_value(0.0000 ~ 1.0000),
            ["text"] = string_value,
        },
        ...
    }

Description​

For backward compatibility, when engine is not specified, the deprecated tesseract engine is used.
Supported engines Apple and Paddle-Lite require software version 1.3.8 or above.

Examples​

-- 1.3.8+
text, info = screen.ocr_text(187, 882, 298, 914, "en-US")   -- iOS 13+ recognizes English using built-in en-US model
sys.toast("Result: "..text:atrim())

text, info = screen.ocr_text(187, 882, 298, 914, "zh-Hans") -- iOS 14+ recognizes Simplified Chinese using built-in zh-Hans model
sys.toast("Result: "..text:atrim())

text, info = screen.ocr_text(187, 882, 298, 914, {
    engine = "apple",
    lang = "zh-Hans"
}, "9D5D39-0F1F26,D3D3D2-2C2C2D")
sys.toast("Result: "..text:atrim())

text, info = screen.ocr_text(0, 0, 0, 0, {
    engine = "paddle",
    lang = "ppocr_ch",
})
sys.toast("Result: "..text:atrim())

Note: Uses sys.toast, string.atrim

Built-in default OCR engine is tesseract 3.02. Incorrect version or damaged language file may crash XXTouch script service.
XXTouch bundles eng language for [A-Za-z0-9].
For Simplified Chinese or other languages, copy the language file into /var/mobile/Media/1ferver/tessdata/.
Provided: Simplified Chinese language file (needs extract)

-- Example 1:
local txt = screen.ocr_text(187, 882, 298, 914) -- default is tesseract English digits mode
sys.toast("Result: "..txt:atrim())
--
-- Example 2:
local txt = screen.ocr_text(465, 241, 505, 269, "eng", "9D5D39-0F1F26,D3D3D2-2C2C2D") -- color-bias binarization
sys.toast("Result: "..txt:atrim())
--
-- Example 3:
local txt = screen.ocr_text(465, 241, 505, 269, "eng", { {0x9D5D39, 0x0F1F26}, {0xD3D3D2, 0x2C2C2D} }) -- same as above
sys.toast("Result: "..txt:atrim())
--
-- Example 4:
local txt = screen.ocr_text(187, 882, 298, 914, {
  lang = "chi_sim",         -- tesseract Simplified Chinese (not bundled)
  white_list = "你我他",
})
sys.toast("Result: "..txt:atrim())
--
-- Example 5:
local txt = screen.ocr_text(187, 882, 298, 914, {
  lang = "eng",
  white_list = "1234567890",      -- only digits
}, "9D5D39-0F1F26,D3D3D2-2C2C2D") -- color-bias binarization
sys.toast("Result: "..txt:atrim())