Skip to main content

! OCR Text in Region (screen.ocr_text)

Declaration​

text, details = screen.ocr_text(left, top, right, bottom [, engine_options, binarization_options ])

Parameters​

  • left, top, right, bottom
    Integer. Region on the screen. 0, 0, 0, 0 means full screen.

  • engine_options
    Optional, table. Choose recognition language and engine.

    Structure
    {
        -- If engine = "apple", iOS 13+ uses Vision.framework
        -- You can use image.vision_supported_recognition_languages() to get available OCR models for Vision.framework
        -- If engine = "paddle", uses Paddle-Lite. You can specify lang, e.g. lang = "ppocr_ch" means model at /var/mobile/Media/1ferver/models/ppocr_ch
        -- Paddle-Lite supports Slim models in *.nb format
        engine = "apple" | "paddle" | "tesseract",
        lang = "zh-Hans",
    }
    Built-in Vision.framework OCR models by iOS version
    { -- iOS 13
        [1] = "en-US",
    }

    { -- iOS 14~15
        [1] = "en-US",
        [2] = "fr-FR",
        [3] = "it-IT",
        [4] = "de-DE",
        [5] = "es-ES",
        [6] = "pt-BR",
        [7] = "zh-Hans",
        [8] = "zh-Hant",
    }

    { -- iOS 16
        [ 1] = "en-US",
        [ 2] = "fr-FR",
        [ 3] = "it-IT",
        [ 4] = "de-DE",
        [ 5] = "es-ES",
        [ 6] = "pt-BR",
        [ 7] = "zh-Hans",
        [ 8] = "zh-Hant",
        [ 9] = "yue-Hans",
        [10] = "yue-Hant",
        [11] = "ko-KR",
        [12] = "ja-JP",
        [13] = "ru-RU",
        [14] = "uk-UA",
    }

  • binarization_options
    Number, threshold. See Image auto binarization
    Table, custom color-bias binarization. See Image manual binarization
    String, custom color-bias binarization. See Image manual binarization

Returns​

  • text
    String. Recognized text.

  • details
    Table.

    OCR result detail structure
    {
        {
            ["y"] = number_value,
            ["x"] = number_value,
            ["w"] = number_value,
            ["h"] = number_value,
            ["confidence"] = number_value(0.0000 ~ 1.0000),
            ["text"] = string_value,
        },
        ...
    }

Description​

Coordinate parameters and return values follow the coordinate system initialized via screen.init.
For backward compatibility, when engine is not specified, the deprecated tesseract engine is used.
Supported engines Apple and Paddle-Lite require software version 1.3.8 or above.
PaddleLite Simplified Chinese standard model (requires extraction)
PaddleLite Simplified Chinese full model (requires extraction)
PaddleLite Traditional Chinese model (requires extraction)
PaddleLite Japanese model (requires extraction)
PaddleLite Korean model (requires extraction)
PaddleLite English alphanumeric model (requires extraction)
You can also download models from the PaddleOCR model list, convert them to det_opt.nb, cls_opt.nb, and rec_opt.nb, and pair them with the appropriate dict.txt

Examples​

-- 1.3.8+
text, info = screen.ocr_text(187, 882, 298, 914, "en-US")   -- iOS 13+ recognizes English using built-in en-US model
sys.toast("Result: "..text:atrim())

text, info = screen.ocr_text(187, 882, 298, 914, "zh-Hans") -- iOS 14+ recognizes Simplified Chinese using built-in zh-Hans model
sys.toast("Result: "..text:atrim())

text, info = screen.ocr_text(187, 882, 298, 914, {
    engine = "apple",
    lang = "zh-Hans"
}, "9D5D39-0F1F26,D3D3D2-2C2C2D")
sys.toast("Result: "..text:atrim())

text, info = screen.ocr_text(0, 0, 0, 0, {
    engine = "paddle",
    lang = "ppocr_ch",
})
sys.toast("Result: "..text:atrim())

Note: Uses sys.toast, string.atrim

Built-in default OCR engine is tesseract 3.02. Incorrect version or damaged language file may crash XXTouch script service.
XXTouch bundles eng language for [A-Za-z0-9].
For Simplified Chinese or other languages, copy the language file into /var/mobile/Media/1ferver/tessdata/.
Provided: Simplified Chinese language file (needs extract)

-- Example 1:
local txt = screen.ocr_text(187, 882, 298, 914) -- default is tesseract English digits mode
sys.toast("Result: "..txt:atrim())
--
-- Example 2:
local txt = screen.ocr_text(465, 241, 505, 269, "eng", "9D5D39-0F1F26,D3D3D2-2C2C2D") -- color-bias binarization
sys.toast("Result: "..txt:atrim())
--
-- Example 3:
local txt = screen.ocr_text(465, 241, 505, 269, "eng", { {0x9D5D39, 0x0F1F26}, {0xD3D3D2, 0x2C2C2D} }) -- same as above
sys.toast("Result: "..txt:atrim())
--
-- Example 4:
local txt = screen.ocr_text(187, 882, 298, 914, {
  lang = "chi_sim",         -- tesseract Simplified Chinese (not bundled)
  white_list = "你我他",
})
sys.toast("Result: "..txt:atrim())
--
-- Example 5:
local txt = screen.ocr_text(187, 882, 298, 914, {
  lang = "eng",
  white_list = "1234567890",      -- only digits
}, "9D5D39-0F1F26,D3D3D2-2C2C2D") -- color-bias binarization
sys.toast("Result: "..txt:atrim())