Skip to main content

Find image on screen (screen.find_image)

Declarations​

x, y = screen.find_image(image [, confidence, left, top, right, bottom ])
results = screen.find_image(image [, options, left, top, right, bottom ])

Parameters​

  • image
    String: the image data to search for, in PNG or JPEG format.
    Image object: an image object (see Image module (image)).
    Text: path to the image file to search for. If the path is not valid, it will be parsed as raw image data.

  • confidence
    Integer, optional. Default 95. Only positions whose confidence is not less than this value will be returned. Range: 0 ~ 100.

  • options
    Table, optional.

    Structure
    {
        find_all = false | true, -- Optional. When true, find all matches; return value is a table. Default false
        confidence_threshold = number_value(0.0 ~ 100.0), -- Optional. Results with confidence >= this are valid. Default 95
        downscale = number_value(0.1 ~ 1.0), -- Optional. Smaller = faster but less precise. Default 1.0 (accuracy first)
        mask = image_object, -- Optional. Marks valid areas of the template image. A binary image of the same size as img; white = valid, black = invalid. Default: all areas valid
    }

  • left, top, right, bottom
    Integers, optional. Search region. Default: full screen.

Returns​

  • x, y
    Integers. When options.find_all is not true, return only the coordinates of the most confident valid match. If none meets the threshold, returns -1, -1.

  • results
    Table.

    When options.find_all is true, returns multiple results with confidence >= options.confidence_threshold
    {
        {
            x = integer_value,
            y = integer_value,
            confidence = number_value(0.0 ~ 100.0),
        },
        ...
    }

Notes​

Find the position of a small image on the screen. This function relies on the image.cv module.
Note: For multi‑resolution compatibility, it is recommended to capture the template on the smallest‑resolution device; screenshots from higher‑resolution devices may not be found on lower‑resolution ones.

Enhancements in version 1.3.8 and later​

  • Support finding all matches above the confidence threshold via find_all mode.
  • Support downscale to speed up by lowering precision.
  • Support mask to mark valid areas in the template for irregular‑shape matching.

Examples​

XXT Picker 1.0.32 for Windows.7z

-- Example 1 (Use XXT Picker: Shift + Left Click to box the region; code like this is generated):
x, y = screen.find_image( -- Original position TL: 354,274 | BR: 358,284
"\x89\x50\x4e\x47\x0d\x0a\x1a\x0a\x00\x00\x00\x0d\x49\x48\x44\x52\x00\x00\x00\x04\x00\x00\x00\x0a\x08\x02\x00\x00\x00\x1c\x99\x68\x59\x00\x00\x00\x61\x49\x44\x41\x54\x78\xda\x63\x78\xfd\xf4\xda\xff\xff\xff\xff\xfd\xfb\xf7\xed\xcb\x5b\x86\xf7\xaf\x1f\xfc\x87\x01\x86\x2f\x1f\x5f\x02\xa9\xef\xa7\xce\x7c\xdd\xb1\x9b\xe1\xe7\xf7\xcf\x40\xce\xeb\xb2\xea\x7b\xb2\x6a\x0c\x7f\xff\xfe\x01\x72\x9e\x78\x06\x82\x38\x20\xdd\xbf\x7e\xdd\x57\xd4\x82\x72\x7e\xdd\xba\x0d\x64\x41\x39\x08\xd3\x80\x38\x6b\xe3\x7f\x86\x2a\x30\x02\x72\x8c\xa6\x40\x39\x00\xd5\x7b\x5f\x2e\xfd\xba\xd5\x32\x00\x00\x00\x00\x49\x45\x4e\x44\xae\x42\x60\x82"
, 95, 0, 0, 639, 1135)
--
-- Example 2:
img = image.load_file("/var/mobile/1.png")
x, y = screen.find_image(img)
x, y = screen.find_image(img, 95)
--
-- Example 4:
x, y = screen.find_image("/var/mobile/1.png", 95, 0, 0, 639, 1135)

-- Example 5 (added in 1.3.8): find multiple targets on full screen
img = image.load_file(XXT_SCRIPTS_PATH..'/1.png')
rets = screen.find_image(img, {
    confidence_threshold = 90, -- valid if confidence >= this value
    downscale = 0.5, -- speed up by lowering precision
    find_all = true, -- find all matches; returns an array of match entries
}, 0, 0, 0, 0) -- region 0,0,0,0 means full screen

Note: In Lua string literals, a sequence starting with \x followed by two hex digits denotes a single byte with that value. For printable characters, see the ASCII reference.

Complex example​

-- Download a small image (part of XXTouch icon) and then find it on screen and tap it
local c, h, r = http.get("https://www.xxtouch.app/img/find_image_test.png", 10)
if (c == 200) then
    local img = image.load_data(r)
    if img then
        x, y = screen.find_image(img, 95)
        if x~=-1 then
            touch.tap(x, y)
        else
            sys.alert("XXTouch icon not found on screen")
        end
    else
        sys.alert("Downloaded data is not a valid image")
    end
else
    sys.alert("Download failed")
end

Note: Uses functions outside this chapter: sys.alert, image.load_data, touch.tap, http.get