Screen Capture & OCR (screen)

screen.screen.color_count

screen.color_count(r, g, b [, x1, y1, x2, y2 [, tolerance]])

Count pixels matching RGB color. NEON SIMD accelerated.

screen.screen.color_offset

screen.color_offset(color1, color2)

Calculate per-channel absolute difference between two colors.

screen.screen.color_similarity

screen.color_similarity(color1, color2)

Calculate weighted similarity between two packed colors. Formula: 100 - (30/255*|r1-r2| + 40/255*|g1-g2| + 30/255*|b1-b2…

screen.screen.detect_barcodes

screen.detect_barcodes([x1, y1, x2, y2 [, timeout_ms]])

Detect barcodes/QR codes on screen using Vision framework.

screen.screen.diff_percent

screen.diff_percent([tolerance])

Get percentage of screen that changed since last capture.

screen.screen.find_all_color

screen.find_all_color(r, g, b [, x1, y1, x2, y2 [, tolerance [, limit]]])

Find ALL pixels matching RGB color. Returns array of coordinates.

screen.screen.find_color

screen.find_color(colors [, similarity, left, top, right, bottom])

Find position matching color pattern on screen.

screen.screen.find_color_rgb

screen.find_color_rgb(r, g, b [, x1, y1, x2, y2 [, tolerance]])

Find first pixel matching RGB color in region. NEON SIMD accelerated row scanning.

screen.screen.find_colors

screen.find_colors(colorTable [, x1, y1, x2, y2 [, tolerance]])

Find multi-point color pattern with offsets. First entry is anchor (dx=0,dy=0), rest are relative offsets.

screen.screen.find_image

screen.find_image(path_or_paths, similarity [, x1, y1, x2, y2 [, mode]])

Find template image on screen. Supports single or multi-template (first match wins). Auto mode: binary → gray → edge fal…

screen.screen.find_text

screen.find_text(text [, x1, y1, x2, y2 [, lang]])

Find text on screen and return center coordinates.

screen.screen.find_text_vision

screen.find_text_vision(text [, x1, y1, x2, y2])

Find text using Vision framework.

screen.screen.get_color

screen.get_color(x, y)

Get pixel color as hex integer.

screen.screen.get_color_rgb

screen.get_color_rgb(x, y)

Get pixel color as separate RGB values.

screen.screen.image

screen.image([left, top, right, bottom])

Capture screen as image object.

screen.screen.init

screen.init(orientation)

Set screen coordinate orientation.

screen.screen.init_home_on_bottom

screen.init_home_on_bottom()

Set orientation to home-on-bottom (portrait). Shortcut for screen.init(0).

screen.screen.init_home_on_left

screen.init_home_on_left()

Set orientation to home-on-left.

screen.screen.init_home_on_right

screen.init_home_on_right()

Set orientation to home-on-right (landscape).

screen.screen.init_home_on_top

screen.init_home_on_top()

Set orientation to home-on-top (upside-down portrait).

screen.screen.is_color

screen.is_color(x, y, r, g, b [, tolerance])

Check if pixel at (x,y) matches target RGB within tolerance.

screen.screen.is_colors

screen.is_colors(colors [, similarity])

Check if multiple points match expected colors.

screen.screen.keep

screen.keep()

Cache current screen for fast repeated color reads. ~50x get_color = 1x keep.

screen.screen.ocr_file

screen.ocr_file(path [, lang])

OCR an image file directly (bypasses screen capture).

screen.screen.ocr_region

screen.ocr_region(x1, y1, x2, y2 [, lang])

OCR text in screen region.

screen.screen.ocr_text

screen.ocr_text([lang])

OCR full screen text (Tesseract engine).

screen.screen.ocr_vision

screen.ocr_vision([x1, y1, x2, y2 [, level]])

OCR using Apple Vision framework. Better accuracy but slower.

screen.screen.orientation

screen.orientation()

Get current coordinate system orientation.

screen.screen.pixel

screen.pixel(x, y)

Get pixel color as R, G, B, A components (0-255 each). Alternative API with alpha channel.

screen.screen.rotate_xy

screen.rotate_xy(x, y, target_orientation)

Rotate coordinates from current orientation space to target orientation space.

screen.screen.save_region

screen.save_region(x1, y1, x2, y2, path)

Save a screen region to PNG file.

screen.screen.scale_factor

screen.scale_factor()

Get screen scale factor. Returns [UIScreen mainScreen].nativeScale. Cached.

screen.screen.size

screen.size()

Get screen resolution.

screen.screen.snap_region

screen.snap_region(x1, y1, x2, y2)

Take a snapshot of screen region into internal buffer for later comparison with screen.diff_percent().

screen.screen.snapshot

screen.snapshot(path)

Save full screenshot to file.

screen.screen.unkeep

screen.unkeep()

Release cached screen buffer.

screen.screen.unrotate_xy

screen.unrotate_xy(x, y)

Convert coordinates from current orientation to native (home-on-bottom) coordinates.

screen.screen.wait_change

screen.wait_change(x1, y1, x2, y2 [, threshold_pct, timeout_s, tolerance])

Wait for screen region to change by at least threshold percentage. Takes initial snapshot, polls every 100ms.

screen.screen.wait_color

screen.wait_color(x, y, r, g, b [, timeout, tolerance])

Wait until pixel matches color.

screen.screen.wait_colors

screen.wait_colors(colorTable [, timeout_s, tolerance])

Wait for ALL colors to match simultaneously. Polls every 100ms.

screen.screen.wait_image

screen.wait_image(path_or_paths, similarity, retry_count [, x1, y1, x2, y2])

Poll screen until template found or retries exhausted. Binary mode only.