Crate smix_screen

Expand description

§smix-screen

Accessibility tree types for iOS-Simulator automation — A11yNode, Rect, Bounds, Role, ElementSummary, ScreenDescription — with inlined visibility primitives that approach single-cycle latency on M-series machines.

Zero-allocation arithmetic, #[inline] on the hot path, criterion bench medians available below for reference. The visibility math is the same semantics maestro and Apple’s XCUITest enforce (any non-empty frame intersection with the viewport, zero-width / zero-height bounds short-circuit to false).

§Quickstart

use smix_screen::{
    A11yNode, Rect, Role, collect_visible_summaries, is_visible_enough,
    summarize_node, visible_area,
};

let node = A11yNode {
    raw_type: "button".into(),
    role: Some(Role::Button),
    identifier: Some("btn-login".into()),
    label: Some("Log in".into()),
    title: None, placeholder_value: None, value: None, text: None,
    bounds: Rect { x: 50.0, y: 100.0, w: 200.0, h: 40.0 },
    enabled: true, selected: false, has_focus: false, visible: true,
    children: vec![],
};
let viewport = Rect { x: 0.0, y: 0.0, w: 390.0, h: 844.0 };
let tree = A11yNode {
    raw_type: "application".into(),
    bounds: viewport,
    children: vec![node.clone()],
    ..node.clone()
};

assert!(is_visible_enough(&node, &tree));
assert_eq!(visible_area(&node, &tree), 200.0 * 40.0);

let summary = summarize_node(&node);
assert_eq!(summary.name.as_deref(), Some("Log in"));

let summaries = collect_visible_summaries(&tree, 100);
assert_eq!(summaries.len(), 2); // root + button

§Performance (criterion bench medians on M-series macOS)

Operation	TS V8 baseline	`smix-screen` Rust	Speedup
`is_visible_enough` (in-view)	21.2 ns	0.996 ns	21×
`is_visible_enough` (zero-bounds early reject)	20.5 ns	0.449 ns	46×
`is_visible_enough` (unknown-root conservative pass)	20.6 ns	0.584 ns	35×
`is_visible_enough` (offscreen)	20.7 ns	0.805 ns	26×
`visible_area` (intersection arithmetic)	20.9 ns	1.112 ns	19×
`dfs_collect` 100-node composite	788 ns	233 ns	3.4×

Numbers reproducible via cargo bench --bench visibility -p smix-screen.

§When to reach for this

Use case	Pick
Need typed `A11yNode` matching Apple’s XCUIElement attribute set	smix-screen
Need to filter “visible enough” candidates on a hot path	smix-screen
Want the same visibility semantics as maestro / XCUITest `.isHittable`	smix-screen
Need a full DOM/HTML accessibility tree (browser context)	use a browser-focused crate; this is iOS-shaped
Need physical device a11y tree	this is sim-shaped (camelCase wire fields match `xcrun simctl io` outputs)

§Wire compatibility

JSON serialization uses serde(rename_all = "camelCase") matching the iOS-Simulator runner conventions: rawType, placeholderValue, hasFocus. children: Vec<A11yNode> is recursive; terminal nodes may omit the field (serde default = "...").

The Role enum mirrors XCUIElement.ElementType literals (29 variants) with a stable camelCase wire name accessor (Role::Button.as_str() → "button").

§Scope

✅ Pure types + visibility primitives
✅ #[inline] + zero-allocation hot path
✅ camelCase JSON wire matches the iOS sim runner format
✅ ScreenDescription + collect_visible_summaries for DFS-pre-order visible-element projection
❌ No tree fetch (use smix-runner-client for the HTTP IPC)
❌ No selector parsing (use smix-selector)
❌ No resolver / spatial filter logic (use smix-selector-resolver)

§License

Dual-licensed under either:

at your option. smix-screen — A11yNode + Rect + Bounds + Role types + visibility primitives (stone). Ported from now-retired TS source: src/core/screen.ts + src/core/resolve-selector.ts:62-114 (visibility/area logic, v1.5 c5i-d semantics 1:1).

§Scope

Pure types (Rect, Bounds, Role, A11yNode) with serde wire compatibility (camelCase JSON, matching the existing Swift-side SmixRunnerCore /tree route shape).
Pure functions (is_visible_enough, visible_area) that the selector resolver consumes. No I/O, no node:* equivalent (跟 mailrs rfc5322 stone 同精神 — protocol parsing/types only).

§Visibility semantics (v1.5 c5i-d, 1:1 with TS)

b.w <= 0 || b.h <= 0 → invisible (zero-bounds early reject)
root.w <= 0 || root.h <= 0 → conservative pass (unknown root)
Otherwise → any non-empty rectangle intersection with tree.bounds

Matches swift TreeRoute.isVisible (any frame ∩ appFrame intersection) and maestro ViewHierarchy.kt:40-50 isVisible(node).

Structs§

A11yNode: Accessibility tree node — mirrors TS A11yNode in src/core/schemas.ts:124-139.
ElementSummary: Element summary — projected view of an A11yNode used in AI-readable failure prompts and driver.describe() output. Mirrors TS elementSummarySchema in src/core/schemas.ts:45-54 1:1.
Rect: Logical-points rectangle (origin top-left, +x right, +y down — matches UIKit / XCUITest coordinate space). All fields f64 because runner /tree route emits floating-point points (sub-pixel scale factors).
ScreenDescription: Aggregate screen description — Mirrors TS ScreenDescription in src/core/schemas.ts:80-88. elements is a DFS-collected ordered list of visible+enabled ElementSummary entries; screenshot is optional base64 PNG; frontApp / summary / captured_at are caller-populated metadata.

Enums§

Role: Accessibility role enum — mirrors TS roleSchema zod enum in src/core/schemas.ts:14-44 (29 variants).

Constants§

DEFAULT_VISIBLE_LIMIT: Default visible-summary limit (跟 TS DEFAULT_VISIBLE_LIMIT line 26).

Functions§

collect_visible_summaries: Collect up to limit visible+enabled nodes (DFS pre-order), project each via summarize_node. Mirrors TS collectVisibleSummaries in src/core/screen.ts:50-65. Default limit = 1000 (v1.5 c5i-a S3).
derive_roles_recursive: Recursively fill node.role from node.raw_type whenever it is currently None. Host-set roles (test fixtures, recorder output) are left untouched.
is_visible_enough: Visibility check. v1.5 c5i-d semantics, 1:1 with TS src/core/resolve-selector.ts:105-114.
role_from_raw_type: Derive the curated Role from the raw XCUIElement type name (e.g. "button", "radioButton", "progressIndicator").
summarize_node: Project an A11yNode to an ElementSummary (跟 TS summarizeNode in src/core/screen.ts:5-17 1:1).
visible_area: Intersection area in logical points². v1.5 c5i-f semantics, 1:1 with TS src/core/resolve-selector.ts:64-78.

Type Aliases§

Bounds: Bounds alias — TS used Bounds = Rect interchangeably. Kept as a thin re-export for port-clarity; downstream code may prefer one or the other.