CLI workflow

Discover, connect, observe and act with explicit execution facts.

Discover and connect

First complete device preparation. Replace device and session placeholders with the IDs actually returned. Disconnect after completing the task.

iphone-use devices
iphone-use connect --device "$IOS_UDID" --mode standard
export SESSION='SESSION_FROM_CONNECT'
iphone-use --session "$SESSION" apps --query Settings
iphone-use --session "$SESSION" open com.apple.Preferences
iphone-use --session "$SESSION" observe
iphone-use --session "$SESSION" disconnect

devices lists CoreDevice-known devices, not guaranteed ready sessions. apps --query searches installed names and bundle IDs. open verifies installation, then activates or launches the app; it does not install or restart it.

Read the observation

Standard mode returns the Apple PNG path, observation ID and screen geometry. Read the actual image. Coordinates are screen points, not image pixels: use the geometry from this observation to convert them. Runner can additionally expose captured accessibility elements such as o1:e10; inspect searches only that cached snapshot.

Use current observations

These are syntax examples. Replace every observation and coordinate using the screen you just read.

iphone-use --session "$SESSION" tap o1 --x 100 --y 200 --observe
iphone-use --session "$SESSION" swipe o2 --from-x 100 --from-y 420 --to-x 100 --to-y 240 --observe

Text input requires confirmed focus. Standard mode binds type to an observation and the complete field rectangle; Unicode uses clipboard-backed paste and may require iOS permission. Confirm the visible result before finishing the clipboard transaction with text-finish. Never replay unknown input. Runner uses element references and XCTest. Text and newline support differ by mode; use the detailed action contract and installed help.

--observe returns separate action and observation receipts. Standard mode waits within a bounded post-action stability budget. A valid screenshot may still carry a stability timeout; neither a stable image nor a successful command proves the business result.

Execution outcomes

completed: Input completed and the execution checks passed. Read the screen to establish the intended result.
not_dispatched: Input was not sent. Follow the error and requiresObservation to correct the request or refresh references.
unknown: Input may have happened. Observe and inspect the actual result before deciding the next action. Do not replay blindly.

Screen checks

Actions compare current screen information against their original observation before dispatch. If the scene changes, read the returned new image and choose again. Video exclusions and caret locations must be explicitly selected from the current screen; do not broaden tolerances to force input. Standard screenshots do not require a persistent media stream. Recording owns its media session; input still checks for competing clients. For native_media_busy, end the other client’s control session and follow recovery without closing foreign sessions.