Interested in a PR for iPhone Action Button support?

[whitneyland]

The idea is that agents could trigger the action button through agent-device like anything else. This would exercise the system’s shortcut/App Intent delivery, including when the app is backgrounded or terminated.

To make this happen, it would use Apple exposed XCUIDevice.shared.press(.action) and hasHardwareButton(.action).

If there's interest glad to propose the PR, or discuss another way to implement it. It should preserve app’s starting state rather than automatically foregrounding it before the press.

Great work on the project. Thanks for any feedback

[thymikee]

Picking this up — with two corrections from the XCUIDevice surface itself.

No long-press. XCUIDevice.press(_:) takes no forDuration: overload (unlike the XCUIRemote.press(_:forDuration:) that our tv-remote drives), so a hold on .action isn't expressible. A press|long-press verb plus --duration-ms would ship a flag that is silently ignored.

No slide on this button. The Action Button's documented input is "press and hold", full stop. The capacitive slide-your-finger surface is the Camera Control — a separate button, exposed as XCUIDevice.Button.camera — and XCUITest exposes no slide or light-press API for either. Camera Control stays out of scope.

So it takes the shape of our home command rather than tv-remote:

agent-device action-button

await client.command.actionButton();

hasHardwareButton(.action) becomes a per-device runtime fact checked at admission, so a device without the button gets a typed unsupported reason before any runner request is issued — and tvOS/macOS/watchOS are refused there too, not mid-execution.

Preserving the app's starting state is the bar we'll hold the PR to: no activate() before the press, and no settle or post-action capture on the command, since delivery to a backgrounded or terminated app is the point.

Issue and PR coming.