4.1K ·GitHub
What is this?
iPhone Use lets Codex control your real iPhone over USB using natural language: it can open apps, read screens, tap, scroll, type text, and organize lists, while showing a live screen preview in the sidebar.
iPhone Use lets Codex control your real iPhone over USB using natural language: it can open apps, read screens, tap, scroll, type text, and organize lists, while showing a live screen preview in the sidebar.Is this a standalone app or a Codex plugin?It is a Codex plugin bundle: a local Python MCP server that talks to the phone via WebDriverAgent (WDA), plus install/usage skills and a live screen widget - nothing extra is installed on the iPhone besides WDA itself.
It is a Codex plugin bundle: a local Python MCP server that talks to the phone via WebDriverAgent (WDA), plus install/usage skills and a live screen widget - nothing extra is installed on the iPhone besides WDA itself.How do I get it running?Copy the provided install prompt into Codex to register the plugin through a local marketplace, then in a new chat run the iphone-use-setup skill, which guides Codex through fetching the pinned WDA build, signing it in Xcode with your Apple account, deploying it to your iPhone and confirming readiness.
Copy the provided install prompt into Codex to register the plugin through a local marketplace, then in a new chat run the iphone-use-setup skill, which guides Codex through fetching the pinned WDA build, signing it in Xcode with your Apple account, deploying it to your iPhone and confirming readiness.What do I need installed before trying this?A Mac with Codex desktop/CLI, a full Xcode install with first-run setup done and your own Apple developer account (Command Line Tools alone are not enough), a USB-connected iPhone, Python 3.9+, and Node.js 20.19+/22.12+/24+ with npm 10+.
A Mac with Codex desktop/CLI, a full Xcode install with first-run setup done and your own Apple developer account (Command Line Tools alone are not enough), a USB-connected iPhone, Python 3.9+, and Node.js 20.19+/22.12+/24+ with npm 10+.What steps can't Codex do for me automatically?Apple account login, trusting the Mac, enabling developer mode, and unlocking the phone still have to be done by you on the device itself; the repo is also currently private, so you need GitHub access or an already-obtained source package.
Apple account login, trusting the Mac, enabling developer mode, and unlocking the phone still have to be done by you on the device itself; the repo is also currently private, so you need GitHub access or an already-obtained source package.How many tools does Codex get access to?Codex has access to 17 tools for things like tapping, typing, batch actions, scrolling list search and collection, plus 2 more that are used only by the screen widget.
Codex has access to 17 tools for things like tapping, typing, batch actions, scrolling list search and collection, plus 2 more that are used only by the screen widget.What happens when Codex can't find a button to tap?It falls back to a screenshot: pixel coordinates are converted to iPhone point coordinates via image.pixeltopoint and handed to the tap tool, instead of blindly retrying.
It falls back to a screenshot: pixel coordinates are converted to iPhone point coordinates via image.pixeltopoint and handed to the tap tool, instead of blindly retrying.Does the phone need to be jailbroken, or do I need a separate Appium Server?No to both - it works on a normal, unjailbroken iPhone through WebDriverAgent, and you don't need to run a separate Appium Server.
No to both - it works on a normal, unjailbroken iPhone through WebDriverAgent, and you don't need to run a separate Appium Server.Will it submit forms or type my passwords for me?No - typing and submitting text are kept separate and nothing is auto-submitted by default; passwords, verification codes and Face ID prompts are left for you to handle, and the preview can be paused while you take over.
No - typing and submitting text are kept separate and nothing is auto-submitted by default; passwords, verification codes and Face ID prompts are left for you to handle, and the preview can be paused while you take over.What are similar projects?mobile-mcp, mcp-chrome, coral, dsh-ios, headroom, rea, skills, vibe-aso
mobile-mcp, mcp-chrome, coral, dsh-ios, headroom, rea, skills, vibe-asoWhich topics is it in?gui-automation, usb, codex-cli, screenshot, xcode, model-context-protocol, testing, local-first, mcp-server, plugin, mobile, ios
gui-automation, usb, codex-cli, screenshot, xcode, model-context-protocol, testing, local-first, mcp-server, plugin, mobile, ios