phonedriver Gives AI Coding Agents Control of Real Phones
A Rust-built MCP server and CLI lets Claude Code, Codex and Cursor read Android and iOS screens as text, tap by element number and pause before any payment button.
phonedriver is a new MCP server and command line tool that hands an AI agent the controls of a phone. The repository, hridaya423/phonedriver, was created on 2026-11-08 and reached 304 stars and 29 forks by 2026-11-20, according to the project's README. It is written mainly in Rust and licensed AGPL-3.0. The pitch is narrow and practical: an agent can see an Android phone or iPhone screen as text, tap a button, and read back what changed.
What the tool actually does
The README describes a standard MCP server over stdio, so Claude Code, Codex, Cursor, VS Code, Gemini CLI and other MCP clients can run it. Setup is npx -y phonedriver init, which registers the server with the agents it finds on the machine and then checks adb, Xcode and connected devices. A --dry-run flag shows what it would change, and phonedriver doctor re-checks the setup later. The first run downloads one binary for macOS, Linux or Windows and caches it. There are plugin bundles for Claude Code, Codex and Cursor, though the README warns to use either the plugin or the MCP setup, not both, or the tools are listed twice. The README lists 17 MCP tools, including snapshot, tap, type, swipe, batch and replay, with snapshot, screenshot, wait and logs marked read-only so a client can run them without a permission prompt.
How a screen becomes text
A screen arrives as a short numbered list. In the Settings app on an Android emulator, entries look like @e10 button "Display & touch", and the model taps by number. After a tap, the reply lists only what changed, with +, - and ~ marking lines that appeared, left or changed. The tool waits until the screen stops changing before it replies, so the model does not have to poll or guess how long a screen takes to load.
Underneath, Android uses a small Java helper started over adb that reads the accessibility tree through UiAutomation and streams H.264 video, which is how the tool knows when the screen settles. iOS uses an XCTest runner. A daemon on the computer keeps the connection open and remembers which element each number points to, so a tap still lands after a list has moved.
The guard before it pays
Before a tap lands on a button labelled Pay, Buy now, Buy with GPay, Place order, Start subscription, Transfer or Delete account, phonedriver holds it and asks in Claude Code's own prompt. The model cannot see that question or answer it. You choose Allow, Allow for this session or Decline. The label check covers English, Spanish, Portuguese, French, German, Italian, Dutch, Japanese, Chinese (Simplified), Korean and Hindi. The README says it does not catch a payment button that is only an icon, or one labelled in another language. PHONEDRIVER_ASK extends the check to send, delete and install actions, and PHONEDRIVER_GUARD=off disables it.
Benchmarks, replay and open questions
The README reports 6.9x fewer tokens than mobile-mcp across nine phone jobs, run twice each with Claude Sonnet 5.5 on the same Android emulator on 10 November 2026. On speed, measured 11 November 2026 with phonedriver 0.1.1 on an Android 17 emulator as a median of 50 runs, reading a screen on a HONOR X8 took 78.5 ms against 2,507 ms for adb, and a screenshot 16 ms against 1,006 ms. These are the author's own benchmarks, with method and raw data published, not independent results.
A session can be saved and replayed with no model involved, stopping at the first step that misses. A step that typed a secret is saved with an environment variable name rather than its text, so a login can replay in CI. The repository also works as a GitHub Action that boots an emulator or Simulator, installs your app and replays those saved files on every commit.
The open questions are real. The payment guard is label-based and admits gaps. The numbers come from the author's own test set. The speed run used version 0.1.1 while the changelog mentions 0.1.6, and the README does not say how much is tested on physical iPhones rather than the Simulator.
What this changes for people building apps
Agents that can drive a phone make a specific kind of testing possible: walking a real signup flow, changing a setting buried three screens deep, finding the step where an app stops working. That work is described in plain language, the same way you describe an app you want built. If you build with Draftship, where apps ship with built-in sign-in and a database, the phrasing is the part worth practising; writing effective prompts covers how to state a job so an agent can act on it. Draftship is not affiliated with phonedriver and does not include it.
Primary sources
- hridaya423/phonedriver: MCP server and CLI that lets AI agents (Claude Code, Codex, Cursor) see and control Android phones and iPhones. Reads the screen as text. Asks before it pays. · GitHub trending AI repositories, Nov 20, 2026
This article was drafted with AI assistance from the primary sources listed below and checked automatically against them. An editor hasn't reviewed it yet.