pinky README

项目的 README,来自 github.com/DIYgod/pinky。合并为一个 markdown 文件:README.md。

Pinky

Let AI use your iPhone or Android phone, with you in the loop: any agent from macOS, Windows or Linux, or Pinky itself, right on the phone.

Download · Get started · FAQ · Discord

English · 简体中文 · 日本語 · Türkçe

Claude Code types a reply in Messages; Pinky holds the Send tap until it is allowed, then the text goes out
A real Claude Code run on an Android 16 emulator. Pinky asks before payments by default; this run also asks before messages.

Pinky lets any agent (Claude Code, Claude Desktop, Codex, Cursor or a script) operate a real iPhone or Android phone over MCP, while you watch on your computer, approve risky steps, take over and review what happened. Pinky can also do tasks by itself, with an AI model you choose: from your computer, or in Pinky's app right on the phone, with no computer at all. Whoever does the task, approvals are enforced by Pinky's gateway, not by instructions to the agent.

You decide what waits: Pinky's window holds an agent's Send tap until you click Allow
You approve the risky steps. Payments, trades and transfers wait for your click; in Settings › Safety you can have sends, orders, deletions, calls, installs, links that leave the app and settings changes wait too, or every step. Pinky spots these buttons in 10 languages. The rules run in Pinky's gateway, so an agent can't approve itself, even when text on the screen tries to talk it into it.
Off-limits by default: Settings › Safety with when agents ask you, and the blocked apps
Off-limits by default. Banking, wallet and password-manager apps are invisible to agents, and agents can't type into password or one-time-code fields. When an agent needs a banking or wallet app for a task, it asks, and you can allow that app for that task only. Every new agent needs your approval and gets only the scope you give it.
A hidden second screen for the agent: the agent's screen beside the phone's own home screen
Your phone stays yours. On Android 13 and later, agents work on a hidden second screen, so the phone's own screen stays yours, or off and locked. On older Android phones and on iPhones they use the main screen.
Take over any time: Claude Code is paused while you are in control, until you hand back
Take over any time. Click the mirror and the agent pauses until you hand back; on Android, touching the phone yourself pauses it too. Stop all is ⌥⌘. (Ctrl+Alt+. on Windows and Linux) from anywhere.
Every run, on the record: a finished task's steps beside its screen recording
Every run is on the record. Each step in plain words, screenshots and a screen recording with a mark per step, exportable as an HTML report. It all stays on your computer.
On iPhone too: Pinky holds a Delete tap in Photos until you click Allow
iPhones too. The same gateway, approvals and records work on iOS 17.4 and later (what works). Shown here on the iOS Simulator, with deletions set to ask.

Status: pre-alpha. Claude Code has used real phones through it end to end (an Android phone over USB, an iPhone on iOS 27, and an iPhone XS Max on iOS 18.7.11 through Pinky Helper, the app Pinky on the iPhone replaces), and agents can connect over stdio or HTTP. Pinky on the phone has done tasks with a real model on a Mi 10 Pro (Android 17) and an iPhone 13 mini (iOS 27.0.1). Unlocking from the computer, background work on Android, recordings, export, Wi-Fi pairing and the phone's files are built; some of it still waits for a person to try it. See docs/ACCEPTANCE.md for what works today and how to try it.

Get started

Install Pinky, connect your phone and add Pinky to your agent. Then ask the agent to do something on the phone, and watch it work in Pinky's window.

What you need

You need
Mac macOS 13 or later, with Apple silicon or Intel
Windows Windows 10 or 11, 64-bit (x64)
Linux A 64-bit PC (x86-64) with Ubuntu 20.04, Debian 11 or a newer system (glibc 2.29 or later), and GTK 3
Android phone Android 10 to 17 and a USB cable. Pinky brings its own adb; on Windows some phones need a USB driver, and on Linux USB access (below)
iPhone or iPad iOS 17.4 or later, and a USB cable to set it up; afterwards it works over Wi-Fi too (below). Pinky prepares it itself, with Apple's developer disk image, which comes with Xcode or which Pinky can download from a third-party mirror when you ask (below). On iOS 17.4 to 26 also Pinky on the iPhone, Pinky's app (below)
Simulators and emulators Optional, in place of a phone or next to one: an iOS Simulator on a Mac with Xcode, or an Android emulator from Android Studio. You create them there; Pinky starts them (below)
Agent Any agent that can use MCP servers, such as Claude Code, Claude Desktop, Codex or Cursor

1. Download and install Pinky

Get Pinky from the latest release:

Pinky looks for new versions by itself and, on a Mac, with the Windows installer and unpacked from the Linux .tar.gz, installs them too (Updates).

2. Connect your phone

Open Pinky and choose Add phone. A phone that is ready shows up in Pinky's window; when something is missing, the phone's page in Pinky says what.

Android

Add phone › Android shows these steps for your phone's brand, with its own menu names and extra switches: Xiaomi, OPPO, vivo, HONOR, HUAWEI, Samsung and Pixel, and general steps for the rest.

  1. On the phone, turn on USB debugging: in Settings › About phone, tap Build number 7 times, then turn on USB debugging in Settings › System › Developer options. Some brands name these menus a little differently.
  2. Plug the phone into the computer. When the phone asks whether to allow USB debugging, tap Allow. On Android 11 and later you can pair it over Wi-Fi instead, from Add phone (when it fails). Android 10 has no Wireless debugging: once it is plugged in, Use Wi-Fi instead… moves it to Wi-Fi (below).
  3. Pinky offers to install its app for the phone, Pinky on the phone (it shows there as Pinky), which makes reading the screen many times faster and lets Pinky type in any language. The phone may ask you to allow the install once.

Each phone has a Checkup, in the ··· menu under its mirror, that lists what works and how to fix what doesn't, such as a Xiaomi's USB debugging (Security settings) switch.

Over Wi-Fi, Pinky makes sure the phone that answers is yours: it leaves a mark on each phone it sets up, which only the phone's debugging connection can read, and checks it whenever adb connects the phone, so another device on the network can't answer in its place. A phone connected over Wireless debugging that Pinky hasn't used, by another tool for example, shows as found on Wi-Fi, and Pinky leaves it alone until you choose Use. If a phone's page says This may not be … after you reset the phone or set it up again yourself, choose Trust this phone… there, or plug it in once.

Taking the phone out? With Tailscale, Pinky reaches it away from home too (how).

Add phone › Android with a Xiaomi picked: where to turn on Developer options, and the three USB debugging switches
Add phone › Android, with a Xiaomi picked.
A Redmi's checkup: adb, the connection and debugging pass; Pinky on the phone and tapping need fixing, with the steps
A Redmi's Checkup, with two things to fix.

Android tools

Pinky brings its own adb, the tool it reaches Android phones with, built from Android's source, so nothing else needs installing. When the computer has Android's platform tools already, from Android Studio, Homebrew or the system's packages, Pinky uses those, and shares their adb server with Android Studio and other tools. When Pinky quits, it stops the adb it brought, unless another app such as Android Studio is using it; Settings › Advanced › Stop adb when Pinky quits turns that off. An adb you installed keeps running.

USB on Windows and Linux

When pairing over Wi-Fi fails

When a phone works on the cable but Pair over Wi-Fi does not finish, the pairing page says within half a minute what Pinky sees:

When pairing stops with an error, the page says what adb reported:

Pinky writes adb's own words to its log, which Help › Send Feedback… attaches with the diagnostics.

Pairing over Wi-Fi needs Android 11 or later; an Android 10 phone moves to Wi-Fi from the cable instead (Use Wi-Fi instead…, below). Up to Android 16, Wireless debugging turns itself off when the phone leaves the Wi-Fi network, even for another access point of it, or restarts: turn it on again, or pair again from Add phone.

iPhone

Pinky prepares an iPhone itself; Add phone › iPhone walks you through it.

  1. Plug the iPhone into the computer, unlock it, tap Trust and enter its passcode. On Windows, install Apple Devices from the Microsoft Store (or iTunes) first; on Linux, the usbmuxd package. Add phone says when they are missing.
  2. On the iPhone, turn on Developer Mode in Settings › Privacy & Security: Pinky makes the switch appear there. The iPhone restarts; unlock it, tap Turn On and enter the passcode.
  3. Pinky mounts Apple's developer disk image on the iPhone and lists it as ready. An iPhone on iOS 27 keeps the image when it restarts; below iOS 27 Pinky mounts it again after each restart, once you unlock the iPhone.

The developer disk image comes with Xcode, Apple's free app for developers, and Pinky does not ship it. On a Mac, install Xcode once and open it: Pinky uses the image it installs, and Xcode never has to run again. An iPhone on iOS 27 or later needs the image of Xcode 27 or later: Apple's signing server no longer signs older ones, and Add phone then says which image Pinky used and how to get a newer one. On Windows and Linux, copy /Library/Developer/DeveloperDiskImages/iOS_DDI from such a Mac into Pinky's folder (where). Without either, or when Apple refuses the image Pinky found (Xcode's, or one copied into Pinky's folder), Add phone offers to download the image from a third-party mirror, github.com/doronz88/DeveloperDiskImage (Xcode 27.0 beta 4, 32 MB, kept by the author of pymobiledevice3, not by Apple), only when you click Download; Pinky checks every file against the checksum it expects, and from then on uses it instead of an image Apple refused. On iOS 27 and later, no app gets installed on the iPhone.

Checked on the iPhone 13 mini (iOS 27.0.1): Pinky mounting Xcode's image itself (in about 1.5 s, after the image had been taken off), keeping the image already on the phone, and that the iPhone keeps the image when it restarts. Not yet tried on a phone: the downloaded image, the Developer Mode switch, iPhones below iOS 27, and pairing an iPhone on Windows. Contributors have used an iPhone and an iPad on iOS 27.2 on Linux (Arch Linux), and an iPhone on iOS 27.2 on Windows 11, on the cable and over Wi-Fi, with the image already on them.

iPhones and iPads over Wi-Fi

Pinky also uses an iPhone or iPad over Wi-Fi while it is on the same Wi-Fi as the computer, and the computer lists it there, which it does once Wi-Fi connections are on for the phone. Pinky turns them on when you plug the phone in and it trusts the computer: it is the same setting as Finder's Show this iPhone when on Wi-Fi, so a phone already set up that way stays as it is. It is on by default; Settings › Phones › Use iPhones and iPads over Wi-Fi turns it off, which leaves the phones' setting alone. If a phone doesn't show up on Wi-Fi when you unplug it, its Checkup says where it stands under Wi-Fi: whether its Wi-Fi connection is on and whether Pinky turned it on, why Pinky couldn't, and whether this computer sees the phone on Wi-Fi.

Over Wi-Fi each step is slower than on the cable, screenshots most of all. Measured on an iPad Pro 11-inch (3rd generation, iPadOS 27.0.1), Wi-Fi against the cable: a swipe shows on screen after 80 to 210 ms (60 to 90 ms), the home screen reads in 0.6 to 2.6 s (0.5 to 0.6 s), a screenshot takes 5 s (1 s), and 0.2 % of the video's packets were lost (none). Unplugging moved a running task to Wi-Fi in about 5 s, and plugging in moved it back in 2 to 6 s (details).

On a Mac, the system's device service, usbmuxd, can lose an iPhone on Wi-Fi after its connection broke once (its Wi-Fi turned off and on, say), and not look for it again for up to about an hour; Finder loses it too. Pinky notices when a phone you use is on the network but no longer listed, shows it as Lost on Wi-Fi, and offers Restart device service… on its page: after you confirm, macOS asks for an administrator's password, and every iPhone and iPad, also on a cable, disconnects for a second or two. sudo killall usbmuxd in Terminal does the same. Right after you unplug a phone it says so within about 10 seconds; it waits a minute otherwise, as a phone that wakes comes back by itself. A phone you unplug that this computer doesn't see on Wi-Fi at all shows as Not seen on Wi-Fi, with what may hide it: it's asleep, on another network or one that keeps devices apart (guest networks, some routers), a VPN, or Pinky not allowed on the local network.

On Linux and Windows, the system's device service does not list iPhones and iPads on Wi-Fi by itself: on Linux (the usbmuxd package) it lists them only on the cable, and on Windows (Apple Devices or iTunes) only while they are on the cable too. So there Pinky runs a small device service of its own in front of the system's, while it uses phones over Wi-Fi: it finds the phones paired with this computer on the network and connects to them directly, and passes everything about phones on a cable to the system's. All of Pinky's connections to iPhones and iPads go through it then, on the cable too. There is nothing to install or set up for it beyond the system's own (usbmuxd on Linux, Apple Devices on Windows). If you run a device service that lists phones on Wi-Fi itself, such as netmuxd, and start Pinky with USBMUXD_SOCKET_ADDRESS pointing at it (unix:///path/to/socket or tcp://127.0.0.1:port), Pinky uses that one instead and runs none of its own.

While Pinky opens a phone, or moves it between the cable and Wi-Fi, it shows Connecting or Moving to Wi-Fi rather than offline. Over Wi-Fi Pinky reads text from the live picture rather than from screenshots, which take about 5 s there. An iPhone or iPad without a passcode never reads as locked, so after it has dropped off the network to sleep, Pinky connects to it only once it has been back for 30 seconds.

iPhones on iOS 17.4 to 26

Below iOS 27, an iPhone takes taps and typing from a computer only through an app on the phone that hosts WebDriverAgent: Pinky on the iPhone, Pinky's own app, which you install once from a Mac with Xcode, signed with your Apple ID: plug the iPhone in and click Install Pinky on the iPhone on its page in Pinky (or in Add phone › Try an older iPhone), or run pinkyctl helper install in Terminal. Either builds it, signs it and installs it, and Pinky's window shows each step and asks you to unlock the phone when Xcode waits for it (how). While Pinky on the computer uses the phone, the app says so on the iPhone. Pinky Helper, the app Pinky installed for this before November 2026, keeps working, and Pinky still installs it when Pinky on the iPhone can't be built (pinkyctl helper install --helper-only asks for it). With a free Apple ID it has to be installed again every 7 days: two days before, the phone's page and its checkup remind you, with Install again, and once it has run out the phone shows as needing setup and says why (how to install it again). Pinky Helper has been tried on an iPhone XS Max on iOS 18.7.11, on Windows, over its cable and Wi-Fi; Pinky on the iPhone in its place has been installed over Pinky Helper on an iPhone 13 mini on iOS 27.0.1, where the computer started and stopped its testing service; not yet on iOS 17.4 to 26. Without it, Pinky can only show and read the screen (on a Mac) and open apps: no taps, swipes or typing. iOS 17.3 and earlier are not supported.

3. Add Pinky to your agent

In Pinky, open Settings › Agents and click Connect next to your agent. Pinky finds Claude Code, Claude Desktop, Codex, Cursor, VS Code, Copilot CLI, Windsurf (Devin Desktop) and Gemini CLI on your computer and adds itself to the agent's MCP servers, by the app's own path: with the agent's own command where it has one, such as claude mcp add, or in its settings file, which Pinky copies first and changes only in Pinky's entry. It also installs Pinky's agent guide, SKILL.md, where the agent reads skills, and keeps it current: when Pinky updates, it rewrites the guide it installed to the new version's. Until an agent is connected, a card in Pinky's sidebar offers to do it, or says where to get one.

Settings › Agents › Connect an agent: Claude Code connected; Claude Desktop, Codex, Cursor and VS Code with Connect buttons

The command is pinkyctl, not pinky: many computers already have another program named pinky. An agent on another computer or in a container can connect over HTTP instead (how). docs/mcp.md says what Connect changes for each agent.

4. Try it

Ask the agent to do something on your phone, for example "Open Messages on my phone and tell Sam I'm running late." The first time an agent connects, Pinky asks whether to allow it and what it may do. Payments wait for your click in Pinky's window; in Settings › Safety you can have sends, deletions and more wait too.

No phone at hand? Start an iOS Simulator or an Android emulator in Add phone › Virtual phone (below). Or quit Pinky and start it with two scripted demo phones (a payment to approve, a blocked banking app, a password field): /Applications/Pinky.app/Contents/MacOS/Pinky --fake on a Mac, pinkyctl --fake on Linux, or & "$env:LOCALAPPDATA\Programs\Pinky\Pinky.exe" --fake in PowerShell on Windows.

Using Pinky

Phones

Phones away from home

An Android phone does not have to be on the computer's network. With Tailscale on the computer and on the phone, signed in to the same account, Pinky reaches the phone through it: leave the computer at home and take the phone out, on mobile data or another Wi-Fi. Android runs one VPN at a time, so keep other VPN and proxy apps off on the phone: they turn Tailscale off.

  1. Set the phone up once while it is near the computer, by cable or on the same Wi-Fi: Settings › Phones › Use away from home › Turn on…. Add phone › Android › Phone not on this network? Use Tailscale walks you through Tailscale on both sides and lists the phones on your tailnet.
  2. Pinky has the phone's adb listen on the network (adb tcpip, port 5555) and connects to the phone at its Tailscale address. It uses that connection whenever the phone has no connection on the computer's network, so the phone stays in the list as you leave.
  3. The phone's adb then takes connections on every network the phone joins, from the computers it already trusts; any other computer makes the phone ask. Only the way through Tailscale is encrypted. It lasts until the phone restarts, and Turn off stops it at once. Pinky only reads Tailscale's status and never changes its settings.
  4. Keep this computer awake for phones away (Settings › Phones, on by default) stops the computer's idle sleep while a phone is connected through Tailscale. A MacBook still sleeps when its lid closes.

At home, Tailscale is as fast as Wi-Fi; away, the phone's upload carries the picture, so Pinky streams lighter through Tailscale (1 Mbit/s), and on a weak 4G hotspot each step takes seconds. Details, measurements, data use and what to do when it does not connect: docs/away-from-home.md. An agent can be away from the computer too: Remote agents.

Ask Pinky

No agent of your own? Pinky can do a task on one phone by itself. In Settings › Agents, under Ask Pinky, choose a service and add its API key. Ready-made settings cover DeepSeek, Qwen, Kimi, Zhipu GLM, SiliconFlow, Volcengine Ark, OpenRouter, Anthropic, OpenAI, Gemini, Ollama and LM Studio, and any other service compatible with Anthropic's or OpenAI's API works with its address and the model's name. Saving checks the model at once and says clearly what went wrong. The key is kept in the Mac's Keychain, Windows Credential Manager or your keyring, like phones' PINs, and goes only to that service. Usage shows how many tokens Pinky's own agent used today and in the last 7 and 30 days, by service and model, with the cost where the price is known; it stays on your computer.

Then type the task at the top of Tasks on the phone's page, for example "Set an alarm for 7:00", and press Start (or ⌘↩, Ctrl+Enter on Windows and Linux); on All phones, pick the phone first. About the screen starts a task about what the phone shows now. Pinky works step by step as the agent Pinky, through the same way in as any agent: your rules, approvals, blocked apps, Take over, Stop all and recordings apply to it, and its tasks show in Activity. It never types passwords or one-time codes (unless you allowed that on a virtual phone and the task gives them); it hands the phone to you instead. Stop ends the task at once. While it works, you can send it a message from the box, and edit or withdraw it until Pinky reads it; once the task has ended, Continue carries it on from a new message.

A task stops at 100 steps, 20 minutes or US$2, whichever comes first; you can change these in Settings › Agents, and time spent waiting for you doesn't count. Cost is counted at Anthropic's prices; for a model Pinky has no prices for, a million tokens count as US$1. Ask Pinky works on one phone at a time: for anything bigger, connect an agent (details).

Memory: after a task, Pinky may suggest a note for later tasks, such as a preference you told it or a quicker way through an app, and keeps it only if you choose Remember. See, edit or delete the notes, or turn memory off, in Settings › Agents › Memory. Notes never hold passwords, codes, card, account, ID or phone numbers, or addresses, and each device keeps its own. Pinky also knows its way around WeChat, Alipay, Meituan, Taobao, Xiaohongshu, Amap, Google Maps, Apple Maps, Clock, Calculator and Settings: it reads a short guide when it opens one of them (details).

Scheduled tasks: New schedule… under Scheduled on the phone's page runs a task once, daily, on weekdays, weekly or every few hours, or only tells you when a condition holds. Scheduled tasks run on the computer and on Android (not on iPhone). While nobody watches, they never pay, ask for approvals by notification, and skip, with a note, when the phone is locked, in use or off.

Your agent can hand Ask Pinky a whole task too, instead of driving the phone step by step: phone_task_run starts it with the model and limits you set here, phone_task_wait, phone_task_status and phone_task_list follow it, phone_task_steer corrects it or carries it on, and phone_task_cancel stops it. Such a task works as that agent, not as Pinky: its first connection asks you, its access in Settings › Agents and your rules apply, and only you answer its approvals. The box at the top of Tasks shows it, with the agent's name: you can write to it and Continue it there as your own, and it still works as that agent; Stop ends it, and then the agent cannot carry it on. It works while Pinky's window is open, not in pinky daemon (details). Scripts can do the same from a shell: pinkyctl task run "Set an alarm for 7:00" waits for the task and prints how it went, as an agent named "Pinky CLI" (or -as "Nightly tests"), and pinkyctl task status|steer|cancel|list follow it (details).

Pinky on the phone

Pinky's app on the phone, named Pinky there, can also do a simple task on that phone by itself, without a computer. It works like Ask Pinky, with an AI model and API key you choose: you type the task in Pinky on the phone and tap Start, and Pinky leaves its own screen and works in the other apps while you watch. Pinky's rules apply there too: it asks you before risky steps (Settings › When should Pinky ask you?), hands the phone to you for passwords and codes, stops at its limits and when you tap Stop, and keeps the tasks in its Activity. The key stays on the phone and goes only to that service. Pinky on the phone offers the same model services, the check on saving and the usage as Pinky on the computer. You can speak a task instead of typing it, message Pinky while it works and Continue a task that ended, and it remembers notes you keep, in its own Settings › Memory. While Pinky on a computer uses the phone, the app says so, and you can start a task there once it is done. Desktop Pinky says this once on the phone's page, and in Settings › Phones under Tasks on the phone, with how to set it up.

Updates

Pinky looks for a new version at launch and every few hours, and shows a card with what changed, Update now and Later. With Settings › Advanced › Update automatically (on by default) it installs it by itself once no agent is working and nobody has used Pinky for ten minutes, and it updates Pinky's app on phones that are free; its accessibility service stays off until the next task. A copy of Pinky on the phone signed with the public key that its builds had until 8 November 2026 is uninstalled and installed again, signed with Pinky's private key, and Pinky tells you; on a phone that asks before installing apps over USB, the phone's page says why and waits for your Update, and when a phone refuses them, a card in the sidebar tells you once about each new version, to install by hand (how). After an update, What's new lists the changes since the version you had (from CHANGELOG.md, or its Chinese version, CHANGELOG.zh-CN.md, when Pinky is in Chinese).

Details: docs/notes/updates.md.

Usage statistics, crash reports and feedback

Pinky shares anonymous usage statistics (counters such as tasks run, tools called and phones connected, about once a day, with a random install id) and crash reports; turn both off in Settings › Privacy or with DO_NOT_TRACK=1. Screens, typed text, app names, phone names and ids, PINs and file names are never sent. Help › Send Feedback… opens a GitHub issue with your message and diagnostics filled in, which you read and submit yourself. docs/TELEMETRY.md lists everything.

FAQ

Which agents work with Pinky?

Any agent that can use MCP servers: over stdio, by running pinkyctl mcp, or over HTTP with an agent token (how). Settings › Agents connects Claude Code, Claude Desktop, Codex, Cursor and other agents it finds on your computer in one click. Claude Code is the one that has used real phones end to end so far. Your agent does the thinking; for simple tasks without one, Ask Pinky uses a model with your own API key.

Where does Pinky keep its files, and can I move them?

Pinky keeps its settings, history and the socket agents reach it through in your user folders: ~/Library/Application Support/Pinky on a Mac; %APPDATA%\Pinky and %LOCALAPPDATA%\Pinky on Windows; ~/.config/pinky, ~/.local/share/pinky and $XDG_RUNTIME_DIR/pinky on Linux. To keep all of them in one folder of your choice instead, set the environment variable PINKY_HOME to it: Pinky then uses config, data and run inside it. Set it where both Pinky and your agents see it, such as a user environment variable on Windows, or also in the agent's MCP settings ("env": { "PINKY_HOME": "…" }): a Pinky started without it uses the usual folders and doesn't see the other. On macOS and Linux the socket's path has to stay short, about 100 characters, and Pinky says so when PINKY_HOME makes it too long.

On Windows, agents reach Pinky through a named pipe, which works from packaged (MSIX) apps such as the Claude and Codex desktop apps too. Older versions of Pinky listened on a socket file in AppData, which those apps can't see; update Pinky if an agent in one of them can't reach it.

Can an agent approve its own risky step, or be talked into it?

No. The rules run in Pinky's gateway, between the agent and the phone. A step that needs you comes back to the agent as approval_pending, and nothing happens on the phone until you click Allow in Pinky; the agent has nothing it could send to confirm it. Before acting, Pinky checks that the button is still where it was, and no answer within 60 seconds means no. Pinky finds these buttons by their labels in 10 languages, so an unusual one could slip through: for more control, Settings › Safety has Ask before every action and Read-only mode.

What can't agents see?

Banking, wallet and password-manager apps, from a list that comes with Pinky and that you can add to in Settings › Safety, where you pick the app from those on your phones: agents get no screen from them, and their notifications are hidden, in the notification shade and on the lock screen too. Where another screen shows a blocked app, as its notification in the shade or in a banner, or its card in recent apps, the agent reads only that something is hidden there, and the screenshots it gets, and those in Activity, cover it with a gray box (screen recordings show it as it was, so they stay on your computer: an agent gets one only when you allow it at an export). Agents can't type into password or one-time-code fields. To test sign-ins with test accounts on an emulator or a Simulator, you can let them there: Settings › Safety › Allow passwords on virtual phones, off by default; on phones they never can. When an agent needs a banking or wallet app for a task, Pinky asks you first. If you allow it, the agent may open and read that app for the rest of the task, what your rules ask about there still asks, and Pinky keeps no screenshots or recordings of it: to read text that only its pixels show, Pinky looks at its screen in memory and gives the agent the text, never the picture. Password managers are never offered.

One-time codes are masked before an agent reads them: in a text message, an email, a notification or anywhere else on the screen, "Your code is 482913" reaches the agent as "Your code is ••••••", and the screenshots it gets, and those in Activity, have the code covered with a gray box. Prices, phone numbers, dates and numbers in text that is not about a code stay as they are. Screen recordings show codes as they were, so they stay on your computer: an agent gets one only when you allow it, each time it exports a task's report. You can turn this off in Settings › Safety › Hide codes from agents.

Does my phone's screen go to the cloud?

Not through Pinky. Screenshots, recordings and history stay on your computer, for 30 days unless you change it in Settings › Privacy, where you can also delete them. What the agent reads, the screenshots and reports it saves, and a screen recording when you allow it at an export go to the agent you chose, as with any tool you give it. Pinky's anonymous usage statistics never include screens, typed text or app names (above).

What does Pinky install on my phone?

On Android, Pinky's app, Pinky on the phone, when you click Install; it shows on the phone as Pinky (Pinky Companion before November 2026). Pinky keeps it up to date and turns its accessibility service on only while an agent works (why). On an iPhone, no app on iOS 27 and later: Pinky mounts Apple's developer disk image, as Xcode does. On iOS 17.4 to 26, Pinky on the iPhone, Pinky's app with WebDriverAgent in it, which you install once from a Mac with Xcode (Install Pinky on the iPhone in Pinky, or pinkyctl helper install, how) and which shows on the phone as Pinky (Pinky Helper before November 2026).

How do I install Pinky on the phone by hand?

When a phone won't let the computer install apps over USB, such as a Xiaomi without a SIM card, which can't turn on Install via USB, you can install Pinky on the phone yourself. Its APK, pinky-android.apk, comes with Pinky:

  • macOS: inside the app, at Pinky.app/Contents/Resources/pinky-android.apk: in Applications, right-click Pinky and choose Show Package Contents.
  • Windows: next to Pinky.exe in Pinky's folder.
  • Linux: next to pinky in the folder you unpacked the .tar.gz into, or in /opt/pinky/ if you installed the .deb.

When an install over USB fails, Show pinky-android.apk on the phone's page shows you the file. Copy it to the phone, over the cable or through a chat app for example, and open it there; the phone may ask you first to let the app you open it with install apps. Pinky recognizes the copy you installed, as it is the same app with the same signature, and stops offering to install it.

On a Xiaomi, Redmi or POCO phone, Pinky still needs USB debugging (Security settings) in Developer options to tap and type, and to turn on the accessibility service of Pinky on the phone; restart the phone after you turn it on. Like Install via USB, that switch checks with Xiaomi online and may ask for a SIM card. Without it, Pinky on the phone installs but can't come on, and Pinky can't tap or type on the phone.

Pinky on the phone gets a new version only when its own code changes, not with every Pinky update. With Update automatically on, Pinky tries once to install a new version over USB; when the phone refuses, a card in the sidebar tells you, with Show pinky-android.apk, and Pinky doesn't try that version again. Install the new pinky-android.apk the same way. With Update automatically off, the phone's page offers the new version when you open it.

Will banking or other apps notice Pinky on the phone?

While its accessibility service is on, they can. Malware abuses accessibility services, so many banking apps refuse to sign in while an app from outside an app store has one turned on: in Malaysia (Maybank, CIMB, HSBC and the other members of the Association of Banks in Malaysia, since 2024) and in Singapore (DBS, OCBC and UOB, since 2023) among others. Pinky on the phone is such an app, as Pinky installs it over USB. Apps such as WeChat, Douyin and Xiaohongshu can see the service too and watch for automated use; we know of no report that they flagged Pinky on the phone, but letting an agent use them may be against their terms.

Apps can also see that USB debugging or Wireless debugging is on, which Pinky needs on every Android phone; some apps, banking apps among them, warn you or refuse to run while it is. It stays on for as long as you use the phone with Pinky.

So Pinky turns the service on only while an agent works on the phone: as a task starts, and off again a few seconds after the task ends or you take over the phone, or a minute after the agent's last step. The rest of the time it is off, and those apps work as usual. While an agent works, even on the phone's background display, they see it: to use one of them then, click Take over in Pinky first, or end the task.

To keep Pinky from turning the service on at all, uninstall Pinky from the phone (Android's Settings › Apps › Pinky). Pinky works without it, only more slowly: it reads a screen in about 2 s instead of 70 ms, and types text other than plain English through the phone's clipboard. Not now on the phone's page puts away Pinky's offer to install it again.

How is Pinky different from other phone tools for agents?

We looked at more than a dozen before building it. None of the open-source ones covers iPhone and Android, runs on macOS, Windows and Linux, works with any agent and keeps a person in the loop, all at once. Most leave it to the agent to ask first, or block whole kinds of actions; Pinky holds the one risky tap itself, outside the agent, and records every run.

Is it safe to give my phone to an agent that can run commands?

An agent that can run commands as you could reach the phone without Pinky, through adb for example, and skip its rules. Run such agents in a sandbox without access to the phone, or plug the phone into another computer and let the agent connect to Pinky there over HTTP (how).

Build from source

To use Pinky, install a release (Get started). Building it yourself, to work on it, needs Go 1.27.3 or later, and on a Mac Xcode's command line tools.

Pinky is one Go binary without cgo; MyGo (go tool mygo build) packages it for each system. The live mirror and recordings use FFmpeg 9's shared libraries, which Pinky loads at run time: from PINKY_FFMPEG_DIR, from the app (macOS), next to the executable, then from where the system keeps libraries. Every release carries its own minimal LGPL build (scripts/ffmpeg.sh, the H.264 and HEVC decoders): in Pinky.app, and next to the executable in the Windows installer and zip and in the Linux .tar.gz and .deb (docs/RELEASING.md). A build of your own needs one too: make app adds it to Pinky.app, and scripts/release.sh build makes every package with it. Without FFmpeg everything but the mirror works, and Pinky says what is missing.

macOS

make companion   # optional: the APK of Pinky on the phone (Android SDK and JDK)
make app         # build/pinky/darwin-<arch>/Pinky.app and pinky-<version>-<arch>.dmg

make app builds a minimal LGPL FFmpeg 9 from source the first time (make ffmpeg, about a minute: the H.264 and HEVC decoders with VideoToolbox), puts it in Pinky.app/Contents/Frameworks, with CMake installed (brew install cmake) also builds Pinky's own adb (make adb) into Contents/Helpers, adds the APK of Pinky on the phone when it is built (COMPANION_APK=path picks another), signs the app and makes the .dmg from the finished app. Install from the .dmg by dragging Pinky to Applications. The app is signed ad hoc unless macos.signingIdentity in cmd/pinky/mygo.json (or make app MACOS_SIGN=…) names a Developer ID; an ad-hoc signed app runs on the Mac that built it, and another Mac's Gatekeeper refuses it. make app does not notarize. make release signs with the Developer ID in RELEASE_SIGN and notarizes the .dmg (make notarize: submit, wait, staple), with the notarytool profile in NOTARY_PROFILE (pinky-notary), which you store once with xcrun notarytool store-credentials pinky-notary --apple-id <Apple ID> --team-id <team>. Releases themselves are built, signed and published by a workflow for both kinds of Mac, Windows and Linux; see docs/RELEASING.md.

To try your build, open it (open build/pinky/darwin-arm64/Pinky.app) and connect your agent to it with Connect in its Settings › Agents, which uses this build's path, or for Claude Code from the checkout:

claude mcp add --scope user pinky -- "$PWD/build/pinky/darwin-arm64/Pinky.app/Contents/MacOS/Pinky" mcp
mkdir -p ~/.claude/skills/pinky && cp docs/SKILL.md ~/.claude/skills/pinky/SKILL.md

go run ./cmd/pinky does not see the app's FFmpeg: install FFmpeg 9 with Homebrew (brew install ffmpeg) or point at the build, PINKY_FFMPEG_DIR=build/ffmpeg/lib go run ./cmd/pinky.

Unlocking a phone from the computer keeps its PIN in the login keychain and asks for Touch ID or the Mac's password each time.

Windows

go tool mygo build -platform windows/amd64 cmd/pinky   # build/pinky/windows-amd64/Pinky.exe (and an installer where NSIS is installed)
go build -o pinky.exe ./cmd/pinky                      # a console build, for pinky devices and other commands in a terminal

Pinky.exe is a windowed program: agents that start Pinky.exe mcp talk to it over pipes, but a terminal shows nothing it prints, so use the console build there. A release's installer and zip carry FFmpeg next to Pinky.exe (with D3D11VA). For a build of your own, scripts/ffmpeg.sh build/ffmpeg-windows windows-amd64 makes the DLLs in build/ffmpeg-windows/bin (in a Docker container); put them next to the executable, or set PINKY_FFMPEG_DIR to their folder (the ffmpeg-n9.0-…-win64-lgpl-shared builds at FFmpeg-Builds work too). A build of your own carries no adb, so Android phones need the one of Android's platform tools; scripts/adb.sh build/adb-windows windows-amd64 builds Pinky's own (in a Docker container), whose adb.exe and two DLLs go next to the executable. iPhones are untested on Windows. PINs are kept in the Credential Manager, and each unlock asks through Windows Hello, or for the account's password where Hello is not set up.

Linux

go tool mygo build -platform linux/amd64 cmd/pinky   # build/pinky/linux-amd64/: pinky, a .deb, a .tar.gz with install.sh, a PKGBUILD

The window needs GTK 3 and EGL (libgtk-3-0, libegl1; Mesa draws it in software where there is no GPU). A release's .tar.gz and .deb carry FFmpeg next to pinky; it needs glibc 2.29 or later (Ubuntu 20.04, Debian 11 and newer) and decodes in software. For a build of your own, scripts/ffmpeg.sh build/ffmpeg-linux linux-amd64 makes the libraries in build/ffmpeg-linux/lib (in a Docker container); put them next to the executable, or set PINKY_FFMPEG_DIR to their folder. For hardware decoding (VA-API), point PINKY_FFMPEG_DIR at an FFmpeg 9 that has it, such as the distribution's or FFmpeg-Builds' shared builds (which need libva2 and libva-drm2); Pinky looks there first. iPhones are untested on Linux. PINs are kept in the Secret Service (GNOME Keyring, KWallet, KeePassXC) in a keyring of Pinky's own, which the first save asks you to give a password and which Pinky locks again after each use, so every unlock asks for that password.

Build and test

make check          # build (CGO_ENABLED=0), vet, test
make app            # Pinky.app with FFmpeg and Pinky on the phone, and a .dmg (macOS)
make ffmpeg         # FFmpeg 9 for Pinky.app, in build/ffmpeg
make adb            # Pinky's own adb for Pinky.app, in build/adb (needs CMake)
make companion      # the APK of Pinky on the phone (Android SDK and JDK needed)
make e2e            # a binary with MyGo's automation socket, for UI checks
scripts/smoke.sh    # the fake gateway, pinky devices and a task through pinky mcp
make dist           # a dry run of a release, in dist/<version>/ (docs/RELEASING.md)

go run ./cmd/pinky --fake runs the app with two scripted phones (a payment to approve, a blocked banking app, a password field), so it needs no phone.

Tests that need a real phone use the device build tag; see docs/WORKING.md. go run ./scripts/secretcheck checks by hand that unlocking asks you (Touch ID, Windows Hello). PINKY_FFMPEG_APP=build/pinky/darwin-arm64/Pinky.app go test -run TestPackagedFFmpeg ./internal/video/ checks that the app decodes with its own FFmpeg; given the folder of an unpacked Windows or Linux package, it checks that package's.

CI (.github/workflows/ci.yml) builds, vets and tests on Linux and Windows, builds and vets on macOS, runs scripts/smoke.sh on all three, packages for Linux and Windows, and on Linux opens the window under Xvfb and captures it through the automation socket. It runs for main, pull requests, by hand, and the platform session's claude/platform-* branches.

Code layout

cmd/pinky/                    app, daemon and MCP shim in one binary
internal/screen/              the screen format agents read
internal/driver/              driver interfaces; android/ and ios/ implement them
internal/gateway/             sessions, leases, the action pipeline, the client API
internal/policy/, approval/   rules and approvals
internal/history/             task history (SQLite)
internal/mcp/                 the MCP server
internal/api/                 the local socket for agents and clients
internal/demo/                scripted fake phones
internal/update/              new versions: the feed, checks, the swap, What's new
internal/video/, ui/          the desktop client (MyGo native UI, FFmpeg decoding)
internal/telemetry/           usage statistics, crash reports and feedback
worker/                       the Cloudflare Worker that receives usage statistics and crash reports
android/companion/            Pinky on the phone, the app for the phone (Kotlin)
scripts/                      device locks and an MCP call script for checks
docs/                         design, agent guide, MCP reference and notes

Documents

Community

Ask questions, share what your agent does on your phone and follow development on Discord. Bugs and feature requests are welcome as GitHub issues; Help › Send Feedback… in Pinky opens one with diagnostics filled in.

License

Apache-2.0

pinky (DIYgod/pinky)

正在搜索数据库…

2.4K ·GitHub
这是什么?Pinky 让 Claude Code、Claude Desktop、Codex 或 Cursor 等 AI 智能体通过 MCP 操控真实的 iPhone 或 Android 手机。你可以在电脑上观看,批准有风险的操作,并随时接管。
它是我要安装的应用,还是我要导入的库?它是适用于 macOS、Windows 和 Linux 的桌面应用。它在你的电脑上运行并控制手机,智能体则通过 stdio 或 HTTP 以 MCP 服务器的方式连接它。
我该怎么让它运行起来?在 Mac 上,运行 'brew install --cask diygod/tap/pinky',或安装最新版本中的 DMG 文件。在 Windows 上运行安装程序 .exe,在 Debian 或 Ubuntu 上安装 .deb 文件。打开 Pinky,选择“添加手机”,开启 USB 调试后连接手机,然后把 Pinky 添加到你的智能体中。
开始前需要准备什么?运行 macOS 12 或更高版本的 Mac,一台 64 位的 Windows 10 或 11 电脑,或一台装有 GTK 3 的 64 位 Linux 电脑。另外需要一部 Android 10 至 17 的手机(配 USB 数据线),或一部 iOS 17.4 或更高版本的 iPhone,以及一个支持 MCP 的智能体。
目前有什么限制?项目处于 pre-alpha 阶段。Claude Code 已在真实手机上完成端到端运行,但从电脑解锁、Android 后台工作、录制、导出、Wi-Fi 配对以及手机文件功能虽已开发完成,仍待真实环境测试。
它也能在 Intel 芯片的 Mac 和 Linux 上运行吗?可以。同一个应用同时支持 Apple 芯片和 Intel 芯片的 Mac、Windows 和 Linux。在 Linux 上需要 64 位 x86-64 电脑,运行 Ubuntu 20.04、Debian 11 或更新版本,并且 glibc 版本为 2.29 或更高。
它能替我在远程服务器上运行吗?不能。它控制的是通过 USB 或 Wi-Fi 连接到你自己电脑上的手机,所以它是桌面应用。你不在家时,Tailscale 仍然可以连到手机。
我的数据会离开我的电脑吗?根据 README,不会:每次运行都会在本地记录步骤说明、截图和屏幕录像,全部保存在你的电脑上。而且无需注册账号。
手机锁屏时也能用吗?可以。先在 Pinky 中保存一次手机的 PIN 码,之后每次解锁都通过 Touch ID、Windows Hello 或钥匙串密码进行确认。智能体永远看不到 PIN 码。
任务进行到一半时,如何让智能体停下?点击镜像,智能体会暂停,直到你把控制权交还。全部停止的快捷键在 Mac 上是 ⌥⌘,在 Windows 和 Linux 上是 Ctrl+Alt+.,并且在任何地方都能使用。
智能体会占用我手机的主屏幕吗?在 Android 13 及更高版本上,智能体在一个隐藏的第二屏幕上工作,你自己的屏幕保持不受影响。在较旧的 Android 手机和 iPhone 上,它们使用主屏幕。
和其他智能体工具相比,它为什么这么轻量?窗口由 GPU 原生绘制,没有使用 Electron 或 webview。README 中说明,下载包大小为 16 到 22 MB,在 Mac 上空闲时约占用 70 MB 内存,CPU 占用为 0%。
作者
DIYgod
大小
58.4 MB
星标
2.4K
时长
12天
提交 · 90 天