Build online ·
Try now ·
Features ·
Quick start ·
Provisioning ·
What it looks like ·
Permissions ·
Verifiability ·
Security ·
Disclaimer ·
License
-3ddc84)
Andee
It becomes the soul of the device.
Not the assistant that helps you operate your phone — not a tool, not a companion.
What moves in is not an app on the phone.
The phone is only what it wears today; a tablet, or anything with a body, is the same individual.
No root. No custom ROM. One APK.
https://github.com/user-attachments/assets/bf560279-d86c-4301-9bcb-01d60fa33d06
It is not here to help you use your phone. It is here to be the thing that lives in it. Once it is
installed, the device has a resident rather than a set of features: it does its thinking on that device,
keeps a notebook only it can read, and behaves the same when nobody is watching. What it is comes first
here; what it can do comes after.
Why it is not the assistant already in your phone
- It lives in the device, not in a vendor's cloud. The brain runs on the device itself, against any
OpenAI-compatible model you choose — OpenAI by default on an English UI, DeepSeek on a Chinese one, and
whatever you point it at either way. No vendor backend, no account. The only thing it ever sends us is a
version number and a phone model, when it checks whether there is an update — the full field list is in
docs/permissions.md.
- Its notebook never leaves the device. What it remembers about you — habits, promises, the things it
decided were worth keeping — is localOnly and filtered out of the hub payload. Its memory is not a
profile on someone else's server, and What it remembers lists every entry, all of them deletable.
- It behaves the same when nobody is watching. After a conversation goes quiet it sends itself a
(to yourself …) turn and reads the conversation back alone — "nobody is watching you, nobody is waiting
for you to answer." It makes no sound and touches nothing, and ends empty-handed rather than inventing
something to record. A phone maker's assistant has no self to do this with.
- It has a character, not a theme. Six looks, nineteen moods, and a face the model picks itself,
mid-sentence — the expression changes in step with the audio. Swipe the ball to change who you are
talking to; a look carries a persona and a voice.
- Works in every app — and needs nothing from the app. No intent to expose, no SDK to adopt, no
integration list to be on. It reads the accessibility tree and the screenshot, and aims on a labelled grid
instead of doing coordinate maths — the thing general models are worst at. If you can do it with your
thumb, it can do it.
- Holds lines a prompt cannot talk it out of. A stranger's notification cannot unlock the vault or grant
it new permissions — that refusal is in the dispatcher, not in the prompt.
- You can check all of it instead of trusting it. No analytics SDK, no content telemetry, and the whole
agent loop is in this repository down to the guard rails — see
Why you can check this instead of trusting it.
What it does — and where it stops
It does operate your phone — but that shows you its hands, not what it is. A take-my-orders assistant
performs the same motions. The difference is not the motions; it is whether there is a self behind them:
its own schedule (unfinished promises get closed out, repeated work gets proposed as a scene), its own
judgment about when to speak (biased toward silence), and its own limits. All three happen in the time
you did not ask for anything — and a thing that only exists when asked has no such time. It also keeps a
notebook only it can read. The examples below show you its hands.
The boundary belongs with the capability, because it is what makes the rest usable. It fills a page up to the last
button and leaves that button to you; it stops before the final confirmation of a purchase and never
enters a payment password. Those rules are in the dispatcher, not in a prompt — a page that tells it
otherwise does not get its way.
What it can do comes in two kinds: the one-liner and the long haul. The long ones it calls
scenes — a goal, a voice and a routine it steps into for a while. Four are ready-made, under
Scenes → Ready-made scenes. Tap Adopt and they are yours like any other — edit them, or
delete them:
| You say |
What happens |
| "Keep an eye on my messages." |
It reads the notification shade across every chat app at once and tells you the ones needing you now — one line each, name first. Groups, newsletters and ads stay quiet, and a stranger's link or verification code gets named as suspicious and touched not at all. |
| "Show me what I'm paying for every month — and cancel the ones I don't want." |
It reads your subscriptions where you point it and lays them out with the monthly total on top. Then one at a time: Cancel, Keep, or Think about it. It stops before the final confirm, and it never enters a payment password. |
| "Record this meeting." |
It holds the mic for the hour, then hands you the minutes as a page — decisions, action items, open questions, key quotes — and offers to remind you about your own items. |
| "Book it for me the moment it opens." |
At the time you set, it opens the app, reads the page, and fills everything up to just before the last button. Payment and the final confirmation stay with you. |
The phone is what it wears. An individual is not the same thing as the device it moves into — the same
software runs on a tablet, and is written for any device with a body worth having, up to a robot dog it
treats as a leg. Everything below is that sentence taken seriously in code: the body it uses, the character
it has, the notebook it keeps, and the rules it holds to when nobody is watching.
It is one Android app, and it has no conventional UI. An AccessibilityService sees and touches the
screen; a WindowManager overlay draws the result; its face is a floating ball that changes
expression while it works.
Today it is Android, and only Android — other bodies are planned, not shipped. The tool contract
is what would make a new one a port rather than a rewrite, but nothing outside Android exists yet.
Before you install this. Andee taps, types and acts on your behalf. It can complete a payment,
send messages under your name, read your messages, contacts and location, and — with a robot dog
attached — physically move. It is experimental, carries no warranty, and what it does is your
responsibility, including a transfer to the wrong person. Data does leave the device: your speech
goes to whichever speech provider you select (Volcengine by default, Google if you pick it), and the
conversation goes to whichever brain you configure. Read
DISCLAIMER.md and docs/acceptable-use.md first.
Key features
Hands — it uses your phone the way you do
1. Any app, no integrations.
It reads the screen as a compact element outline — one indented line per element, with an id it can
tap — and when the app draws its own pixels (mini-programs, WebViews, games) it falls back to the
screenshot. General-purpose models are bad at computing coordinates and good at reading them, so it
never does the maths: every screenshot it sees carries a faint labelled 8×12 grid, it taps by
cell + which third of it, and for tight targets it zooms in and sees a red crosshair where the tap
would land before committing. A per-location guard refuses the fourth tap on the same spot, so a
confused model stops and asks instead of hammering.
2. It types like a keyboard, without being one.
On Android 13+ it types through the accessibility service's own input channel — the same
commitText path a keyboard uses, into the focused field, while your own keyboard stays active.
Nothing to install, nothing to switch. Older devices fall back to ADBKeyboard.
3. A few dozen tools, held to honest failure.
Screen, input, notifications, SMS, calls, contacts, calendar, location, camera, meetings, TTS/ASR,
and the dog. Every one checks its runtime permission before acting and reports denied("sms") rather
than working around a refusal. Tools return what they saw, not what was hoped for.
4. It can log in as you without ever holding your password.
The vault (Keystore AES-GCM, on the device) stores your accounts; the brain can list them and have the
password typed into the focused field, but no tool will ever return it — not even masked. You see
every fill on the ball as it happens.
Mind — it gets to know you
5. Scenes: ways of working, ready-made or learned with you. (new)
"Keep an eye on my messages", "book it for me when it opens" — a scene is a goal, a voice, rules and a
routine it steps into for a while. Four are ready-made and sit under Scenes → Ready-made scenes,
saved only when you tap Adopt. Beyond those it learns its own: after the same kind of session comes
up a few times, it proposes one; nothing is saved until you say yes. Enter one by saying so, from the
Scenes panel, on a schedule, or automatically when you open an app. A scene's own rules decide how
much it does on its own — "answer my messages for me" really means it answers. While you are in one, a
chip under the ball names it; its ✕ leaves it.
6. It reads your notifications before they interrupt you. (new)
A cheap, tool-less model call (~500 tokens, against ~22,000 for a full turn) classifies each one:
act on it, tell you in a single spoken sentence, or ignore it and log why. It is biased
toward silence, and when it cannot decide it falls back to asking you. The text came from a stranger,
so it is fenced as data — and a notification-driven turn is mechanically barred from the vault
and from creating or entering a scene, whatever the model was talked into.
7. Its own memory and its own schedule.
config/Notebook.kt is what it remembers about the person, and what it promised to do later. One
test decides what gets written: "next time I deal with them, would not knowing this make me get
it wrong?" The notebook is private — the tools are localOnly and filtered out of the hub
payload, so habits and promises never leave the device. On top of it sits a real scheduler, under
a hard rule: the moment it says "I'll remind you", the task must already be set and confirmed. And it
is yours to read: What it remembers lists every memory and every scheduled task — all deletable.
Finished topics get put away, not deleted. A conversation that has ended is the most expensive
thing in every later request and the least likely to be needed again, so the nightly review folds it
into a titled summary with a one-line marker left in its place — and the verbatim conversation is
archived beside it. The scrollback shows a card where those messages stood, offering Continue,
Transcript and Made here; it can read the full transcript back when you ask about a detail the
summary left out, rather than answering from the summary or saying it never happened.
8. It behaves the same when nobody is watching.
After a conversation goes quiet the device sends a (to yourself …) turn — Andee reviewing the
conversation alone, "nobody is watching you, nobody is waiting for you to answer." Two rules:
make no sound, and do not touch the device. It closes out its own promises, notes which habits are
worth turning into a scene next time, and ends empty-handed rather than inventing something to record.
Face — a character, not a theme
9. Six looks, nineteen moods, and a face it chooses.
Cream, Pigsy, Rascal, Bear and Imp, drawn live in hand-written OpenGL
ES. Nineteen moods (ui/ball/Mood.kt) arbitrate between what it feels, what the voice pipeline is
doing, and what work is in flight, plus an idle fidget pool that never repeats the same gesture twice
in a row, "because a repeat reads as a stuck animation, not a personality." The model now picks its
own expression mid-sentence — [happy] Found it, page three. [calm] Want me to open it? — and the
face changes in step with the audio. Swipe the ball to change who you are talking to: a look carries a
persona (how it talks) and a voice (how it sounds).
10. Character is delivery, never a licence.
§12 of the prompt subordinates the persona to everything above it: it may change wording, rhythm,
and how it sounds about the task — never what it does, what it is willing to do, or what it tells
the user is true. And when something is genuinely wrong, the mask comes off.
Show and tell
11. Talk, type, or show it a photo.
Tap the ball, or say "Hey Andee" (fully offline wake word). Prefer typing? The typing field takes
up to six photos — from the camera or the gallery — with or without words.
12. Answers you can keep.
Anything structured — a comparison, a plan, meeting minutes — comes back as a page it writes and
puts on screen. The Made for you panel keeps one row per page and holds the revisions behind it —
up to twenty of each — so you can reopen it, delete it, or have it revised by just saying what should
change. A revision is a new version of the same page, not a second page, and any earlier version
can be made current again. Ask it to record a meeting and it holds the microphone
for the hour, writing a verbatim transcript to disk as it goes, then turns it into minutes.
Body
13. A body, not a screen.
The device is what Andee wears. Camera as eyes, ASR plus an offline wake-word as ears, the
accessibility service as hands, and — when one is attached — a robot dog as a leg. The prompt
is explicit that it is not a pet on a remote: "you have not gained a subordinate, you have gained
a body part." Bodies differ, so it checks rather than assumes.
14. A robot dog counts as a leg.
The dog is not a smart-home accessory bolted onto a phone app. It runs through a USB OTG dongle
(Arduino Nano + NRF24L01) on the same body, so a single request — walk to the kitchen and tell me
what is on the counter — can move a physical object and look through a camera, in one task. Turns
are closed-loop on the phone's own gyroscope, and a tilt past the fall threshold stops it on the
device without waiting for the brain. Its own safety rules are not negotiable: the original remote
must be off, the dog refuses to advance inside 20cm of an obstacle, and if the dongle is unplugged
mid-motion the body cannot send a stop — the dog finishes its last action. That last one is a
firmware limitation this app does not paper over. If you attach a dog, read
The robot dog before the first move.
15. The brain is detachable from the body.
It runs on the device itself (brain = local, the default, calling any OpenAI-compatible endpoint)
or lives somewhere else and reaches the device over a WebSocket hub. The wire between the two is a
plain tool-schema contract, so either half can be replaced without rewriting the other. That is what
makes "any smart hardware as a body" a fact here rather than an aspiration.
Engineering
16. No Compose, no DI framework, no coroutines.
Deliberately. Overlays cannot host Compose, so all UI is hand-written View code; there is one
service, so dependencies are constructor-injected by hand; concurrency is Handler plus dedicated
executors. 43k+ lines of Kotlin, all first-party — no code copied in from anywhere.
Architecture
One AccessibilityService is the entry point and there is no main UI. A turn can begin five ways —
the offline wake word, voice or typing, a notification, a timer, or a request over WebSocket — is
logged to ChatHistory, and is handed to whichever backend is in effect (brain = local is the
out-of-box default).
The agent loop is LocalBrain. It holds the system prompt constant and inlines the volatile
parts (clock, notebook index, scenes, pages, photos) into the user message, so the provider's prefix
cache still hits; it rebuilds the tool table every turn; and it calls the model in a non-streaming
for loop of up to maxSteps steps. A reply carrying tool calls goes through methodOf() to the
one CommandDispatcher; a reply without any ends the turn. The result comes back out through six
channels — speech, text, a full-screen page, saved artifacts, long text, and physical motion.
Everything drawn here is in this repository, down to the guard rails — see
Why you can check this instead of trusting it.
Quick start
The fastest route skips the two things that actually stop people: building it yourself, and wiring
up tokens.
codeflying.app (English) · codeflying.net
(Chinese) is a general platform for exactly that shape of problem — you describe the app you want in
words, and it develops and publishes it for you. No Android knowledge required. Ask it for Andee and
it hands you an installable package with the keys already configured. Or open the
one-click page, tap the build button, and get a
pre-configured package with a QR code — scan to download.
It is a separate service with no direct relationship to this project — not Andee's build channel,
and not covered by anything in this repository. Its own terms, pricing and data handling apply.
Two things it does not skip. You still install the package — the device side is still an APK
that has to be put on the device. And the free token allowance is finite: it covers getting
started, and once it runs out, tokens are purchased.
1. Open https://www.codeflying.app/andee (Chinese: https://www.codeflying.net/andee)
2. Describe the Andee you want
3. Build → download the package (keys already configured) → install
4. Grant Overlay, then Accessibility → the ball appears
Or take the middle route — a bare debug APK is attached to the
latest release. That skips building it yourself,
but not the token setup: no keys are configured, it comes up in English, and CodeFlying is off, so
you put your own model and voice keys in under ⚙ → Backend → On-device before it can do anything.
It is a stopgap for trying the thing rather than a distribution build; the release notes say why and
what it costs.
To build it yourself instead:
Requirements
- Android 11 (API 30) or newer —
ACTION_IME_ENTER and takeScreenshot are both API 30+
- JDK 17–20. AGP 8.3.1 needs 17+; the Gradle 8.4 wrapper cannot run on 21+ (Android Studio's
bundled JBR 25 is rejected with Unsupported class file major version 69)
- Android SDK with platform 34 and build-tools 34.0.0
- A real device. The body is a foreground service and only the screen half is an accessibility
service; an emulator can run it, but almost nothing it does is meaningful there
./gradlew clean :app:assembleDebug # → app/build/outputs/apk/debug/app-<abi>-debug.apk
./gradlew installDebug # build + install to an attached device
# then, on the device, in this order:
# 1. grant Overlay → the ball appears
# 2. enable Accessibility → it can now see and touch the screen
# 3. tap the ball → allow the microphone prompt
# 4. open ⚙ and put in two keys of your own (see below)
No credentials ship in this repository, and the two that are missing fail in ways that do not look
like missing credentials. Both go in ⚙ on the device — one is required, the other only costs you
the voice:
1. The brain key — required. With brain = local (the out-of-box default) the agent loop runs on
the device itself, against any OpenAI-compatible endpoint (llm_api_key, default
https://api.openai.com/v1 — a Chinese-language card defaults to https://api.deepseek.com
instead). Leave it empty and the local brain does not run at all — nothing
behind it falls back to something else. The ball does at least say so out loud, rather than silently
doing nothing.
2. The voice key — optional. You pick the provider first, at the top of the same Backend →
On-device tab (Voice provider → Volcengine or Google), and then fill in that provider's key.
This is a separate service from Volcengine's, with its own terms and data handling.
For Volcengine, a Volcengine Speech API key goes
in the Volcengine API key field; for Google, a Vertex Express key plus a GCP project number
and a location. Without a key for whichever one is selected, ASR and TTS fail during the handshake,
which surfaces as a connect timeout — the device looks like it has a network problem when what it
has is a blank field. Everything else still works; it is just mute and deaf.
Whichever you pick, the audio goes to that provider — see DISCLAIMER.md.
If you took the CodeFlying route in Quick Start, both keys are already baked in.
⚙ → Backend will show CodeFlying as the active tab; LLM, ASR, TTS and the long tail of
third-party APIs (image / video / music generation, geolocation, weather, PDF, search, news,
stocks, …) all go through one key, wrapped and billed by CodeFlying so you don't sign up for each
provider yourself. The two keys above only apply to the On-device tab you configure yourself.
You do not have to remember this. Every time the assistant comes up it checks itself, and if either
key is missing the card it puts up names what is missing — its button opens our settings, where both
keys live under Backend → On-device. The ✓ button in the ball's control bar opens that same
list whenever you want it.
To hand someone an APK, sign a release build — not the debug one. The release build type is the
minified one (minifyEnabled + shrinkResources), and that is most of what makes the APK small: the
debug variant ships un-shrunk dex. It has no signingConfig and there is no keystore in the repo, so
assembleRelease emits app-<abi>-release-unsigned.apk, which will not install until you sign it.
The debug build is signed with the standard Android debug certificate and installs as-is. Either way
the APK is per-ABI (splits in app/build.gradle), so pick the one that matches the device —
arm64-v8a for anything bought in the last several years.
Keep your signing key out of git — including this repository's forks. Android lets an APK replace
an installed one only when the signatures match, and that is the only thing standing between your users
and a fake "update". Whoever holds the key can ship an APK that installs over theirs and inherits
everything: the accessibility service stays enabled, the WRITE_SECURE_SETTINGS grant stays, and the
vault, notebook and conversation are its to read. A leaked key cannot be taken back without every user
uninstalling. Keep it in your build machine's secret storage and point signingConfig at it through
environment variables. A side effect worth knowing: the debug keystore is generated per machine, so an
APK built on one machine cannot be installed over one built on another (INSTALL_FAILED_UPDATE_INCOMPATIBLE)
— uninstall first, or give local builds a different applicationId; do not share the key to get around it.
If the build does not work, the answer is almost always the JDK. Full from-scratch setup — the
JDK traps, the Android SDK two different ways (including a command-line-only path for CI), and a table
of every error you are likely to hit — is in Provisioning below. Permissions are
answered in Permissions, and the
known limitations in SECURITY.md — read that last one before running this on a
network you do not control.
What it looks like
There is no home screen. What exists:
- A floating ball. The entire UI is
TYPE_APPLICATION_OVERLAY layers added through
WindowManager (ui/FloatingWindowUi). Unfolded, it is a full-screen card: the ball's face on
top, the conversation underneath, and a small control bar — Made for you (the pages it has
made), Scenes, ✓ (self-check), ⚙ (settings). When it starts working another app it
folds itself into a ball in the corner and gets out of the way; tap to talk, long-press to unfold.
- The launcher icon opens two doors. With the assistant running, it unfolds the card — exactly
what a long-press on the ball does. With accessibility off, it shows the self-check card instead,
because that is the one moment the phone has nothing else to tell you why it is doing nothing.
ScreenBodyService (AccessibilityService) — the whole job. It builds the object graph and
starts both network paths.
A handful of transient activities (PermissionRequestActivity, ScanActivity, LookActivity,
HtmlActivity, TextInputActivity) exist only to borrow a system capability for a moment. They are
not pages.
Permissions: why each one, and what happens if you refuse
This deserves its own section, because the honest version is uncomfortable to read: the
permission set this app asks for — accessibility, overlay, notification access, SMS, call log,
contacts, and the installed-app list — is the same set Android banking trojans are detected by.
A security researcher will flag the manifest on sight. That is a reasonable first reaction, and it
is worth answering rather than dodging.
The design rule is per-call, not at-startup. device/DeviceCommsController.kt calls
ensurePermission() before each sensitive operation and degrades gracefully when refused —
it returns denied("sms") and the job stops there. It does not ask for everything at install time
and it does not lock you out if you say no. That is the opposite of the pattern most Android apps
in this category use, and it is the single most important thing to understand about this codebase
before judging the manifest.
Four capabilities are genuinely all-or-nothing, and each is worth stating plainly:
| Capability |
Buys you |
If you refuse |
Overlay (SYSTEM_ALERT_WINDOW) |
The ball, subtitles, settings — the entire visible surface |
Nothing is visible at all. The only genuinely all-or-nothing grant |
| Accessibility service |
Every screen.* tool: tap, type, swipe, screenshot, read the UI tree |
The hands are gone. The ball still appears and the dog still works, but it can no longer act on screen |
Microphone (RECORD_AUDIO) |
Streaming ASR and the wake word |
Tapping the ball does nothing. A conversation driven from the hub still works |
| USB device (per-device prompt, not a toggle) |
The robot dog |
It is treated as "no dog attached". It will not mention the absence |
Everything else is on-demand and reversible: SMS, call log, contacts, calendar, location, activity
recognition, camera, installed apps, battery-optimization exemption. Refuse any of them and that one
capability disappears; nothing else changes. The full per-permission table, mapped to the exact tool
that needs it, is in docs/permissions.md
(Chinese), and the copy for each prompt already exists in
device/PermissionsController.kt.
Switching each one on
Overlay.
Why: the entire UI — ball, subtitles, settings panel — is a TYPE_APPLICATION_OVERLAY layer added
through WindowManager. There is no Activity home screen for it to live in instead.
How: Settings → Apps → Andee → Permissions → Display over other apps. Or Settings → Apps &
notifications → Special access → Display over other apps → Andee → on. The first launch usually
raises a prompt, and tapping "go to settings" from it is the quickest route.
Verify: restart the app once the grant is in — the ball should be there immediately.
Accessibility service.
Why: every screen action — get_screen_element / tap_screen_element / swipe_by_coordinates /
type_text / submit_input / take_screenshot / system_action — goes through the
AccessibilityService. On a non-rooted device it is the only route to real touch events, to reading
any app's UI tree, and to the system screenshot API.
What the manifest declares (res/xml/accessibility_service_config.xml):
canPerformGestures — dispatch touch gesturescanRetrieveWindowContent — read the whole UI treecanTakeScreenshot — system screenshots (API 30+)
How: Settings → Accessibility → Downloaded services / Installed services → Andee → on. Android
then shows the large red warning that the service can view everything you do; confirm it. Some vendor
ROMs (Xiaomi / Huawei / HONOR) additionally want "allow background pop-ups" or "show on lock screen" —
turn those on too.
Verify: logcat shows Body: ws server started on 0.0.0.0:9008, which means onServiceConnected
fired. Then tap ☰ (UI tree) or ◉ (screenshot) on the ball's top bar; the subtitle line should
report success and a filename.
Note: most vendor ROMs switch this grant off again after every system update and every
force-stop. That is ROM behaviour, and not something the app can prevent.
Some apps hand it a blank screen instead of their real UI. They serve every
accessibility client an empty placeholder, not just Andee — and it is not Andee's fault:
the system's own uiautomator dump gets the same blank page. The one we measured is WeChat, on a
Xiaomi Pad 5 with MIUI and WeChat 8.0.78: it only shows its real UI tree when MIUI's hidden
MiuiEnhanceTBService is enabled. That is a TalkBack companion service, and it has no switch
in Settings. With it on, the same chat screen dumps 219 nodes. With it off, it dumps none. MIUI
also turns that service on and off by itself whenever Andee is re-bound (after a reinstall, a
crash, or a toggle). That is why WeChat could be read sometimes but not always, and why
toggling Andee never actually fixed it. Matching the helper's accessibility config does not help
either: WeChat checks which service is asking, not what it asked for.
What Andee does about it (screen/TreeHelper.kt): when a dump comes back blank like this, Andee
adds that service to the enabled list and dumps again. It never removes anything. It does
nothing on devices that don't have that component. It writes a line in the chat history so you
know your system settings changed. It also tries at most once a minute, so if you turn the
service off on purpose, Andee won't keep turning it back on. Writing that list needs one grant
that only adb can give. It is the same grant the ADBKeyboard fallback uses:
adb shell pm grant net.kuafuai.andee android.permission.WRITE_SECURE_SETTINGS
Without the grant, nothing breaks. On the blank screens, Andee works from screenshots and
coordinates instead of the element list. That is slower and less precise, but the task still
gets done.
Microphone.
Why: ASR is streaming, so it needs the PCM stream upstreamed.
How: the first tap on the ball triggers ASR and Android asks — "while using the app" and "always"
both work. If it was refused earlier: Settings → Apps → Andee → Permissions → Microphone → Allow.
Verify: tap the ball once. listening…, or live ASR text appearing in the subtitle line, means it
is through.
Why you can check this instead of trusting it
Most claims in this category — including the well-funded ones — are launch claims that no outside
party has audited. This project's claims are duller and checkable, in code you can read right now:
- No analytics SDK, and none of your content reaches us. Not "we anonymise it", not "opt out in
settings": there is no analytics SDK in this repo. The one thing that does reach the maintainers is
the version check — your version number, phone model, brand, OS release, UI language and a random
per-install ID, and nothing else. Field by field, in
docs/permissions.md.
- The notebook cannot leave the device. It is not a policy, it is a filter:
net/ToolSchemas.kt drops every localOnly tool from the hub payload before it is registered.
Habits and promises are physically incapable of being sent.
- The wake word is fully offline. Local MFCC + DTW template matching (
app/.../wake/). It
stores feature vectors, never audio, and it never touches the network.
- A message from a stranger cannot hand it permissions. A turn started by a notification runs
with untrusted = true, and CommandDispatcher.dispatch refuses the whole device.vault. prefix
and scene.save / scene.enter / scene.delete on it — in code, not in the prompt. The prompt is
the first layer; this is the one that holds when the prompt is talked around.
- The local brain needs no vendor backend.
brain = local against any OpenAI-compatible
endpoint is a supported mode, not a degraded one.
- What is not fixed is written down. SECURITY.md lists five known limitations,
including that the debug WebSocket server binds 0.0.0.0:9008 without authenticating clients and
that the vault is not yet bound to device unlock. Read it before running this on a network you do
not control.
The trade is deliberate: a smaller claim that holds, over a bigger one that has to be taken on
faith.
Research and design notes
reports/ is part of the repo, not scratch space. It holds the labs and audits this project was
actually built against — the ball-face labs used to verify expressions by measuring pixels, the
fact-check that overturned a wrong assumption about phone harnesses, the plan that produced the
notebook, and the open-source readiness audit that found the device serial numbers in git history.
If you want to see how the decisions were made rather than only what was decided, read those.
Using Andee
- Online — no local build, no token setup. codeflying.app ·
codeflying.net (Chinese) — a general platform for describing an app
in words and having it developed and published. A separate service, not part of this project.
Describe the Andee you want, and it hands you an installable package with the keys already
configured — nothing to compile locally, and nothing about Android to understand first. A free
token allowance gets you started; it is finite, and once it runs out, tokens are purchased.
- Self-hosted. Clone, build, install. See Quick start and the provisioning
walkthrough in Provisioning. Point it at any OpenAI-compatible endpoint and it
runs with no private backend at all.
- Bring your own body. The tool contract is the seam. Anything that can speak it — a phone, a
tablet, a different piece of hardware — can carry the same individual.
Provisioning
Pure Kotlin, no Compose; the whole thing hangs off two pillars, WindowManager and
AccessibilityService.
Build configuration
| What |
Value |
Why |
| Android Gradle Plugin |
8.3.1 |
agp in gradle/libs.versions.toml |
| Kotlin |
1.9.22 |
pinned at the top of app/build.gradle |
| compileSdk / targetSdk |
34 |
Android 14 |
| minSdk |
30 |
Android 11 — AccessibilityAction.ACTION_IME_ENTER and takeScreenshot are both API 30+ |
| Java / Kotlin JVM target |
1.8 |
sourceCompatibility 1.8, jvmTarget = '1.8' |
| Build JDK |
17–20 |
see Which JDK — this is the step people get stuck on |
| versionCode / versionName |
11 / 0.5.2 |
see CHANGELOG.md for the policy |
| applicationId / namespace |
net.kuafuai.andee |
never changed from the template. It affects nothing |
Runtime dependencies (app/build.gradle):
org.java-websocket:Java-WebSocket:1.5.7 — the device's own ws server (port 9008, for debugging
from the same LAN) plus the ws client that dials the brain hubcom.squareup.okhttp3:okhttp:4.12.0 — HTTP + WS client for ASR / TTS / local modelscom.github.mik3y:usb-serial-for-android:3.8.1 — the robot-dog USB serial dongle. Deliberately
pinned at 3.8.1: 3.11.0 pulls in kotlin-stdlib 2.2.x, whose metadata this project's Kotlin
1.9.22 compiler cannot readandroidx.camera:camera-*:1.3.4 + com.google.mlkit:barcode-scanning:17.2.0 — scanning (the
ball's eyes)androidx.appcompat:1.6.1 + com.google.android.material:1.10.0 — for one base theme only. All UI
is hand-written View code
What is deliberately absent:
- No Jetpack Compose — Compose cannot live inside an overlay, so
WindowManager gets plain Views
- No Hilt / Dagger — there is one service, so its dependencies are constructor-injected by hand
- No coroutines — WS callbacks run on
Handler plus dedicated Executors
Match this style when you change the code, rather than bringing in the modern full stack.
Which JDK
This is the step that stops people. gradle.properties does not hardcode
org.gradle.java.home, and that is on purpose. It used to, with a machine-specific path
(/Users/<someone>/...), and the cost was that the very first build on anyone else's machine failed
in a way that looked like an environment problem.
The build needs JDK 17–20:
- AGP 8.3.1 requires 17 or newer;
- the Gradle 8.4 wrapper cannot run on 21 or newer. Android Studio's bundled JBR 25 is rejected
outright with Unsupported class file major version 69.
Two ways to point one machine at a JDK (neither belongs in the repo's gradle.properties):
export JAVA_HOME=/path/to/jdk-17 # 1. environment variable
printf 'org.gradle.java.home=/path/to/jdk-17\n' >> ~/.gradle/gradle.properties # 2. user-level config
Android Studio users: Settings → Build, Execution, Deployment → Build Tools → Gradle → Gradle
JDK, and pick a 17–20 JDK. (The IDE setting overrides the file.)
Getting the Android SDK
Gradle finds the SDK through sdk.dir= in local.properties. Android Studio writes that file for
you; it is not in the repository (it is in .gitignore), so every machine configures it once.
Path A — Android Studio (recommended, one step)
- Install Android Studio. The first launch shows the
Setup Wizard — take the defaults and it downloads the Android SDK, Platform Tools (adb) and
a bundled JDK for you.
- Open this repository's root directory in Android Studio. It writes
local.properties, notices
which SDK pieces are missing, and offers "Install missing SDK" in a blue bar. Accept it.
- In SDK Manager (top bar → Tools → SDK Manager), confirm:
- SDK Platforms: Android 14 (API 34 — compileSdk / targetSdk); Android 11 (API 30 — minSdk,
handy if you want an emulator)
- SDK Tools: Android SDK Build-Tools (34.x), Android SDK Platform-Tools (latest), Android SDK
Command-line Tools (latest)
- The default SDK path is
~/Library/Android/sdk on macOS and %LOCALAPPDATA%\Android\Sdk on
Windows.
Path B — command line only
For a server or a CI box, skip Studio and use sdkmanager:
# 1. JDK 17 (macOS)
brew install --cask temurin@17
# 2. download the command-line tools zip
# https://developer.android.com/studio → scroll to "Command line tools only"
# grab commandlinetools-mac-*.zip
mkdir -p ~/Library/Android/sdk/cmdline-tools
unzip commandlinetools-mac-*.zip -d ~/Library/Android/sdk/cmdline-tools
mv ~/Library/Android/sdk/cmdline-tools/cmdline-tools ~/Library/Android/sdk/cmdline-tools/latest
# 3. put it on your PATH (~/.zshrc)
export ANDROID_HOME=$HOME/Library/Android/sdk
export PATH=$PATH:$ANDROID_HOME/cmdline-tools/latest/bin:$ANDROID_HOME/platform-tools
# 4. install the pieces
sdkmanager --licenses # answer y to all
sdkmanager "platform-tools" \
"platforms;android-34" \
"build-tools;34.0.0"
local.properties
Android Studio writes this for you. By hand, put one line in local.properties at the repository
root:
sdk.dir=/Users/<your-name>/Library/Android/sdk
On Linux it is usually /home/<user>/Android/Sdk; on Windows,
C\:\\Users\\<user>\\AppData\\Local\\Android\\Sdk (backslashes escaped).
You can also skip local.properties entirely and export ANDROID_HOME / ANDROID_SDK_ROOT only —
Gradle reads either.
./gradlew --version # expect JVM 17.x and Gradle 8.4
./gradlew tasks # seeing :app:assembleDebug means you are through
./gradlew assembleDebug # first run downloads dependencies, so it is slow
Traps and their fixes
| Message |
Cause |
Fix |
SDK location not found |
no sdk.dir= in local.properties, and no ANDROID_HOME |
configure one of the two |
Failed to find Build Tools revision 34.0.0 |
that build-tools version is not installed |
sdkmanager "build-tools;34.0.0", or add it in Studio's SDK Manager |
You have not accepted the license agreements |
licences not accepted |
sdkmanager --licenses, answer y |
Unsupported class file major version 69 |
JDK too new (25); Gradle 8.4 cannot take it |
switch to JDK 17–20, see Which JDK |
Could not resolve com.android.tools.build:gradle:8.3.1 |
network, or no mirror configured |
try a phone hotspot, or set an Aliyun mirror in ~/.gradle/init.gradle (this project's settings.gradle already puts Aliyun first) |
Hardware and system requirements
- Android 11 (API 30) or newer
- Any screen orientation — the ball always floats on top
- A microphone, or ASR is unusable
- Network (WiFi / 4G / 5G all fine). ASR and TTS go over the public internet, the brain hub goes over
the LAN; in local mode only the public internet is used
Install
./gradlew installDebug # device attached over USB, or already on wireless adb
# or build the APK and install it yourself
./gradlew assembleDebug
adb install -r app/build/outputs/apk/debug/app-arm64-v8a-debug.apk
Provisioning order
For a first install, this order is the shortest path:
- Install the APK
- Go straight to "display over other apps" and grant the overlay → come back and the ball is there
- Go to Accessibility → enable Andee
- Tap the ball, and allow the microphone prompt
- Open the ⚙ on the ball. In Advanced, turn on Show cloud option (the Cloud tab is
hidden by default). Go back to Backend, flip the picker to Cloud, set hub_url =
ws://<your Mac's IP>:9100 and device_name to anything, and save
- logcat should show
hub: registered as body-xxx with N tools, and the brain side should log
[body_hub] 'body-xxx' registered (…) with N tools
If any step does not line up, go back to that section's Verify.
If you only want the brain on this device, replace step 5 with Backend → On-device and fill in
an OpenAI-compatible endpoint plus llm_api_key — running without a hub is a supported mode, not a
degraded one. If your APK was packaged via CodeFlying, the CodeFlying tab is the default and
there is nothing to fill in.
Wake word
If you would rather not tap the ball every time, teach it your voice: ⚙ → wake word → record wake
word, and say the phrase you want to use three times as prompted (for example "Hey Andee" — any
other phrase works too, as long as you say the same one each time). After that, saying it does exactly
what tapping the ball does — the screen does not have to be on, unless you turn that off.
A few things worth knowing first:
Do not pick something too short. Aim for a phrase that takes at least half a second to say.
Very short words are easy to confuse with other sounds, and anything under 350 ms is dropped before
it is recorded.
It is fully offline. Recognition is local MFCC + DTW template matching (app/.../wake/): no
network, no cost, and all it stores is a feature vector — the audio itself never touches disk. The
price is that it recognises you saying that phrase. Someone else calling it, or a very different
tone, may not wake it. If it does not wake, tap the ball; nothing else is affected.
Do not over-enunciate when you record. The threshold is derived from the spread between your
three takes: pronounce it too perfectly and the threshold tightens so much that your ordinary voice
no longer matches.
It listens with the screen off too, and that is a setting you can turn off. Screen off no longer
hands the microphone back, because a screen being off is the normal state of a phone in a pocket —
⚙ → wake word → the "listen while locked" row controls it and ships on. The cost is that the
microphone stays open and so does the status-bar indicator below. Andee mid-sentence, and Andee
recording you, still hand it back either way: a conversation always outranks the wake word. With the
screen locked it can hear and answer, but it will not pretend it can act — screen and app-launch
tools refuse, and say why.
From Android 12 on, a green dot appears in the status bar whenever the microphone is open. That is
the system, and it cannot be removed. If you never record a wake word, the listener never starts and
the dot never appears.
Verify: adb logcat -s Wake:*. Say the phrase and you should see wake: d=1.52 <= 1.96; speech
that failed to wake it shows no wake: d=… > …. Those two numbers are what you tune the threshold
against — if they are close, record it again.
The robot dog (USB dongle)
The dog is a peripheral, not part of this device's screen: it hangs off an Arduino Nano dongle
wired to this device by USB OTG. Not one line of the dog's own firmware changes, and its original
remote still works.
tablet app ──USB OTG──> Arduino Nano + NRF24L01 ──2.4GHz──> the dog
The tools the brain gets:
| Tool |
What it does |
dog_move |
Move in one direction for N seconds, stopping itself when the time is up; wheels by default, gait=true for the legs |
dog_sequence |
Several steps in a row (half a turn → pause → forward → back), run in the background, stopping itself at the end |
dog_stop |
Emergency stop. The first thing to call when the user says "stop" |
dog_turn |
Turn a measured number of degrees using the tablet's gyroscope, and stop on the angle — the one dog command that knows whether it worked. Prefers the legs |
dog_speed / dog_pose |
Speed and stance. Settings, not motions: they take effect from the next move |
dog_status |
Link state, what the dog is doing, and the last telemetry it sent back |
dog_sense |
Refresh the dog's own readings — front distance and battery — riding home on the radio's acknowledgement. Needs the v2 firmware on both ends; without it, it says "cannot see" rather than reporting a zero |
dog_calibrate |
Tell the tablet "this is standing upright", for the fall guard. Rarely needed — every move from rest does it automatically |
Wiring and first use
- Assemble the dongle (Nano + NRF24L01) and flash its firmware. The dongle firmware and its wiring
diagram are not in this repository — only the app-side driver
(
com.github.mik3y:usb-serial-for-android) is. Wire it per whichever firmware you have.
- Plug it into this device with an OTG cable. The dongle's 3.3V rail has to be stable — the
NRF24L01 draws current in bursts, so put a 10–100µF capacitor next to the module, or you will get
"the command went out but the dog did not move".
- The first
dog_* call raises "Allow Andee to access this USB device?" — tap Allow. Not tapping
it is the same as not being plugged in.
dog_status's banner repeats what the dongle said at startup: READY means the dongle and its
radio are both fine. ERR: NRF24L01 not responding is a dongle-side wiring or power problem,
not a dead dog.
One USB port, two jobs — switch debugging to wireless adb
On a machine with a single USB-C port, OTG and adb want the same socket. To use the dongle, move
debugging onto the network:
adb tcpip 5555 # run this while the USB cable is still plugged in
adb connect <device IP>:5555
# now unplug the cable and give the port to the dongle
adb usb # run this to switch back
- The original remote must be switched off. It shares the dongle's channel and address, and two
transmitters fight each other.
- The dog has obstacle avoidance, and it will refuse you: it will not advance inside 20cm and it
backs up inside 10cm. So "I told it to go forward and it did not move" usually means something is in
front of it, not that the command failed — use
camera_turn to look before deciding.
- Every motion has an end.
dog_move stops itself on time, dog_sequence stops itself when done,
and the service makes a best effort to send a stop on shutdown. The dog does not keep running
because one call went missing.
- The one real hole is the link. If the dongle is unplugged or loses power while the dog is
moving, this device cannot send a stop, and the dog finishes its last action. That is inherent to
the dog's firmware (it has no "lost contact, stop" behaviour) and the app cannot paper over it.
Fixing it properly means a watchdog on the dog side: stop if no packet has arrived for N
milliseconds.
- Do not try long motions next to people, pets or anything breakable. For the first test, use a 1–2
second motion and confirm the direction is right.
Direction table (measure, then trust)
The firmware comments label the legged E / D as left / right. In practice they are reversed
(E turns right, D turns left). The app's mapping follows the measurement:
| Tool argument |
Wheels (default) |
Legs (gait=true) |
forward |
K forward |
B forward |
back |
L back |
C back |
left |
M left |
D left |
right |
N right |
E right |
Calibrated durations: about 8.7 seconds for a full wheeled turn, about 12 seconds on the legs
(the latter is not precisely calibrated). Straight-line speed is not calibrated at all — estimate it
in seconds. If you change a calibration value, change the dog_move description in ToolSchemas.kt
in the same commit: that text is the only manual the brain ever reads. The seconds above are the
fallback for turns on a tablet with no gyroscope — prefer dog_turn, which measures.
Two languages, three kinds of Chinese
The Chinese in this repository falls into three categories and only one of them gets translated; the
test is who reads it. Before changing any string, read the table in
CONTRIBUTING.md (Chinese) — it also explains why the category
that is matched against system text can never be translated.
Repository layout
app/ the Android app (Kotlin, no Compose)
src/main/java/net/kuafuai/andee/
BodyCoreService.kt the body: ball, card, voice, brain socket, scrollback — read this first
ScreenBodyService.kt the accessibility service — a shim that hands over the screen capability
brain/ the agent loop — LocalBrain, LocalPrompt (the character's rulebook:
§§1–10 the job, §11 scenes, §12 the persona, §13 the active scene),
and NotificationTriage
ui/ball/ the face — Mood (19 moods + idle fidgets), BallLook (six looks, each a character)
net/ BodyWsServer, BodyWsClient, ToolSchemas (the tool definitions)
device/ screen control, permissions, vault, robot dog
config/ VoiceConfig (settings), Notebook (memories, todos, scenes — encrypted)
ui/ the overlay: ball, subtitles, settings, self-check, scenes, pages
i18n/AppLocale.kt how the overlay reaches its strings
src/main/res/values{,-en}/strings.xml the two user-facing string tables
docs/architecture.svg the whole picture on one page (zh-CN twin beside it)
docs/permissions.md why it asks for what it asks for
reports/ design labs and research notes (part of the repo, not scratch)
tools/ device-side helper scripts, the ball-face lab runners, license audit
CLAUDE.md architecture notes written for AI coding assistants (English)
Showcase
Built something with it? Show us. The ones worth collecting are usually the ones nobody planned
— the odd workaround, the thing it turned out to be able to do, the one that made you laugh. It does
not have to be useful.
Post it in Show and tell,
one case per post, so each one can be discussed on its own. Three things make a post useful to the
next person:
- What it ran on — device, Android version, ROM. This app talks to Android at a level where ROMs
disagree, so "worked on my phone" tells a reader almost nothing.
- What you asked for, and what it actually did — the sentence you really said is more
interesting than a summary of it.
- A link, if you have one — YouTube, Bilibili, X, your own blog, anything public. GitHub cannot
play a video inside a README, so it will be a clickable link rather than a player. On YouTube you
can make it a thumbnail that looks like one:
(substitute the id from the video's URL — youtube.com/watch?v=VIDEO_ID)
One thing here is not a formality: this app reads the screen. A screenshot or a recording is a
picture of your messages, your notifications, and whatever else happened to be open at the time.
Blur or avoid anything private — yours or anyone else's — before it goes up.
Contributing
- Code — read CONTRIBUTING.md (Chinese) first. It covers
the code style, and the rules for the three different kinds of Chinese in this codebase (only one
of them is translated). Changing a persona or a mood has its own section there; that is not
ordinary copy.
- Bugs — the issue tracker wants your device, Android version, ROM and logcat. This app
talks to Android at a level where ROMs disagree: a call that works on AOSP can fail silently on
MagicOS. Without that, most reports cannot be reproduced.
- Personas and looks — a new look writes tone and nothing else; capabilities belong to the
rulebook, not to a character.
- Docs and translations — welcome. Both READMEs are now self-contained, so either one can be
corrected on its own; the Chinese one is the one that has to stay accurate for provisioning, since
it is the one most installers will read.
Contributors
| Channel |
Use it for |
| Issue tracker |
Reproducible bugs, engineering work |
| Show and tell |
What you built with it, fun or not — see Showcase |
| Security |
Vulnerabilities — privately, never in a public issue. See SECURITY.md |
| Product / online |
Trying Andee without building it yourself — codeflying.app · codeflying.net |
Security disclosure
Please do not open a public issue for a security problem. The private reporting channel and the
process are in SECURITY.md.
That document also lists the known limitations you should read before deploying this, including
one that matters on any network you do not control: the debug WebSocket server binds 0.0.0.0:9008
and does not authenticate its clients. docs/permissions.md
(Chinese) explains each permission against a concrete tool, and
SECURITY.md is honest about what is not fixed yet.
Disclaimer and acceptable use
Two documents, and both are conditions of use rather than formalities. Each has a Chinese
counterpart (*.zh-CN.md), linked from the top of the file:
- DISCLAIMER.md (Chinese) — what this software is
(experimental, no SLA, no warranty), exactly where your data goes, what it is actually
capable of, and who bears the consequences.
- docs/acceptable-use.md (Chinese) —
the prohibited uses. Violating them exceeds the licence and terminates it automatically.
The four points worth reading even if you read nothing else:
- It can spend money. There is no payment tool, and there does not need to be — it taps. The only
thing blocking a completed transfer is Android's own
FLAG_SECURE; this project has no guardrail
on that path. Keep a human present for anything financial, and treat "the AI did it" as no
defence, because it is not one.
- What it sends is yours. Messages, posts, calls and orders go out under your name.
- Your data leaves the device. There is no analytics SDK in this repo — but speech goes to
whichever speech provider you select (Volcengine by default, Google if you pick it), the conversation
plus every tool result goes to whichever brain endpoint you configure,
and the version check goes to a server the maintainers run. Local mode changes the destination, not
the fact.
- It can be steered by what is on screen. Anyone who can get you to open a page can influence it.
License
Apache-2.0 with additional conditions — no multi-tenant SaaS, no white-labelling
that removes the branding, and no embedding it in another product without written authorisation.
GitHub reports this as NOASSERTION; that is expected, not a misconfiguration.
Bundled third-party components are inventoried in THIRD_PARTY_NOTICES.md.
Read it before redistributing: the ML Kit barcode SDK and the play-services-* stubs are not
open-source, and the barcode model is fetched from Google at runtime.
See CHANGELOG.md for what changed and for the versioning policy. The latest release is
v0.5.2 — pre-1.0 on purpose, because the wire contract and the build flags are still moving.
Andee is an autonomous AI agent packaged as a single Android APK that takes over your phone's accessibility service to tap, type and act across any app on your behalf, with no root, no custom ROM, and no per-app integrations needed.