ai-phone README

The project's README from github.com/oxnw/ai-phone. As one markdown file: README.md.

ai-phone

An AI phone agent that runs on your computer.

Documentation · Architecture · AGPL-3.0

ai-phone setup running in a terminal

Installation

Requirement Details
Platform macOS on Apple Silicon. Intel Macs, Linux and Windows are not currently supported.
Memory 16 GB recommended for the default 4B model; 32 GB for 9B. Setup also offers a smaller 2B model for 8 GB systems.
Dependencies Git, Homebrew and uv. The installer handles Python packages and the native SIP library.
Storage Several GB for model downloads, depending on the models selected. Setup checks available disk space.
Phone connection A SIP address for direct calls, or your own SIP trunk, Telnyx or Twilio account for telephone numbers. Carrier charges apply. 
git clone https://github.com/oxnw/ai-phone.git
cd ai-phone
./scripts/install.sh
uv run ai-phone setup

Setup lets you choose incoming calls, outgoing calls, or both, then configures the model, line and voice. It recommends a local model based on detected RAM and downloads the required weights. Use uv run ai-phone setup --plain for plain prompts.

The current speech backends use MLX on Apple Silicon, even when a cloud model handles the conversation. Supporting Linux or Windows would require different speech backends and changes to installation and platform integration. The platform requirement applies to the computer running ai-phone; callers can use any phone or SIP client.

Usage

Start the daemon after setup:

uv run ai-phone run

Keep it running while you want to receive calls. For outgoing calls only, start it when needed and stop it with Ctrl-C afterward. Setup does not install a background service.

From another terminal in the repo, give the phone agent instructions for a call:

uv run ai-phone call +15555550100 --json \
  --prompt "Ask whether they have a table for two at eight tonight."

Replace the example number with the number you want to call. The command returns the transcript when the call ends. Every outgoing call starts with an AI disclosure. Incoming calls use the greeting and instructions you set during setup. English and one call at a time are currently supported.

For a free local test, enable incoming calls, select A SIP address in setup, and keep the name agent:

pjsua --local-port 5070 sip:[email protected]

Run setup again to change the voice, model, call directions or caller ID, then restart the daemon.

Agent integration

An agent with shell access can use relay mode to control each turn:

uv run ai-phone call +15555550100 --relay --json
uv run ai-phone say "Hi, I'd like to book a table for two." --json
uv run ai-phone listen --json
uv run ai-phone say "Eight o'clock, please." --json
uv run ai-phone hangup --say "Thank you. Goodbye." --json

With no active call, listen --json waits for an incoming call and handles it in relay mode. Each command with --json returns one JSON object on stdout; call also sends progress events to stderr. An unanswered relay turn times out after 25 seconds by default. See the command reference for call IDs, transcript retrieval, timeouts and exit codes.

Models

Component Options
Speech recognition Parakeet, running locally through MLX.
Language model A local MLX model, your Claude Code or Codex subscription, or an OpenAI-compatible endpoint.
Voice Phonon with its bundled English voice, or Kokoro with American and British voices and adjustable speed.

Speech recognition and synthesis stay local. A remote language model receives the call's text, instructions and conversation history. An external agent's cloud provider also sees any transcripts passed to it. Phone audio travels over SIP/RTP; carrier charges and subscription limits apply.

Documentation

Install and setup have been checked from a fresh clone. Local call and interruption tests have run; live carrier and end-to-end agent acceptance checks remain pending. The latest local run measured 1.50 s median first-audible response under heavy memory pressure; it missed the one-second target. Full conditions and earlier results are documented.

License

AGPL-3.0-or-later. The bundled PJSIP interface files use GPLv2-or-later; see the license notes.

ai-phone (oxnw/ai-phone)

kokoro
Searching Database…

391 ·GitHub
Ai-phone
What is this?ai-phone is an AI phone agent that runs on your own computer, answering and placing SIP phone calls using local speech recognition and synthesis plus a configurable language model.
How do I get it running?Clone the repo and run the setup wizard, which detects your RAM, recommends a local model, downloads the weights, and configures your line, voice and call directions; then start the daemon to begin handling calls.
What do I need installed before setup?You need Git, Homebrew and uv on the machine; the installer itself takes care of the Python packages and the native SIP library.
Does it run on any Mac, or other operating systems?Only macOS on Apple Silicon is supported right now; Intel Macs, Linux and Windows don't work because the speech backends depend on MLX.
How does it actually connect to a real phone number?It uses SIP: you can test for free with a direct SIP address, or hook up your own SIP trunk, Telnyx or Twilio account to get a real telephone number, which brings carrier charges.
How much RAM and storage do I need?16 GB RAM is recommended for the default 4B model (32 GB for the 9B option), with a smaller 2B model offered for 8 GB machines; model downloads take several GB of disk space.
Where does my call data go?Speech recognition and voice synthesis always stay local on your Mac, but if you pick a remote language model (Claude Code, Codex, or an OpenAI-compatible endpoint), that provider receives the call's text, instructions and conversation history.
What language model and voice options does it support?You can use a local MLX model, your Claude Code or Codex subscription, or any OpenAI-compatible endpoint for the conversation; voices are Phonon's bundled English voice or Kokoro with adjustable American/British accents and speed.
Can a coding agent take direct control of a call?Yes, relay mode lets an agent with shell access answer and drive each turn itself; commands with --json return structured JSON for call IDs, transcripts, timeouts and exit codes.
How fast does it respond during a call?The latest measured run showed a 1.50-second median first-audible response under heavy memory pressure, which missed the project's one-second target.
How many calls or languages does it handle at once?Only one call at a time and English conversations are currently supported.
What are similar projects?Lampline, VoiceStudioV2, slotstream, speech-swift, Rapid-MLX, VoiceStudio, unsloth, lipflow
Which topics is it in?kokoro, daemon, voice-ai, mlx, on-device-ai, voice-assistant, voice, local-ai, metal, speech-recognition, tts, text-to-speech
Creator
oxnw
Size
2.3 MB
Stars
391
Updated
CommitNov 01
Age
5d
Commits · 90 Days