chimera-vocal-synth README

The project's README from github.com/jamesondh/chimera-vocal-synth. As one markdown file: README.md.

CHIMERA

A physical-modeling voice synthesizer in the browser. Every sound comes from a simulated airway (lungs, vocal folds or syrinx, a branched vocal tract, lips, nose), not from samples. The engine is Rust, compiled to WebAssembly for an AudioWorklet; the same code runs natively for tests and the chimera CLI.

Vision & scope

  1. One instrument: a physically modeled voice (lungs, vocal folds, throat, mouth). No samples, no neural audio.
  2. The web app is the primary surface: anyone should be able to make songs in it without an LLM's help. Mobile is the likely next surface. The plugin (CLAP/VST3/AU) still ships, but it's secondary.
  3. As realistic as we can make it: the main push is to see how close the simulation gets to a real voice. The robotic, Vocaloid-like sound people like today stays playable as a preset (#93).
  4. Languages enter through one shared front-end design, not one-off hacks. Each needs a native speaker's check and must leave English unchanged.
  5. The design language (the vector display) is settled; the look within it can still evolve.
  6. Nothing copyrighted in git; MIT; a deterministic, real-time-safe engine.
  7. Not now: a DAW or sequencer, a light theme, MPE, multi-voice dialogue, translating the interface (the voice speaks and sings many languages; the labels stay in English).

What it does

Male, female and child bodies blend continuously. Everything on screen animates from live engine state: pressure in the airway, airflow, the vocal folds, spectra and formants.

Run it

Needs Rust with the wasm32-unknown-unknown target, and Bun.

bun install
bun run dev        # builds the wasm, then Vite on 127.0.0.1:5317
bun run test       # wasm build, cargo test --workspace, bun test tests

The AudioWorklet needs a secure context, so open it on localhost or over HTTPS (for example tailscale serve). Vercel builds it with vercel.json.

Offline renders and measurement:

cargo run --release -p chimera-cli -- render speech|text|song|notes|beatbox|creature|vowels …
cargo run --release -p chimera-cli -- diagnose       # nasal leakage, formants, bandwidths, pitch
cargo run --release -p chimera-cli -- compare csd    # against a real singer (docs/csd-comparison.md)
cargo run --release -p chimera-cli -- compare tsp    # against 23 real speakers (docs/speech-comparison.md)
bun tools/spectrogram.ts in.wav out.png
bun tools/musicxml-to-notes.ts score.mxl P1 --out p1.json   # a score's part → `render notes` melody

Docs

Contributing: see CONTRIBUTING.md. Agents: see AGENTS.md.

License

MIT. Bundled third-party files keep their own licenses: the CMU Pronouncing Dictionary (public/data/CMUDICT-LICENSE.txt) and the fonts (public/fonts/).

chimera-vocal-synth (jamesondh/chimera-vocal-synth)

Searching Database…

What is this?A voice synthesizer that makes sound by simulating a human-like airway (lungs, vocal folds, branched vocal tract, lips, nose) instead of playing recorded samples. It runs in the browser.
Is it a browser app or a library?It's a browser app. The Rust engine is compiled to WebAssembly for an AudioWorklet, and the same code also runs natively for tests and a 'chimera' CLI.
How do I run it?Install Rust with the wasm32-unknown-unknown target and Bun, then open the app on localhost.
Any catches?The audio engine needs a secure context, so the app must be opened on localhost or over HTTPS, for example with tailscale serve. The README gives no commands for offline renders and measurement.
Can it read text aloud, not just sing?Yes. It takes English text (CMUdict with a letter-to-sound fallback) or IPA and turns it into gestures and prosody, with the gestural score and a consonant chart shown on screen.
What is the bestiary?A set of non-human voices built on the same airway model: a two-voice songbird syrinx, a seal, an alligator and an elephant.
What is the Duet view?Two 3D heads replay a pre-rendered song, and male, female and child bodies blend continuously. Everything on screen animates from live engine state such as airway pressure, airflow and formants.
Creator
jamesondh
Size
18.8 MB
Stars
2.7K
Updated
CommitNov 18
Age
13d
Commits · 90 Days