It doesn't use a language model. It's a small-vocabulary template recognizer plus a statistical test that tells you whether the signal is real.
subvox (joeblau/subvox)
Searching Database…
The project's README from github.com/joeblau/subvox. As one markdown file: README.md.
A macOS prototype that tests whether AirPods Pro head-motion data (CMHeadphoneMotionManager) carries enough information to turn silently mouthed words into text.
It doesn't use a language model. It's a small-vocabulary template recognizer plus a statistical test that tells you whether the signal is real.
brew install xcodegen # once
xcodegen generate
open Subvox.xcodeproj # ⌘R
Or from the command line:
xcodebuild -project Subvox.xcodeproj -scheme Subvox -derivedDataPath build -allowProvisioningUpdates build
open build/Build/Products/Debug/Subvox.app
xcodebuild -project Subvox.xcodeproj -scheme Subvox -derivedDataPath build test
Requires macOS 14+, AirPods Pro or another headphone that supports head tracking, connected as the Mac's audio output. On first launch, grant Motion. The mic and speech prompts only appear when you start the mic.
| Screen | What it does |
|---|---|
| Sensors | Live plots of user acceleration, rotation rate and attitude, the measured sample rate, a mic input picker and level meter, and a recorder for experiment sessions. |
| Train | Pick a word, hold Space, mouth it, release. Each take is stored as a template. Evaluate runs leave-one-out classification and a 500× label-permutation test, then shows accuracy against chance, a p-value and a confusion matrix. The feature checkboxes (acceleration / rotation / attitude) let you test which channels carry the signal. |
| Decode | The motion-to-text screen. Hold Space, or use auto-detect after calibrating the noise floor. Each segment is DTW-matched against your templates and added to the transcript. A live mic transcript runs alongside it for comparison. |
CaptureController). Space-hold, or an energy trigger above a calibrated noise floor with pre-roll and hangover.MotionFeatures). Per frame: user acceleration (3), rotation rate (3), and attitude change since the start of the segment (3). Each channel is mean-centered, and quiet lead-in and lead-out frames are trimmed.DTW, TemplateClassifier). Each channel is z-scaled across the training set. Matching is 1-nearest-neighbour per word, using dynamic time warping with a Sakoe–Chiba band. A segment is rejected if the best distance is above max distance or the margin to the runner-up is below min confidence.Evaluation). The leave-one-out accuracy is compared with the same procedure run on shuffled labels. If p < 0.05, the motion stream distinguishes your words better than chance.Use Record Session in the toolbar or on the Sensors screen. Recordings go to ~/Documents/Subvox/Sessions/<timestamp>-<condition>/:
motion.csv: every motion sample. t is seconds from session start.audio.caf: microphone audio, if the mic is running.events.csv: every Space-held segment, with its label or prediction.meta.json: condition, prompt, and the audio start offset for aligning audio with motion.Suggested runs: the same 5-word script under Normal speech, Whisper and Silent mouthing, 10 reps each. Then, in Train, compare the Evaluate results across feature sets.
The training set lives at ~/Library/Application Support/Subvox/dataset.json.
It doesn't use a language model. It's a small-vocabulary template recognizer plus a statistical test that tells you whether the signal is real.