VoxCPM README

The project's README from github.com/OpenBMB/VoxCPM. As one markdown file: README.md.

VoxCPM2: Tokenizer-Free TTS for Multilingual Speech Generation, Creative Voice Design, and True-to-Life Cloning

English | 中文

Project Page Live Playground Documentation Hugging Face ModelScope DemoPage VoxCPM2 Technical Report

VoxCPM Logo

OpenBMB%2FVoxCPM | Trendshift

👋 Join our community for discussion and support!
Feishu  |  Discord  |  📚 MiniCPM Wiki

VoxCPM is a tokenizer-free Text-to-Speech system that directly generates continuous speech representations via an end-to-end diffusion autoregressive architecture, bypassing discrete tokenization to achieve highly natural and expressive synthesis.

VoxCPM2 is the latest major release — a 2B parameter model trained on over 2 million hours of multilingual speech data, now supporting 30 languages, Voice Design, Controllable Voice Cloning, and 48kHz studio-quality audio output. Built on a MiniCPM-4 backbone.

✨ Highlights

🌍 Supported Languages (30)
Arabic, Burmese, Chinese, Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Khmer, Korean, Lao, Malay, Norwegian, Polish, Portuguese, Russian, Spanish, Swahili, Swedish, Tagalog, Thai, Turkish, Vietnamese

Chinese Dialect: 四川话, 粤语, 吴语, 东北话, 河南话, 陕西话, 山东话, 天津话, 闽南话

News

Contents

🚀 Quick Start

Installation

pip install voxcpm

Requirements: Python ≥ 3.10 (<3.13), PyTorch ≥ 2.5.0, CUDA ≥ 12.0. See Quick Start Docs for details.

Python API

🗣️ Text-to-Speech

from voxcpm import VoxCPM
import soundfile as sf

model = VoxCPM.from_pretrained(
  "openbmb/VoxCPM2",
  load_denoiser=False,
)

wav = model.generate(
    text="VoxCPM2 is the current recommended release for realistic multilingual speech synthesis.",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("demo.wav", wav, model.tts_model.sample_rate)
print("saved: demo.wav")

If you prefer downloading from ModelScope first, you can use:

pip install modelscope

from modelscope import snapshot_download
snapshot_download("OpenBMB/VoxCPM2", local_dir='./pretrained_models/VoxCPM2') # specify the local directory to save the model

from voxcpm import VoxCPM
import soundfile as sf
model = VoxCPM.from_pretrained("./pretrained_models/VoxCPM2", load_denoiser=False)

wav = model.generate(
    text="VoxCPM2 is the current recommended release for realistic multilingual speech synthesis.",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("demo.wav", wav, model.tts_model.sample_rate)

🎨 Voice Design

Create a voice from a natural-language description — no reference audio needed. Format: put the description in parentheses at the start of text(e.g. "(your voice description)The text to synthesize."):

wav = model.generate(
    text="(A young woman, gentle and sweet voice)Hello, welcome to VoxCPM2!",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("voice_design.wav", wav, model.tts_model.sample_rate)

🎛️ Controllable Voice Cloning

Upload a reference audio. The model clones the timbre, and you can still use control instructions to adjust speed, emotion, or style.

wav = model.generate(
    text="This is a cloned voice generated by VoxCPM2.",
    reference_wav_path="path/to/voice.wav",
)
sf.write("clone.wav", wav, model.tts_model.sample_rate)

wav = model.generate(
    text="(slightly faster, cheerful tone)This is a cloned voice with style control.",
    reference_wav_path="path/to/voice.wav",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("controllable_clone.wav", wav, model.tts_model.sample_rate)

🎙️ Ultimate Cloning

Provide both the reference audio and its exact transcript for audio-continuation-based cloning with every vocal nuance reproduced. For maximum cloning similarity, pass the same reference clip to both reference_wav_path and prompt_wav_path as shown below:

wav = model.generate(
    text="This is an ultimate cloning demonstration using VoxCPM2.",
    prompt_wav_path="path/to/voice.wav",
    prompt_text="The transcript of the reference audio.",
    reference_wav_path="path/to/voice.wav", # optional, for better simliarity 
)
sf.write("hifi_clone.wav", wav, model.tts_model.sample_rate)

🔄 Streaming API

import numpy as np

chunks = []
for chunk in model.generate_streaming(
    text="Streaming text to speech is easy with VoxCPM!",
):
    chunks.append(chunk)
wav = np.concatenate(chunks)
sf.write("streaming.wav", wav, model.tts_model.sample_rate)

CLI Usage

# Voice design (no reference audio needed)
voxcpm design \
  --text "VoxCPM2 brings studio-quality multilingual speech synthesis." \
  --output out.wav

# Controllable voice cloning with style control
voxcpm design \
  --text "VoxCPM2 brings studio-quality multilingual speech synthesis." \
  --control "Young female voice, warm and gentle, slightly smiling" \
  --seed 42 \
  --output out.wav

# Voice cloning (reference audio)
voxcpm clone \
  --text "This is a voice cloning demo." \
  --reference-audio path/to/voice.wav \
  --output out.wav

# Ultimate cloning (prompt audio + transcript)
voxcpm clone \
  --text "This is a voice cloning demo." \
  --prompt-audio path/to/voice.wav \
  --prompt-text "reference transcript" \
  --reference-audio path/to/voice.wav \ # optional, for better simliarity
  --output out.wav

# Batch processing
voxcpm batch --input examples/input.txt --output-dir outs

# Optional post-generation timestamps with stable-ts
pip install "voxcpm[timestamps]"
voxcpm design \
  --text "VoxCPM2 brings studio-quality multilingual speech synthesis." \
  --output out.wav \
  --timestamps \
  --timestamp-level word \
  --timestamp-language en

# Character timestamps are best-effort and are derived from word alignment
voxcpm design \
  --text "欢迎使用 VoxCPM2。" \
  --output out.wav \
  --timestamps \
  --timestamp-level char \
  --timestamp-language zh

# Help
voxcpm --help

Web Demo

python app.py --port 8808  # then open in browser: http://localhost:8808

Use --device to choose the runtime device:

python app.py --device auto

Supported values are auto, cpu, mps, cuda, and cuda:N. On Apple Silicon Macs, auto uses MPS when available.

🚢 Production Deployment (Nano-vLLM)

For high-throughput serving, use Nano-vLLM-VoxCPM — a dedicated inference engine built on Nano-vLLM with concurrent request support and an async API.

pip install nano-vllm-voxcpm

from nanovllm_voxcpm import VoxCPM
import numpy as np, soundfile as sf

server = VoxCPM.from_pretrained(model="/path/to/VoxCPM", devices=[0])
chunks = list(server.generate(target_text="Hello from VoxCPM!"))
sf.write("out.wav", np.concatenate(chunks), 48000)
server.stop()

RTF as low as ~0.13 on NVIDIA RTX 4090 (vs ~0.3 with the standard PyTorch implementation), with support for batched concurrent requests and a FastAPI HTTP server. See the Nano-vLLM-VoxCPM repo for deployment details.

🏭 Production Serving (vLLM-Omni)

For production multi-tenant deployments, use vLLM-Omni — the official vLLM project's omni-modal extension with native VoxCPM2 support. PagedAttention KV cache, continuous batching, and a drop-in OpenAI-compatible /v1/audio/speech endpoint.

# Install from source (latest main — vllm-omni is rapidly evolving)
uv pip install vllm==0.19.0 --torch-backend=auto
git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni
uv pip install -e .

See the vLLM-Omni installation guide for other platforms (ROCm, XPU, MUSA, NPU) and Docker images.

# Launch an OpenAI-compatible TTS server (--omni enables omni-modal serving)
vllm serve openbmb/VoxCPM2 --omni --port 8000

# Call it from any OpenAI client
curl http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model":"openbmb/VoxCPM2","input":"Hello from VoxCPM2 on vLLM-Omni!","voice":"default"}' \
  --output out.wav

Built on the upstream vLLM scheduler, with batched concurrent requests, streaming chunk delivery, and multi-GPU deployment out of the box. See the VoxCPM2 example for full deployment recipes.

📱 On-Device Inference (llama.cpp-omni)

For on-device / edge deployment without Python, use llama.cpp-omni — a high-performance C++ inference engine built on llama.cpp, with native VoxCPM2 GGUF support on CPU / Metal / CUDA / Vulkan.

1. Download GGUF weights from HuggingFace | ModelScope — you need one BaseLM (F16 or Q8_0) + the Acoustic file. Q8_0 halves the download with negligible quality loss.

2. Build

git clone https://github.com/tc-mb/llama.cpp-omni.git && cd llama.cpp-omni
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --target voxcpm2-cli -j

CMake auto-detects Metal (macOS) or CUDA (Linux with NVIDIA GPU).

3. Run

# Basic TTS
./build/bin/voxcpm2-cli \
    -t "Hello, this is VoxCPM2 running through llama.cpp-omni." \
    -o output.wav VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf

# Voice cloning (reference audio)
./build/bin/voxcpm2-cli \
    -t "Cloned voice." -r speaker.wav -o clone.wav \
    VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf

# Ultimate cloning (reference audio + transcript)
./build/bin/voxcpm2-cli \
    -t "Target text." --prompt-wav speaker.wav --prompt-text "transcript of speaker.wav" \
    -o clone.wav VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf

RTF ~1.76 (Q8_0) on Apple M4 Pro / Metal. Key flags: --cfg (guidance scale), --timesteps (CFM steps), --seed, --temperature, --stream. See the llama.cpp-omni repo and GGUF weights page for full details.

Full parameter reference, multi-scenario examples, and voice cloning tips → Quick Start Guide | Usage Guide | Cookbook

📦 Models & Versions

VoxCPM2 VoxCPM1.5 VoxCPM-0.5B
Status 🟢 Latest Stable Legacy
Backbone Parameters 2B 0.6B 0.5B
Audio Sample Rate 48kHz 44.1kHz 16kHz
LM Token Rate 6.25Hz 6.25Hz 12.5Hz
Languages 30 2 (zh, en) 2 (zh, en)
Cloning Mode Isolated Reference & Continuation Continuation only Continuation only
Voice Design ✅ — —
Controllable Voice Cloning ✅ — —
SFT / LoRA ✅ ✅ ✅
RTF (RTX 4090) ~0.30 ~0.15 ~0.17

VoxCPM (OpenBMB/VoxCPM)

Searching Database…

39.5K ·GitHub
VoxCPM
这是什么？VoxCPM 是一个无需分词器的文本转语音系统，它通过端到端的扩散自回归架构，直接生成连续的语音表示，跳过离散分词环节，从而得到更自然、更有表现力的语音。VoxCPM2 是最新的 2B 参数版本，基于超过 200 万小时的多语言音频训练。
这是我可以调用的 Python 库，还是现成的应用？它可以作为 Python API、CLI 工具或 Web Demo 使用。此外，还有通过 vLLM-Omni 实现的生产级部署方案，以及面向没有 Python 环境的设备端 C++ 引擎（llama.cpp-omni）。
我该怎么把它跑起来？需要 Python 3.10 或更高版本，然后安装软件包，并通过 Python API、CLI 或 Web Demo 运行。若用于生产环境，vLLM-Omni 提供兼容 OpenAI 的 '/v1/audio/speech' 接口；llama.cpp-omni 则提供基于 GGUF 的设备端推理。
要获得最佳性能，是否必须依赖 vLLM 或 llama.cpp？标准方案只需要 PyTorch 和 Python 3.10+。若想要更快的实时推理，可以使用 Nano-vLLM 或官方的 vLLM-Omni 服务栈加速，也可以通过 llama.cpp-omni 的 GGUF 引擎完全离线运行。
有没有不依赖 Python、可以完全离线运行的版本？有。llama.cpp-omni 是一个独立的 C++ 推理引擎，原生支持 VoxCPM2 的 GGUF 格式，可在 CPU、Metal、CUDA 和 Vulkan 上运行，适用于无需 Python 的设备端或边缘部署。
没有 NVIDIA 显卡的 Mac 能运行吗？可以。llama.cpp-omni 的构建会自动检测环境：macOS 上使用 Metal，装有 NVIDIA 显卡的 Linux 上使用 CUDA。据报告，在 Apple M4 Pro 上通过 Metal 运行 Q80 量化版本时，RTF 约为 1.76。
我可以把生成的声音用于商业用途吗？可以。权重和代码均以 Apache-2.0 协议发布，并明确说明可免费用于商业用途。
要实现实时语音生成，是否必须有 NVIDIA 显卡？NVIDIA 显卡速度最快，在 RTX 4090 上 RTF 低至约 0.3，搭配 Nano-vLLM 或 vLLM-Omni 时可降至约 0.13。不过，通过 llama.cpp-omni 引擎，它也可以在 CPU、Apple Metal 和 Vulkan 上运行，适用于设备端或边缘场景。
什么是 Voice Design（声音设计）？Voice Design 让你仅凭自然语言描述就能创建一个全新的声音，描述内容可以包括性别、年龄、语气、情绪和语速，完全不需要任何参考音频。
Controllable Cloning 和 Ultimate Cloning 有什么不同？Controllable Cloning 只需一段简短的参考音频，并可选择性地指定情绪和语速等风格指导，即可克隆声音；Ultimate Cloning 则还需要参考音频的文字稿，使模型能以最高保真度复现所有人声细节，包括音色、节奏、情绪和风格。
支持多少种语言和方言？VoxCPM2 支持 30 种语言，包括英语、中文、阿拉伯语、印地语、日语和俄语，另外还支持九种中国方言，如四川话、粤语和吴语。
开箱即用能获得什么样的音质？VoxCPM2 接受 16kHz 的参考音频，但输出 48kHz 的录音棚级音质。这得益于 AudioVAE V2 非对称的编码/解码设计，内置超分辨率功能，无需外部上采样器。
What are similar projects?MOSS-TTS-Nano, MOSS-TTS, TTS, Speech, DramaBox, OmniVoice, FireRedTTS3, Index-Translate
Which topics is it in?tts-model, deeplearning, speech, llama-cpp, on-device, voice-cloning, speech-synthesis, gguf, lora, tts, vllm, text-to-speech
Creator
OpenBMB
Size
4.1 MB
Stars
39.5K
Updated
CommitOct 08Release2.0.3May 11
Age
1y
Commits · 90 Days