Audio8_TTS README

项目的 README,来自 github.com/Edge0-AI/Audio8_TTS。合并为一个 markdown 文件:README.md。

Audio8

Audio8 TTS

Compact text-to-speech models for zero-shot voice cloning.

GitHub License

English · 中文

0.6B family · 0.1B family · llama.cpp / GGUF

2026-11-16 · llama.cpp / GGUF — Native local inference, voice cloning and streaming for Realistic Air, with one shared F32 model bundle. Runbook · GGUF files.

Releases

0.6B family
Audio8-TTS-Realistic-Air — More realistic speech. More languages. Still 0.6B.

Model Demo arXiv

Audio8-TTS-Realistic-Air builds on Preview with substantially improved speech realism, retaining the same compact 0.6B architecture and zero-shot voice cloning. It supports 23 languages, adding 12 languages compared with Preview.

Release Focus and languages Model Listen Guide
Realistic Air Realism and prosody · 23 languages HF Demo Details

ONNX: There is currently no Realistic Air ONNX export.

Native deployment: llama.cpp / GGUF runbook · Shared model folder.

0.6B details: languages, quick start, and evaluation

Realistic Air language support

Coverage Languages
11 languages supported in Preview Cantonese, Chinese, Dutch, English, French, German, Italian, Japanese, Korean, Polish, Spanish
12 additions in Realistic Air Arabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish, Vietnamese

Note: Quality varies by language; benchmark coverage differs from the supported-language list. Per-language results.

Realistic Air quick start

Use Python 3.10+ and a CUDA-capable GPU. Run from the repository root. The explicit model ID selects Realistic Air independently of the scripts' existing Preview default.

git clone https://github.com/Edge0-AI/Audio8_TTS.git
cd Audio8_TTS
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

python audio8_tts_infer.py \
  --model Edge0/Audio8-TTS-Realistic-Air \
  --text "Welcome to Audio8 TTS. It is wonderful to hear from you." \
  --output outputs/realistic.wav

For voice cloning, add --reference-audio reference.wav --reference-text "The exact transcript of the recording.".

Installation, cloning and batch inference · Supervised fine-tuning

0.6B evaluation summary

Error rates are percentages (lower is better). SIM uses a 0–1 scale (higher is better). Neither directly measures speech realism.

Audio8-TTS-Preview and Audio8-TTS-Realistic-Air, both 0.6B:

Benchmark Coverage Preview error % ↓ Realistic Air error % ↓ Preview SIM ↑ Realistic Air SIM ↑
Seed-TTS English / Chinese 1.23 1.25 0.682 0.704
CV3 8 shared languages 4.80 4.56 — 0.663
CV3 9 core languages — 4.86 — 0.661
MiniMax 24 evaluated languages — 5.89 — 0.754

Note: Values are unweighted language macro averages. Preview results are calculated from its published model card, with SIM converted from percentages to a 0–1 scale. The two releases were evaluated separately. A dash means the corresponding result was not reported on Preview's card.

Seed-TTS averages English WER and Chinese CER, excluding hard Chinese. The shared CV3 subset is Chinese, English, Japanese, Korean, German, French, Spanish, and Italian; the 9-language set also includes Russian. CV3 averages exclude hard subsets and the 13 additional languages. MiniMax averages all 24 evaluated languages, including languages outside the supported list.

Full Realistic Air evaluation and comparison notes · Original Preview evaluation

Previous release · Preview 0.6B

Realistic Air builds on Preview. The original checkpoint, 11-language coverage, evaluation, and listening page remain available for Preview users.

Release Model Listen Guide Deployment
Preview 0.6B HF Demo Details ONNX INT4

0.1B family
Audio8-TTS-0.1B — How small a zero-shot cloning TTS can be?

Model 0.1B ONNX INT8 Demo 0.1B

Meet Audio8-TTS-0.1B: a compact, portable text-to-speech model with zero-shot voice cloning.

Designed for a smaller deployment footprint, with Chinese and English as its primary languages. The dedicated ONNX INT8 runtime supports CPU inference, streaming, and local web/HTTP serving.

Release Focus and languages Model Listen Guide
Preview 0.1B Compact hybrid · primarily Chinese / English HF Demo Details

0.1B quick start · CPU / ONNX runtime · 0.1B deployment · Model license


Shared tools

Installation and inference · Supervised fine-tuning

The shared tools accept an explicit model ID. Use the quick-start and deployment instructions for your chosen release.

llama.cpp / GGUF

The native C++ route for Realistic Air targets local applications, desktop deployment and embedded integration, without Python, PyTorch or ONNX Runtime at inference time. It provides speech generation, reference-audio voice cloning, WAV output and HTTP/streaming through the Audio8 runtime linked below.

Entry Contents
Standalone runbook Download, build, generate, serve, stream and troubleshoot
Implementation and converter Pinned upstream patch, source verification and F32 export
HF model folder One shared bundle across platforms; four payloads total 3.76 GB

F32 is the default. For GPU services with dynamic batching, see SGLang Omni below.

Note: Use CPU on macOS; Metal is experimental. The Windows build instructions have not yet been tested on a Windows machine.

SGLang Omni Serving

SGLang Omni provides paged attention, dynamic batching, and an OpenAI-compatible speech API. The 0.6B family and the 0.1B hybrid model have their own configurations. Existing performance results remain paired with their original model, hardware, and test conditions in the deployment guide.

Deployment matrix, installation and server configuration · API, streaming and troubleshooting

License and acknowledgements

The repository code and both 0.6B releases use Apache License 2.0.

The 0.1B base checkpoint and its ONNX INT8 export both use Audio8 Community License v1.0. Commercial use is royalty-free for entities whose annual revenue, including parent companies, subsidiaries, and affiliates, is below US$2,000,000, subject to the license conditions; at or above that threshold, a separate written commercial license is required. See the 0.1B license guide and ONNX model license for details.

See NOTICE for attribution. We thank the Fish Audio team for the DualAR architecture used in Fish Audio S2 Pro.

Audio8_TTS (Edge0-AI/Audio8_TTS)

正在搜索数据库…

这是什么?Audio8TTS 是一个紧凑的多语言文本转语音模型,参数量为 0.6B,支持零样本语音克隆。它附带 Hugging Face 推理代码、命令行工具,以及一套独立的 SFT 训练流程。
这是现成的应用,还是需要我自己运行的模型?它是一个 Hugging Face 模型检查点加上 Python 推理代码,不是图形界面应用。你可以通过命令行脚本运行,也可以使用随包提供的 ONNX CPU 部署方案,或者使用 SGLang Omni 服务适配器,提供兼容 OpenAI 的服务。
我该怎么把它跑起来?从 Hugging Face 下载模型检查点,并将其放入仓库的 model/ 目录(预期路径为 model/audio8tts06Bpreview/),然后用 audio8ttsinfer.py 运行推理;你也可以把 --model 指向任意 Hugging Face 模型 ID。
运行它需要什么?主要运行方式推荐使用 Python 3.10+ 和支持 CUDA 的 GPU。纯 CPU 的 ONNX Runtime 部署不需要 PyTorch 或 Transformers,而 SGLang Omni 服务适配器则需要经过测试的特定版本的 SGLang Omni、SGLang、PyTorch 和 Transformers。
用之前有什么需要注意的地方吗?这是一个预览版(Preview)版本,语言覆盖范围有意做了限制。建议主要使用其中 11 种推荐语言,并将每段输入控制在 150 个字符以内以获得最佳效果(较长文本请拆分成段)。
它需要特定的 GPU,还是也能在 CPU 上运行?GPU 推理面向 NVIDIA 显卡(SGLang 适配器针对 H20/H100 等 Hopper GPU 做了调优,并能自动检测 RTX 5090 等消费级 Blackwell 显卡以切换后端)。同时也提供仅 CPU 的 INT4 ONNX Runtime 方案,已在 Apple M2 上测试,占用约 1 GiB 内存。
哪些语言在这个预览版模型上效果最好?预览版检查点针对 11 种语言做了调优:粤语、中文、荷兰语、英语、法语、德语、意大利语、日语、韩语、波兰语和西班牙语。
它采用什么架构?它采用受 Fish Audio S2 Pro 启发的 DualAR 架构:慢速 AR Transformer 为每个音频帧预测语义 token,快速 AR Transformer 则基于该状态预测编解码器的码本,并由随包提供的神经编解码器负责编码和解码。
有更小或更快的版本吗?有。Audio8-TTS-0.1B 将纯注意力的慢速主干替换为 Falcon-H1 混合架构(Mamba 2 SSM + 注意力),会根据 config.json 自动检测,同时与 0.6B 模型共享快速码本 AR 和声码器。
通过 SGLang 适配器提供服务的速度如何?在单张 NVIDIA H20 上,使用 BF16 权重和 CUDA Graphs,热启动后的单路流式延迟 p50 为 0.691s,RTF 为 0.116,生成约 6 秒音频。
作者
Edge0-AI
大小
107.3 MB
星标
2.5K
更新
提交Nov 16
时长
3个月
提交 · 90 天