FunASR README

The project's README from github.com/modelscope/FunASR. As one markdown file: README.md.

(简体中文|English|日本語|한국어)

FunASR

Industrial speech recognition toolkit for offline, streaming, and edge deployment.
ASR · VAD · punctuation · speaker pipelines · emotion and audio-event models · OpenAI-compatible serving

PyPI Stars Downloads Docs MCP Toplist

modelscope%2FFunASR | Trendshift

Quick Start · Model selection · Models · Deployment matrix · Deployment hub · Docs · Benchmark · Contribute

Quick Start

Native Transformers

For Fun-ASR-Nano transcription with the Hugging Face API, start with the Transformers 5.17.0 CPU quickstart. No FunASR toolkit or remote Python code is needed.

Space · Notebook · Python / batch examples

FunASR toolkit and pipelines

Open In Colab

No local setup? Open the Colab quickstart to transcribe a public sample or upload your own audio in a browser.

Found FunASR useful? Star the project so more builders can find it.

# CPU-only installs can use the default PyPI wheels.
pip install torch torchaudio
pip install funasr

For GPU quickstarts, install the PyTorch and torchaudio wheels that match your NVIDIA driver from pytorch.org before installing FunASR. After installation, confirm the GPU is visible:

python - <<'PY'
import torch
print(torch.cuda.is_available())
PY

Only use device="cuda" when this prints True; otherwise use device="cpu" or reinstall PyTorch with the correct CUDA wheel.

FunASR toolkit GPU example: Fun-ASR-Nano (Chinese, English, Japanese, and Chinese dialect groups and regional accents; the separate native Transformers CPU path is linked above):

from funasr import AutoModel

model = AutoModel(model="FunAudioLLM/Fun-ASR-Nano-2512", device="cuda")
result = model.generate(input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav")
print(result[0]["text"])

For the separate 31-language checkpoint, use Fun-ASR-MLT-Nano-2512. Language coverage is checkpoint-specific, so Nano and MLT-Nano should be treated as distinct model choices.

For a CPU-first example with five-language ASR plus emotion and audio-event tags, use SenseVoiceSmall. The pipeline below combines it with FSMN-VAD and CAM++ for speaker-aware VAD segments; these are not native speaker outputs of the SenseVoiceSmall checkpoint. See the SenseVoice paper, Hugging Face checkpoint, and GGUF edge checkpoint.

from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad", spk_model="cam++", device="cpu")
result = model.generate(
    input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav",
    batch_size_s=300,
)

# The AutoModel pipeline returns VAD segments with speaker ids and timestamps:
for seg in result[0]["sentence_info"]:
    print(f"[{seg['start']/1000:.1f}s] Speaker {seg['spk']}: {rich_transcription_postprocess(seg['sentence'])}")

This prints each returned segment's start time in seconds, anonymous speaker index, and text with SenseVoice tags removed. Text and segment boundaries depend on the audio and checkpoint; no fixed transcript is asserted here.

CAM++ extracts spk_embedding vectors. AutoModel clusters those embeddings and assigns speaker indices to VAD segments. Indices are local to a recording, not known-person identities. See the SDK contract for the component and result boundaries. Change to device="cuda" only after verifying a compatible GPU environment as described above.

Scale & deploy the flagship

At scale, accelerate Fun-ASR-Nano with vLLM (batch processing):

from funasr.auto.auto_model_vllm import AutoModelVLLM

model = AutoModelVLLM(model="FunAudioLLM/Fun-ASR-Nano-2512", tensor_parallel_size=1)
results = model.generate(["audio1.wav", "audio2.wav"], language="auto")

Deploy as API server: Local SenseVoice CPU recipe · Nano GPU serving and pinned vLLM setup

Use with AI agents: MCP Server for Claude/Cursor · OpenAI API for LangChain/Dify/AutoGen

Use with voice agents: OpenClaw realtime plugin for self-hosted Talk and Voice Call transcription

Why FunASR?

FunASR is a toolkit: choose the task, checkpoint, and runtime separately. Support in one model or adapter does not imply support in every serving backend.

Task Checkpoint or pipeline Runtime entrypoint Important limitation
File transcription with emotion/event tags SenseVoiceSmall Python AutoModel, CPU or GPU Five-language checkpoint; tags do not identify speakers.
LLM-based file transcription Fun-ASR-Nano AutoModel; split-engine AutoModelVLLM for the documented GPU path Base Nano covers zh/en/ja and Chinese dialects/accents; timestamp support depends on checkpoint and path.
Broader multilingual transcription Fun-ASR-MLT-Nano Python AutoModel Separate 31-language checkpoint; do not transfer its coverage to base Nano.
Chunked live transcription Paraformer-zh-streaming Streaming SDK or runtime WebSocket service Use the streaming checkpoint and per-session cache, not an offline checkpoint.
Speaker-aware file transcription SenseVoiceSmall + FSMN-VAD + CAM++ AutoModel with VAD and embedding clustering Anonymous indices within a recording, not enrolled-speaker identification.
Joint text, timestamps, and speakers MOSS-Transcribe-Diarize, third-party OpenMOSS FunASR adapter or upstream backend in the MOSS guide Offline, recording-local anonymous labels; no external VAD/speaker pipeline for its unified path.
Native CPU/edge transcription Fun-ASR-Nano or SenseVoiceSmall GGUF llama.cpp runtime Requires matching converted weights; GGUF is not a Python AutoModel checkpoint.

See the Model Zoo and deployment matrix for checkpoint, interface, and licensing boundaries. Benchmark on your own audio and hardware before choosing a runtime.

Trying FunASR for the first time? Use the Colab quickstart before setting up a local environment. Choosing a first model? Start with the model selection guide. Planning a switch from Whisper or a cloud ASR provider? Use the migration guide and benchmark example to test representative audio, map features, and roll out safely.

Installation

pip install funasr
From source / Requirements 
git clone https://github.com/modelscope/FunASR.git && cd FunASR
pip install -e ./

Requirements: Python ≥ 3.8. Install PyTorch + torchaudio first (pytorch.org), then pip install funasr.

Model Zoo

This list includes third-party models. OpenMOSS publishes MOSS-Transcribe-Diarize; FunASR provides an adapter, not ownership of its weights. Its unified path is offline, with anonymous labels scoped to each recording, not realtime or known-person identification. Model licenses are separate from the toolkit's MIT license.

Model Task Languages Params Links
Fun-ASR-Nano ASR zh/en/ja + Chinese dialects and accents 800M ⭐ HF / Transformers · HF / FunASR GGUF
Fun-ASR-MLT-Nano ASR 31 languages 800M ⭐ 🤗
SenseVoiceSmall ASR + emotion + events zh/en/ja/ko/yue 234M ⭐ 🤗 GGUF paper
MOSS-Transcribe-Diarize Third-party OpenMOSS: offline ASR + timestamps + anonymous speakers See official card See official card 🤗 guide
Paraformer-zh ASR + timestamps zh/en 220M ⭐ 🤗
Paraformer-zh-streaming Streaming ASR zh/en 220M ⭐ 🤗
Qwen3-ASR ASR, 52 languages multilingual 1.7B usage
GLM-ASR-Nano ASR, 17 languages multilingual 1.5B usage
Whisper-large-v3 ASR + translation multilingual 1550M usage
Whisper-large-v3-turbo ASR + translation multilingual 809M usage
ct-punc Punctuation zh/en 290M ⭐ 🤗
fsmn-vad VAD zh/en 0.4M ⭐ 🤗
cam++ Speaker embeddings (pipeline component) — 7.2M ⭐ 🤗
emotion2vec+large Emotion recognition — 300M ⭐ 🤗

Usage

Python tutorial · SDK parameters and outputs · Training · Model registration

from funasr import AutoModel

# Chinese production (VAD + ASR + punctuation + speaker)
model = AutoModel(model="paraformer-zh", vad_model="fsmn-vad", punc_model="ct-punc", spk_model="cam++", device="cuda")
result = model.generate(input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav", hotword="关键词 20")

# Optional Silero VAD (install first: python -m pip install "funasr[silero]")
model = AutoModel(
    model="paraformer-zh", vad_model="silero-vad", device="cuda",
    vad_kwargs={"silero_threshold": 0.5, "silero_min_silence_duration_ms": 100},
)
result = model.generate(input="audio.wav")

# Streaming real-time (feed audio chunk by chunk)
import soundfile as sf
model = AutoModel(model="paraformer-zh-streaming", device="cuda")
audio, sr = sf.read("speech.wav", dtype="float32")   # 16 kHz mono
chunk_size = [0, 10, 5]                               # 600 ms chunks
chunk_stride = chunk_size[1] * 960
cache = {}
n_chunks = (len(audio) - 1) // chunk_stride + 1
for i in range(n_chunks):
    chunk = audio[i * chunk_stride : (i + 1) * chunk_stride]
    res = model.generate(input=chunk, cache=cache, is_final=(i == n_chunks - 1),
                         chunk_size=chunk_size, encoder_chunk_look_back=4, decoder_chunk_look_back=1)
    if res[0]["text"]:
        print(res[0]["text"], end="", flush=True)

# Emotion recognition
model = AutoModel(model="emotion2vec_plus_large", device="cuda")
result = model.generate(input="audio.wav", granularity="utterance")

CLI (Agent-Friendly)

# Transcribe audio (simplest)
funasr audio.wav

# JSON output (for AI agents)
funasr audio.wav --output-format json

# SRT subtitles
funasr audio.wav --output-format srt --output-dir ./subs

# Speaker diarization + timestamps
funasr audio.wav --spk --timestamps -f json

# Choose model and language
funasr audio.wav --model paraformer --language zh

# Batch transcribe
funasr *.wav --output-format srt --output-dir ./output

Available models: sensevoice (default), paraformer, paraformer-en, fun-asr-nano

Deploy

Start a local SenseVoice CPU service from a fresh directory in a POSIX shell with Python 3.11. This installs the PyPI release into a separate environment, not this source checkout. Keep the unauthenticated service on loopback; use the security guide before exposing it to other clients.

python3.11 -m venv .venv-funasr-http
. .venv-funasr-http/bin/activate
python -m pip install torch torchaudio
python -m pip install funasr fastapi uvicorn python-multipart
python -m pip check
funasr-server --host 127.0.0.1 --port 8000 --model sensevoice --device cpu

Wait for model download and server startup. In a second terminal, use the same directory and curl 7.76+ to download a public Chinese audio sample and transcribe it. The request uses the preloaded model; no fixed transcript or speaker labels are promised.

curl --fail --location https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/BAC009S0764W0121.wav -o sample.wav && \
curl --fail-with-body http://127.0.0.1:8000/v1/audio/transcriptions \
  -F [email protected] \
  -F model=sensevoice \
  -F response_format=verbose_json

For offline joint ASR and anonymous speaker labels (moss-transcribe-diarize), prepare the separate environment in the MOSS service, Docker, Kubernetes, vLLM, SGLang, LocalAI, and FunClip guide →. It is an alternative service, not another command in the CPU environment. Stop the CPU service before reusing port 8000. For Nano GPU serving, follow the pinned split-engine guide and inspect the actual backend logs; selecting a model does not by itself prove that vLLM was loaded.

# Docker streaming service
docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12

CPU / Edge — llama.cpp / GGUF (no GPU, no Python)

Run SenseVoice / Paraformer / Fun-ASR-Nano as a single self-contained binary on CPU and edge devices — this is to FunASR what whisper.cpp is to Whisper, but with ~3× lower CER than whisper.cpp on Chinese. Built-in FSMN-VAD, no Python at runtime.

# Linux / macOS: run from the extracted release directory
bash download-funasr-model.sh sensevoice ./gguf        # or: paraformer | nano
./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
# → 欢迎大家来体验达摩院推出的语音识别模型

# Windows PowerShell: run from the extracted archive root (with the `hf` CLI installed)
hf download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir .\gguf
hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf
.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav
# Use the windows-x64-vulkan package with a current AMD, Intel, or NVIDIA Vulkan driver:
.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav --backend vulkan
# Use the windows-x64-cuda package on RTX 30-class GPUs:
.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav --backend cuda

Use funasr-llamacpp-linux-x64-vulkan.tar.gz on Linux GPU systems with a working Vulkan driver/ICD:

./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav --backend vulkan

The Windows Vulkan ZIP uses the system Vulkan loader supplied by the GPU driver; installing the Vulkan SDK is only necessary when building from source. Both Vulkan packages currently accelerate SenseVoiceSmall.

Tagged releases provide two Windows CUDA packages. The standard windows-x64-cuda ZIP targets CUDA architecture 86, while windows-x64-cuda-blackwell targets architecture 120 (sm_120) for RTX 50 / Blackwell GPUs. Both ZIPs bundle the required cuBLAS DLLs and use the static MSVC runtime, so users need a compatible NVIDIA driver but not a separate CUDA Toolkit installation. CI verifies the architecture and package boundary; it does not prove inference on physical Blackwell hardware.

Prebuilt binaries: Releases · v0.2.6 · Linux Vulkan tarball · Windows Vulkan zip · Windows CUDA zip · Windows Blackwell CUDA zip · Download & quickstart: funasr.com/deploy/llama-cpp · GGUF models: Hugging Face · Docs & benchmarks: runtime/llama.cpp/

OpenAI API example → · [Gradio demo →](./examples/openai_api

FunASR (modelscope/FunASR)

Searching Database…

20.7K ·GitHub
FunASR
这是什么？FunASR 是一个开源的工业级语音识别工具箱，涵盖 ASR、VAD、标点恢复、说话人分离以及情感/音频事件模型，可以离线部署、流式部署、部署到边缘设备，或者以 OpenAI 兼容的 API 形式提供服务。
它是 Python 库、命令行工具，还是服务器？三样都有，合在一个工具箱里：可以用 pip 安装的 Python 库（AutoModel 和 pipelines）、对 AI 代理友好的命令行工具，以及多种服务方式——OpenAI 兼容的 API 服务器、MCP 服务器，还有独立的 llama.cpp 二进制文件。
怎么把它跑起来？最快的方式是用 Colab 快速上手——在浏览器里就能转写音频，本地什么都不用装。本地安装的话，先装 Python 3.8+，再从 pytorch.org 添加 PyTorch 和 torchaudio，然后运行 pip install funasr。
需要 PyTorch 吗？一定要有 GPU 吗？Python 工具箱必须装 PyTorch 和 torchaudio；GPU 不是必需的。SenseVoiceSmall 优先在 CPU 上运行，只有在检查脚本打印 True 之后才应该传 device='cuda'——否则就用 CPU，或者重新安装与 CUDA 版本匹配的 PyTorch 轮子。
挑模型的时候有哪些坑需要注意？语言支持因检查点而异：Fun-ASR-Nano 支持中/英/日加上中文方言，而 Fun-ASR-MLT-Nano 是另一个支持 31 种语言的独立检查点——两者的语言覆盖不能互相通用。说话人标签只是每段录音内的匿名编号，并不能识别出某个具体的人。
在 Windows 上能用吗，还是只支持 Linux？只要能跑 Python 3.8+ 的地方都能安装这个 Python 包，而且还有预编译的 Windows 二进制文件：Vulkan 和 CUDA 的 ZIP 包，其中还包括面向 RTX 50/Blackwell 显卡的 sm120 专门版本。这些包自带 cuBLAS DLL，所以只需要 NVIDIA 驱动，不需要装 CUDA Toolkit。
能完全离线运行吗？可以。离线文件转写用任何检查点都能在本地运行，GGUF/llama.cpp 运行时则以独立二进制文件的形式提供，面向 CPU 和边缘设备，内置 FSMN-VAD——运行时完全不需要 Python。
用 API 服务器时，我的音频会传到哪里？快速上手的服务没有鉴权，默认只应在 loopback 上使用；README 提醒在把它暴露给其他客户端之前先查阅安全指南。所以只要你没有主动把服务器开放出去，音频就一直留在你自己的电脑上。
所有东西都是 MIT 许可证吗？工具箱本身是 MIT 许可，但 Model Zoo 里的每个模型都有自己的许可证——请查看部署矩阵了解各检查点的许可边界。像 MOSS-Transcribe-Diarize 这样的第三方模型是通过适配器接入的，并不是 FunASR 自己的权重。
能实时转写现场音频吗？可以——通过 Streaming SDK 或运行时的 WebSocket 服务，可以在 Paraformer-zh-streaming 上进行分块的实时转写。必须使用带会话缓存的流式检查点，离线检查点做不了这件事。
和 Whisper 相比怎么样？README 把 llama.cpp/GGUF 运行时定位为 FunASR 对 whisper.cpp 的回应，声称在中文上的 CER 比 whisper.cpp 低约 3 倍。Model Zoo 里也能跑 Whisper-large-v3 及其 turbo 检查点，还提供了迁移指南，帮你从 Whisper 或云 ASR 服务商切换过来。
Model Zoo 里有哪些模型？有从 234M 的 SenseVoiceSmall（支持中/英/日/韩/粤，还带情感和音频事件标签）到 1.7B 的 Qwen3-ASR（支持 52 种语言）等 ASR 检查点，此外还有 Whisper-large-v3、Paraformer-zh、做标点恢复的 ct-punc、fsmn-vad、cam++ 说话人嵌入，以及用于情感识别的 emotion2vec+large。
怎么把它接到 AI 代理上？可以把它部署成 OpenAI 兼容的 API 给 LangChain、Dify 和 AutoGen 使用，通过 MCP Server 连接 Claude 和 Cursor，用 vLLM 做批量处理，或者用 OpenClaw 实时插件来做自托管的语音对话和电话转写。
What are similar projects?WhisperLiveKit, VibeVoice, Qwen3-ASR, DeepSpeech, hayamimi, speechbrain, Whisper, TalkLikeYou-Streaming-RealTime-Demo
Which topics is it in?emotion-recognition, voice-activity-detection, speaker-diarization, asr, llama-cpp, transcription, gguf, speech-to-text, speech-recognition, vllm, openai-compatible, multilingual
Creator
modelscope
Size
124.9 MB
Stars
20.7K
Updated
CommitOct 16Releasev1.4.16Sep 18
Age
3y
Commits · 90 Days