F5-TTS README

The project's README from github.com/SWivid/F5-TTS. As one markdown file: README.md.

F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching

python arXiv demo hfspace msspace lab lab lab

F5-TTS: Diffusion Transformer with ConvNeXt V2, faster trained and inference.

E2 TTS: Flat-UNet Transformer, closest reproduction from paper.

Sway Sampling: Inference-time flow step sampling strategy, greatly improves performance

Thanks to all the contributors !

News

Installation

Create a separate environment if needed

# Create a conda env with python_version>=3.10  (you could also use virtualenv)
conda create -n f5-tts python=3.11
conda activate f5-tts

# Install FFmpeg if you haven't yet
conda install ffmpeg

Install PyTorch with matched device

NVIDIA GPU 
# Install pytorch with your CUDA version, e.g.
pip install torch==2.8.0+cu128 torchaudio==2.8.0+cu128 --extra-index-url https://download.pytorch.org/whl/cu128

# And also possible previous versions, e.g.
pip install torch==2.4.0+cu124 torchaudio==2.4.0+cu124 --extra-index-url https://download.pytorch.org/whl/cu124
# etc.
AMD GPU 
# Install pytorch with your ROCm version (Linux only), e.g.
pip install torch==2.9.1+rocm7.2 torchaudio==2.9.1+rocm7.2 --extra-index-url https://download.pytorch.org/whl/rocm7.2

# For older GPUs (RDNA1/2/3 only):
# pip install torch==2.5.1+rocm6.2 torchaudio==2.5.1+rocm6.2 --extra-index-url https://download.pytorch.org/whl/rocm6.2

Note: RDNA 3.5 and RDNA 4 GPUs (Radeon 8050S/8060S, RX 9060/9070 series) require ROCm 7.x — these architectures (gfx1151/gfx1201) are not included in ROCm 6.x (6.2 compatibility matrix vs 7.2 compatibility matrix). Using ROCm 6.x on these GPUs causes HIP error: invalid device function (#1236).

Intel GPU 
# Install pytorch with your XPU version, e.g.
# Intel® Deep Learning Essentials or Intel® oneAPI Base Toolkit must be installed
pip install torch torchaudio --index-url https://download.pytorch.org/whl/test/xpu

# Intel GPU support is also available through IPEX (Intel® Extension for PyTorch)
# IPEX does not require the Intel® Deep Learning Essentials or Intel® oneAPI Base Toolkit
# See: https://pytorch-extension.intel.com/installation?request=platform
Apple Silicon 
# Install the stable pytorch, e.g.
pip install torch torchaudio

Then you can choose one from below:

1. As a pip package (if just for inference)

pip install f5-tts

2. Local editable (if also do training, finetuning)

git clone https://github.com/SWivid/F5-TTS.git
cd F5-TTS
# git submodule update --init --recursive  # (optional, if use bigvgan as vocoder)
pip install -e .

Docker usage also available

# Build from Dockerfile
docker build -t f5tts:v1 .

# Run from GitHub Container Registry
docker container run --rm -it --gpus=all --mount 'type=volume,source=f5-tts,target=/root/.cache/huggingface/hub/' -p 7860:7860 ghcr.io/swivid/f5-tts:main

# Quickstart if you want to just run the web interface (not CLI)
docker container run --rm -it --gpus=all --mount 'type=volume,source=f5-tts,target=/root/.cache/huggingface/hub/' -p 7860:7860 ghcr.io/swivid/f5-tts:main f5-tts_infer-gradio --host 0.0.0.0

Runtime

Deployment solution with Triton and TensorRT-LLM.

Benchmark Results

Decoding on a single L20 GPU, using 26 different prompt_audio & target_text pairs, 16 NFE.

Model Concurrency Avg Latency RTF Mode
F5-TTS Base (Vocos) 2 253 ms 0.0394 Client-Server
F5-TTS Base (Vocos) 1 (Batch_size) - 0.0402 Offline TRT-LLM
F5-TTS Base (Vocos) 1 (Batch_size) - 0.1467 Offline Pytorch

See detailed instructions for more information.

Inference

1. Gradio App

Currently supported features:

# Launch a Gradio app (web interface)
f5-tts_infer-gradio

# Specify the port/host
f5-tts_infer-gradio --port 7860 --host 0.0.0.0

# Launch a share link
f5-tts_infer-gradio --share
NVIDIA device docker compose file example 
services:
  f5-tts:
    image: ghcr.io/swivid/f5-tts:main
    ports:
      - "7860:7860"
    environment:
      GRADIO_SERVER_PORT: 7860
    entrypoint: ["f5-tts_infer-gradio", "--port", "7860", "--host", "0.0.0.0"]
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

volumes:
  f5-tts:
    driver: local

2. CLI Inference

# Run with flags
# Leave --ref_text "" will have ASR model transcribe (extra GPU memory usage)
f5-tts_infer-cli --model F5TTS_v1_Base \
--ref_audio "provide_prompt_wav_path_here.wav" \
--ref_text "The content, subtitle or transcription of reference audio." \
--gen_text "Some text you want TTS model generate for you."

# Run with default setting. src/f5_tts/infer/examples/basic/basic.toml
f5-tts_infer-cli
# Or with your own .toml file
f5-tts_infer-cli -c custom.toml

# Multi voice. See src/f5_tts/infer/README.md
f5-tts_infer-cli -c src/f5_tts/infer/examples/multi/story.toml

Training

1. With Hugging Face Accelerate

Refer to training & finetuning guidance for best practice.

2. With Gradio App

# Quick start with Gradio web interface
f5-tts_finetune-gradio

Read training & finetuning guidance for more instructions.

Evaluation

Development

Use pre-commit to ensure code quality (will run linters and formatters automatically):

pip install pre-commit
pre-commit install

When making a pull request, before each commit, run: 

pre-commit run --all-files

Note: Some model components have linting exceptions for E722 to accommodate tensor notation.

Acknowledgements

Citation

If our work and codebase is useful for you, please cite as:

@article{chen-etal-2024-f5tts,
      title={F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching}, 
      author={Yushen Chen and Zhikang Niu and Ziyang Ma and Keqi Deng and Chunhui Wang and Jian Zhao and Kai Yu and Xie Chen},
      journal={arXiv preprint arXiv:2410.06885},
      year={2024},
}

License

Our code is released under MIT License. The pre-trained models are licensed under the CC-BY-NC license due to the training data Emilia, which is an in-the-wild dataset. Sorry for any inconvenience this may cause.

F5-TTS (SWivid/F5-TTS)

Searching Database…

15.4K ·GitHub
F5-TTS
What is this?F5-TTS is a text-to-speech system that generates fluent, faithful speech using flow matching with a Diffusion Transformer (ConvNeXt V2 backbone), and it supports cloning a speaker's voice from a short reference clip.
Is this a Python library, a ready app, or both?It's a pip-installable Python package for inference, plus a bundled Gradio web UI and CLI; if you clone the repo as a local editable install you also get the training and finetuning code.
How do I get it running?Create a Python environment, install the PyTorch build matching your device, then either pip-install F5-TTS for inference or do a local editable install for training/finetuning; a Docker image is also provided.
Do I need anything besides Python installed first?Yes, you need a PyTorch build matched to your hardware (NVIDIA CUDA, AMD ROCm, Intel, or Apple Silicon) before installing F5-TTS itself.
Any GPU install pitfalls I should know about?Newer AMD RDNA 3.5/4 cards (Radeon 8050S/8060S, RX 9060/9070 series) need ROCm 7.x specifically — running them on ROCm 6.x triggers a HIP invalid-device-function error.
Which GPUs does it actually run well on?It works with NVIDIA, AMD, and Intel GPUs as well as Apple Silicon, each with its own matched PyTorch install path.
What can I do in the included Gradio app?It offers basic chunked TTS, multi-style/multi-speaker generation, a voice chat mode powered by Qwen2.5-3B-Instruct, and custom inference with extra language support.
What is 'Sway Sampling'?It's an inference-time flow-step sampling strategy F5-TTS uses to noticeably improve generation quality and speed.
How fast is production inference with this?On a single L20 GPU served through Triton and TensorRT-LLM, F5-TTS Base reaches an RTF of about 0.04 at 253ms average latency, versus roughly 0.15 RTF for plain offline PyTorch inference.
Why did it adopt a Diffusion Transformer instead of a UNet like E2 TTS?The DiT + ConvNeXt V2 design (F5-TTS) was built to train and infer faster than the Flat-UNet Transformer used by E2 TTS, which the repo also reproduces for comparison.
What are similar projects?OmniVoice, FireRedAudio, moshi, DramaBox, bark, FireRedTTS3, VoxCPM, fish-speech
Which topics is it in?amd-gpu, diffusion, voice-cloning, speech-synthesis, gradio, text-to-speech, transformer, apple-silicon, audio, huggingface, cuda, pytorch
Creator
SWivid
Size
2.4 MB
Stars
15.4K
Updated
CommitOct 14Release1.1.22Jul 23
Age
2y
Commits · 90 Days