LongCat-Video README

The project's README from github.com/meituan-longcat/LongCat-Video. As one markdown file: README.md.

LongCat-Video

LongCat-Video
LongCat-Video
LongCat-Video-Avatar 1.5
placeholder

Model Introduction

We introduce LongCat-Video, a foundational video generation model with 13.6B parameters, delivering strong performance across Text-to-Video, Image-to-Video, and Video-Continuation generation tasks. It particularly excels in efficient and high-quality long video generation, representing our first step toward world models.

Key Features

For more detail, please refer to the comprehensive LongCat-Video Technical Report.

🎥 Teaser Video

🔥 Latest News!!

Quick Start

Installation

Clone the repo:

git clone --single-branch --branch main https://github.com/meituan-longcat/LongCat-Video
cd LongCat-Video

Install dependencies:

# create conda environment
conda create -n longcat-video python=3.10
conda activate longcat-video

# install torch (configure according to your CUDA version)
pip install torch==2.6.0+cu124 torchvision==0.21.0+cu124 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124

# install flash-attn-2
pip install ninja 
pip install psutil 
pip install packaging 
pip install flash_attn==2.7.4.post1

# install other requirements
pip install -r requirements.txt

# install longcat-video-avatar requirements
conda install -c conda-forge librosa
conda install -c conda-forge ffmpeg
pip install -r requirements_avatar.txt

FlashAttention-2 is enabled in the model config by default; you can also change the model config ("./weights/LongCat-Video/dit/config.json") to use FlashAttention-3 or xformers once installed.

Model Download

Models Description Download Link
LongCat-Video foundational video generation 🤗 Huggingface
LongCat-Video-Avatar single- and multi-character audio-driven video generation (wav2vec2) 🤗 Huggingface
LongCat-Video-Avatar-1.5 upgraded avatar model with Whisper-large-v3 audio encoder, distillation-based fast inference 🤗 Huggingface

Download models using huggingface-cli:

pip install "huggingface_hub[cli]"
huggingface-cli download meituan-longcat/LongCat-Video --local-dir ./weights/LongCat-Video
huggingface-cli download meituan-longcat/LongCat-Video-Avatar --local-dir ./weights/LongCat-Video-Avatar
huggingface-cli download meituan-longcat/LongCat-Video-Avatar-1.5 --local-dir ./weights/LongCat-Video-Avatar-1.5

Run Text-to-Video

# Single-GPU inference
torchrun run_demo_text_to_video.py --checkpoint_dir=./weights/LongCat-Video --enable_compile

# Multi-GPU inference
torchrun --nproc_per_node=2 run_demo_text_to_video.py --context_parallel_size=2 --checkpoint_dir=./weights/LongCat-Video --enable_compile

Run Image-to-Video

# Single-GPU inference
torchrun run_demo_image_to_video.py --checkpoint_dir=./weights/LongCat-Video --enable_compile

# Multi-GPU inference
torchrun --nproc_per_node=2 run_demo_image_to_video.py --context_parallel_size=2 --checkpoint_dir=./weights/LongCat-Video --enable_compile

Run Video-Continuation

# Single-GPU inference
torchrun run_demo_video_continuation.py --checkpoint_dir=./weights/LongCat-Video --enable_compile

# Multi-GPU inference
torchrun --nproc_per_node=2 run_demo_video_continuation.py --context_parallel_size=2 --checkpoint_dir=./weights/LongCat-Video --enable_compile

Run Long-Video Generation

# Single-GPU inference
torchrun run_demo_long_video.py --checkpoint_dir=./weights/LongCat-Video --enable_compile

# Multi-GPU inference
torchrun --nproc_per_node=2 run_demo_long_video.py --context_parallel_size=2 --checkpoint_dir=./weights/LongCat-Video --enable_compile

Run Interactive Video Generation

# Single-GPU inference
torchrun run_demo_interactive_video.py --checkpoint_dir=./weights/LongCat-Video --enable_compile

# Multi-GPU inference
torchrun --nproc_per_node=2 run_demo_interactive_video.py --context_parallel_size=2 --checkpoint_dir=./weights/LongCat-Video --enable_compile

Run LongCat-Video-Avatar

💡 User tips for 1.5
  • Lip synchronization accuracy: Audio CFG works optimally between 3–5. Increase the audio CFG value for better synchronization.
  • Prompt Enhancement: Longer, more descriptive prompts yield better consistency and naturalness than short ones. We recommend including rich details such as character appearance, actions, and scene context (e.g., "A young woman with long black hair is speaking and smiling, wearing a white blouse, sitting in a bright café") for best results.
  • Mitigate repeated actions: Setting the reference image index（--ref_img_index, default to 10） between 0 and 24 ensures better consistency; setting it to 30 helps reduce repeated actions. Additionally, increasing the mask frame range (--mask_frame_range, default to 3) can further help mitigate repeated actions, but excessively large values may introduce artifacts.
  • Super resolution: Our model is compatible with both 480P and 720P, which can be controlled via --resolution.
  • Dual-Audio Modes: Merge mode (set audio_type to para) requires two audio clips of equal length, and the resulting audio is obtained by summing the two clips; Concatenation mode (set audio_type to add) does not require equal-length inputs, and the resulting audio is formed by sequentially concatenating the two clips with silence padding for any gaps, where by default person1 speaks first and person2 speaks afterward.
  • Model versions: --model_type avatar-v1.0 uses wav2vec2 audio encoder (default); --model_type avatar-v1.5 uses Whisper-large-v3 audio encoder for better lip sync quality.
  • Distillation mode: Add --use_distill to enable distillation sampling (fewer steps, faster inference). This is required when using --model_type avatar-v1.5.
  • INT8 quantization: Add --use_int8 to load the INT8 quantized DiT model for reduced VRAM usage. Only supported with --model_type avatar-v1.5.
💡 User tips for 1.0
  • Lip synchronization accuracy:​​ Audio CFG works optimally between 3–5. Increase the audio CFG value for better synchronization.
  • Prompt Enhancement: Include clear verbal-action cues (e.g., talking, speaking) in the prompt to achieve more natural lip movements.
  • Mitigate repeated actions: Setting the reference image index（--ref_img_index, default to 10） between 0 and 24 ensures better consistency, while selecting other ranges (e.g., -10 or 30) helps reduce repeated actions. Additionally, increasing the mask frame range (--mask_frame_range, default to 3) can further help mitigate repeated actions, but excessively large values may introduce artifacts.
  • Super resolution: Our model is compatible with both 480P and 720P, which can be controlled via --resolution.
  • Dual-Audio Modes: Merge mode (set audio_type to para) requires two audio clips of equal length, and the resulting audio is obtained by summing the two clips; Concatenation mode (set audio_type to add) does not require equal-length inputs, and the resulting audio is formed by sequentially concatenating the two clips with silence padding for any gaps, where by default person1 speaks first and person2 speaks afterward.

LongCat-Video-Avatar-1.5

# Audio-Text-to-Video
torchrun --nproc_per_node=2 run_demo_avatar_single_audio_to_video.py --context_parallel_size=2 --checkpoint_dir=./weights/LongCat-Video-Avatar-1.5 --stage_1=at2v --input_json=assets/avatar/single_example_1.json --use_distill --model_type avatar-v1.5 --use_int8

# Audio-Image-to-Video
torchrun --nproc_per_node=2 run_demo_avatar_single_audio_to_video.py --context_parallel_size=2 --checkpoint_dir=./weights/LongCat-Video-Avatar-1.5 --stage_1=ai2v --input_json=assets/avatar/single_example_1.json --use_distill --model_type avatar-v1.5 --use_int8

# Audio-Text-to-Video and Video-Continuation
torchrun --nproc_per_node=2 run_demo_avatar_single_audio_to_video.py --context_parallel_size=2 --checkpoint_dir=./weights/LongCat-Video-Avatar-1.5 --stage_1=at2v --input_json=assets/avatar/single_example_1.json --num_segments=5 --ref_img_index=10 --mask_frame_range=3 --use_distill --model_type avatar-v1.5 --use_int8

# Audio-Image-to-Video and Video-Continuation
torchrun --nproc_per_node=2 run_demo_avatar_single_audio_to_video.py --context_parallel_size=2 --checkpoint_dir=./weights/LongCat-Video-Avatar-1.5 --stage_1=ai2v --input_json=assets/avatar/single_example_1.json --num_segments=5 --ref_img_index=10 --mask_frame_range=3 --use_distill --model_type avatar-v1.5 --use_int8

# Audio-Image-to-Video
torchrun --nproc_per_node=2 run_demo_avatar_multi_audio_to_video.py --context_parallel_size=2 --checkpoint_dir=./weights/LongCat-Video-Avatar-1.5 --input_json=assets/avatar/multi_example_1.json --use_distill --model_type avatar-v1.5 --use_int8

# Audio-Image-to-Video and Video-Continuation
torchrun --nproc_per_node=2 run_demo_avatar_multi_audio_to_video.py --context_parallel_size=2 --checkpoint_dir=./weights/LongCat-Video-Avatar-1.5 --input_json=assets/avatar/multi_example_1.json --num_segments=5 --ref_img_index=10 --mask_frame_range=3 --use_distill --model_type avatar-v1.5 --use_int8

Run Streamlit

# Single-GPU inference
streamlit run ./run_streamlit.py --server.fileWatcherType none --server.headless=false

Evaluation Results

Text-to-Video

The Text-to-Video MOS evaluation results on our internal benchmark.

MOS score Veo3 PixVerse-V5 Wan 2.2-T2V-A14B LongCat-Video
Accessibility Proprietary Proprietary Open Source Open Source
Architecture - - MoE Dense
# Total Params - - 28B 13.6B
# Activated Params - - 14B 13.6B
Text-Alignment↑ 3.99 3.81 3.70 3.76
Visual Quality↑ 3.23 3.13 3.26 3.25
Motion Quality↑ 3.86 3.81 3.78 3.74
Overall Quality↑ 3.48 3.36 3.35 3.38

Image-to-Video

The Image-to-Video MOS evaluation results on our internal benchmark.

MOS score Seedance 1.0 Hailuo-02 Wan 2.2-I2V-A14B LongCat-Video
Accessibility Proprietary Proprietary Open Source Open Source
Architecture - - MoE Dense
# Total Params - - 28B 13.6B
# Activated Params - - 14B 13.6B
Image-Alignment↑ 4.12 4.18 4.18 4.04
Text-Alignment↑ 3.70 3.85 3.33 3.49
Visual Quality↑ 3.22 3.18 3.23 3.27
Motion Quality↑ 3.77 3.80 3.79 3.59
Overall Quality↑ 3.35 3.27 3.26 3.17

Community Works

Community works are welcome! Please PR or inform us in Issue to add your work.

License Agreement

The model weights are released under the MIT License.

Any contributions to this repository are licensed under the MIT License, unless otherwise stated. This license does not grant any rights to use Meituan trademarks or patents.

See the LICENSE file for the full license text.

Usage Considerations

This model has not been specifically designed or comprehensively evaluated for every possible downstream application.

Developers should take into account the known limitations of large language models, including performance variations across different languages, and carefully assess accuracy, safety, and fairness before deploying the model in sensitive or high-risk scenarios. It is the responsibility of developers and downstream users to understand and comply with all applicable laws and regulations relevant to their use case, including but not limited to data protection, privacy, and content safety requirements.

Nothing in this Model Card should be interpreted as altering or restricting the terms of the MIT License under which the model is released.

Citation

We kindly encourage citation of our work if you find it useful.

@misc{meituanlongcatteam2025longcatvideotechnicalreport,
      title={LongCat-Video Technical Report}, 
      author={Meituan LongCat Team and Xunliang Cai and Qilong Huang and Zhuoliang Kang and Hongyu Li and Shijun Liang and Liya Ma and Siyu Ren and Xiaoming Wei and Rixu Xie and Tong Zhang},
      year={2025},
      eprint={2510.22200},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2510.22200}, 
}

@misc{meituanlongcatteam2026longcatvideoavatar15technicalreport,
      title={LongCat-Video-Avatar 1.5 Technical Report}, 
      author={Meituan LongCat Team and Xunliang Cai and Meng Cheng and Feng Gao and Zhe Kong and Jiamu Li and Le Li and Weiheng Li and Hongyu Liu and Shuai Tan and Xiaoming Wei and Tianyu Yang and Yong Zhang},
      year={2026},
      eprint={2605.26486},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2605.26486}, 
}

@misc{meituanlongcatteam2025longcatvideoavatartechnicalreport,
      title={LongCat-Video-Avatar Technical Report}, 
      author={Meituan LongCat Team},
      year={2025},
      eprint={},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={}, 
}

Acknowledgements

We would like to thank the contributors to the Wan, UMT5-XXL, Diffusers and HuggingFace repositories, for their open research.

Contact

Please contact us at [email protected] or scan the QR code to join our WeChat Group if you have any questions.

LongCat-Video (meituan-longcat/LongCat-Video)

Searching Database…

9.2K ·GitHub
LongCat-Video
这是什么？LongCat-Video 是一个拥有 13.6B 参数的基础视频生成模型，它在同一个架构中统一了文本生视频、图像生视频和视频续写生成，旨在实现高效的长视频生成，并被定位为迈向世界模型的第一步。
这是现成的应用，还是需要我自己运行的模型？这是一个需要你自己下载并运行的模型，而不是托管服务。你需要克隆仓库、安装依赖、从 Hugging Face 拉取权重，然后在自己的硬件上本地运行 Python 推理脚本或自带的 Streamlit 界面。
我该如何让它跑起来？先克隆仓库，安装依赖，然后用 huggingface-cli 从 Hugging Face 下载模型权重，之后再运行文生视频、图生视频、视频续写或 Avatar 数字人的推理脚本（同时也提供了一个 Streamlit 应用）。
运行它需要什么硬件？需要性能较强的 GPU。默认启用 FlashAttention-2（也可以换成 FlashAttention-3 或 xformers），并且专门为 Avatar 数字人模型提供了 INT8 量化，以降低显存（VRAM）占用。
在使用之前，有哪些局限性需要我知道？作者表示，该模型尚未针对所有下游应用场景进行全面评估，因此在敏感或高风险场景中部署之前，你需要自行评估其准确性、安全性和公平性。
这里的 MIT 许可证具体涵盖哪些内容？模型权重和仓库中的贡献内容均以 MIT 许可证发布，但该许可证并不授予使用美团商标或专利的任何权利。
LongCat-Video-Avatar 是什么？它是基础模型的扩展，用于音频驱动的角色动画，可将音频加上文本或图像输入转换为富有表现力的说话视频，并支持单音频流和多音频流。
Avatar-1.5 更新有哪些变化？它用 Whisper-Large 替换了 Wav2Vec2 音频编码器，以实现更准确的口型同步；新增了步数蒸馏，支持快速的 8 步推理；并且在动漫、动物等风格化领域上的泛化能力更好。
视频生成能达到什么分辨率和速度？LongCat-Video 利用 Block Sparse Attention，采用由粗到精的时空策略，可在几分钟内生成 720p、30fps 的视频，并通过分辨率参数支持 480P 和 720P 两种输出。
为什么项目把自己描述为迈向“世界模型”的一步？因为它原生基于视频续写任务进行预训练，因此能够生成长达数分钟的视频，且不出现色彩漂移或质量下降。作者将这一能力视为迈向世界模型的第一步。
What are similar projects?LTX-Video, LTX-2, Evoke, SolarWM, SparkDiffusion, CogVideo, DreamX-Creator, ComfyUI-MiniMaxH3-TimelineDirector
Which topics is it in?talking-head, digital-human, lipsync, ai-video-generation, grpo, streamlit, world-models, foundation-models, image-to-video, text-to-video, whisper, diffusion-models
Creator
meituan-longcat
Size
1.3 GB
Stars
9.2K
Updated
CommitMay 27
Age
11mo
Commits · 90 Days