10.9K ·GitHub
What is this?
This is the official codebase for 'MagicAnimate' (CVPR 2024), a diffusion-model method that animates a reference human image according to a motion sequence to produce temporally consistent video.
This is the official codebase for 'MagicAnimate' (CVPR 2024), a diffusion-model method that animates a reference human image according to a motion sequence to produce temporally consistent video.Is this a ready-made app or just research code?It's the official research release for the paper - inference scripts and a Gradio demo built on top of Stable Diffusion V1.5, not a packaged library or installer.
It's the official research release for the paper - inference scripts and a Gradio demo built on top of Stable Diffusion V1.5, not a packaged library or installer.How do I get it running?Download the Stable Diffusion V1.5 base model, the MSE-finetuned VAE, and the MagicAnimate checkpoints from Hugging Face, place them in the expected folder structure, then run the provided inference script.
Download the Stable Diffusion V1.5 base model, the MSE-finetuned VAE, and the MagicAnimate checkpoints from Hugging Face, place them in the expected folder structure, then run the provided inference script.What do I need installed before trying this?Python 3.8+, CUDA 11.3+, and ffmpeg; git-lfs is recommended for pulling the large pretrained checkpoints.
Python 3.8+, CUDA 11.3+, and ffmpeg; git-lfs is recommended for pulling the large pretrained checkpoints.Any install pitfalls I should know about?You have to manually download several large pretrained checkpoints (SD base model, VAE, MagicAnimate weights) and place them in an exact folder layout before anything runs.
You have to manually download several large pretrained checkpoints (SD base model, VAE, MagicAnimate weights) and place them in an exact folder layout before anything runs.What makes MagicAnimate different from simple pose-transfer methods?It specifically targets temporal consistency across frames using a diffusion model conditioned on pose, reducing the flicker and appearance drift common in frame-by-frame animation approaches.
It specifically targets temporal consistency across frames using a diffusion model conditioned on pose, reducing the flicker and appearance drift common in frame-by-frame animation approaches.Can I try it without installing anything?Yes, there's an online Gradio demo (set up with help from the Hugging Face team) you can use directly in your browser without local setup.
Yes, there's an online Gradio demo (set up with help from the Hugging Face team) you can use directly in your browser without local setup.Is it free to use, including commercially?Yes, it's released under the BSD-3-Clause license, which is permissive and allows commercial use.
Yes, it's released under the BSD-3-Clause license, which is permissive and allows commercial use.What are similar projects?FramePack, ControlNet, generative-models, wuji-ego-mint, Open-Sora, HunyuanVideo-1.5, SolarWM, DreamX-Creator
FramePack, ControlNet, generative-models, wuji-ego-mint, Open-Sora, HunyuanVideo-1.5, SolarWM, DreamX-CreatorWhich topics is it in?pose-estimation, diffusion, image-to-video, stable-diffusion, gradio, paper, academic, video-generation, video, bsd, animation, huggingface
pose-estimation, diffusion, image-to-video, stable-diffusion, gradio, paper, academic, video-generation, video, bsd, animation, huggingface