ViMax README

The project's README from github.com/HKUDS/ViMax. As one markdown file: README.md.

ViMax Logo

ViMax: Agentic Video Generation

HKUDS%2FViMax | Trendshift HKUDS%2FViMax | Trendshift

MIT License ViMax v1.2.0

🚨 Current Video Generation Limitations:

💡 ViMax Solution:

🎬 Director, Screenwriter, Producer, and Video Generator All-in-One! We're exploring a future where AI becomes a complete creative powerhouse. 💡 Simply input your concept. ViMax autonomously handles the rest. It orchestrates scriptwriting, storyboarding, character creation, and final video generation—all end-to-end. 🚀

https://github.com/user-attachments/assets/5bad46b2-8276-4e1d-9480-3522640744b2

📰 News

📑 Table of Contents

✨ Key Features

ViMax is an agentic video creation framework that connects narrative planning, visual consistency, image generation, video generation, and final assembly in one extensible workflow.

🔮Video Demos Generated from Scratch

🖥️ ViMax Web UI

ViMax Web UI workspace

The Web UI brings agent conversation, project artifacts, storyboard previews, and render progress into one visual workspace.

🎯 End-to-End Video Creation Engine

The Challenges:

ViMAX: eliminates these production bottlenecks by automating the entire video creation pipeline from narrative input to final video output.

🔥 Why ViMax?

🧠 Effortless Production 🚀 Complete Creative Freedom 🔊 Audio and Video Binding 🎨 Professional Quality 🤩 Interactive Video
One-Prompt to Finished Video From Any Narrative to Reality Synchronized Storytelling Movie-Grade Output Make Your Own Cameo Video
Skip the technical complexity—just describe your vision and let ViMax handle script generation, storyboarding, shot design, reference management, and consistency validation No creative limits—whether it's a trailer, short story, novel chapter, or original concept, ViMax intelligently structures narratives and designs cinematography to bring any idea to life Seamlessly integrate character voice, and sound effects with visual content to create immersive experiences where audio and video work in perfect harmony Automated quality control ensures character consistency, proper scene composition, and professional visual standards across every frame of your video Interact in your own short stories by uploading your photo—ViMax intelligently integrates you as a character with consistent appearance and natural interactions throughout the entire video

ViMax now also includes an Agents Loop + TUI workflow for interactive planning, revision, rendering control, session reuse, and context compaction while preserving the original direct pipeline entrypoints.

☄️ Roadmap

🚀Quick Start

🖥️ Environment

OS: Linux, Windows

📥 Clone and Install

We use uv to manage the environment. For uv installation, please refer to the https://docs.astral.sh/uv/getting-started/installation/.

git clone https://github.com/HKUDS/ViMax.git
cd ViMax
uv sync
Agent TUI

ViMax also provides a minimal TUI for interactive agent-based video creation. Create a private local configuration from the tracked example first:

cp configs/agent.example.yaml configs/agent.local.yaml

Then configure the model and API key information in configs/agent.local.yaml, including the LLM, image generator, and video generator, as shown below.

llm:
  model_provider: openai
  model: <YOUR_LLM_MODEL>
  base_url: <YOUR_LLM_BASE_URL>
  api_key: <YOUR_API_KEY>

image:
  model: <YOUR_IMAGE_MODEL>
  base_url: <YOUR_IMAGE_BASE_URL>
  api_key: <YOUR_API_KEY>

video:
  model: <YOUR_VIDEO_MODEL>
  base_url: <YOUR_VIDEO_BASE_URL>
  api_key: <YOUR_API_KEY>

Then, start the TUI from the ViMax root directory:

vimax tui

Start a new session or resume an existing one:

vimax tui new
vimax tui resume
vimax tui resume <session_id>

You can also keep configs/agent.local.yaml empty and provide the same values through environment variables, such as VIMAX_LLM_API_KEY, VIMAX_IMAGE_API_KEY, and VIMAX_VIDEO_API_KEY.

Web UI

The Web UI uses the same ViMax agent runtime, sessions, tools, and private configs/agent.local.yaml configuration as the TUI. Node.js 18 or newer is required.

From the ViMax repository root, install the frontend dependencies once and start the local server:

cd web
npm install
npm run dev

Open http://127.0.0.1:4173 in your browser. The Web UI supports named projects, agent conversations, slash commands, artifact and render inspection, storyboard previews, file uploads, and provider settings.

The server binds to 127.0.0.1 by default. When ViMax runs on a remote machine, forward the port from your local computer:

ssh -N -L 4173:127.0.0.1:4173 <user>@<server>

Use VIMAX_WEB_PORT when another local port is required:

VIMAX_WEB_PORT=4174 npm run dev
Usage

main_idea2video.py is used to convert your ideas into videos. You need to configure the model and API key information in the configs/idea2video.yaml file, including three parts—the chat model, the image generator, and the video generator, as shown below

chat_model:
  init_args:
    model: google/gemini-2.5-flash-lite-preview-09-2025
    model_provider: openai
    api_key: <YOUR_API_KEY>
    base_url: https://openrouter.ai/api/v1

image_generator:
  class_path: tools.ImageGeneratorNanobananaGoogleAPI
  init_args:
    api_key: <YOUR_API_KEY>

video_generator:
  class_path: tools.VideoGeneratorVeoGoogleAPI
  init_args:
    api_key: <YOUR_API_KEY>

working_dir: .working_dir/idea2video

Then, provide a simple yet thoughtful idea and the corresponding creative requirements in main_idea2video.py.

idea = \
"""
If a cat and a dog are best friends, what would happen when they meet a new cat?
"""
user_requirement = \
"""
For children, do not exceed 3 scenes.
"""
style = "Cartoon"

main_script2video.py generates a video based on a specific script. You similarly need to set up the API configuration in configs/script2video.yaml file. Then, provide a scene script and the corresponding creative requirements in main_script2video.py, as shown below.

script = \
"""
EXT. SCHOOL GYM - DAY
A group of students are practicing basketball in the gym. The gym is large and open, with a basketball hoop at one end and a large crowd of spectators at the other end. John (18, male, tall, athletic) is the star player, and he is practicing his dribble and shot. Jane (17, female, short, athletic) is the assistant coach, and she is helping John with his practice. The other students are watching the practice and cheering for John.
John: (dribbling the ball) I'm going to score a basket!
Jane: (smiling) Good job, John!
John: (shooting the ball) Yes!
...
"""
user_requirement = \
"""
Fast-paced with no more than 20 shots.
"""
style = "Animate Style"

ViMax (HKUDS/ViMax)

Searching Database…

12.7K ·GitHub
ViMax
What is this?ViMax is an agentic AI framework that acts as an all-in-one director, screenwriter, producer, and video generator: give it a concept, script, or novel, and it autonomously handles scriptwriting, storyboarding, character creation, and final video assembly.
Is this a Python library I import, or something I run as an app?It's a self-hosted agentic pipeline you run from the cloned repo: either through main scripts like idea2video.py and script2video.py, an interactive terminal TUI, or a browser-based Web UI, not something you pip-install into another project.
How do I get it running?Install dependencies with uv, copy configs/agent.local.yaml from the tracked example and fill in your LLM, image generator, and video generator settings, then launch the TUI from the repo root; for the Web UI, install Node.js 18+ frontend dependencies and start the local server, which binds to 127.0.0.1 by default.
What API keys or tools do I need before I can generate a video?You need credentials for a chat/LLM provider, an image generator, and a video generator, which can be set via environment variables like VIMAX_LLM_API_KEY, VIMAX_IMAGE_API_KEY, and VIMAX_VIDEO_API_KEY, plus uv for the Python environment and Node.js 18+ if you want the Web UI.
Does it actually solve the usual AI video problems like short clips and inconsistent characters?ViMax targets those exact pain points with reference management and consistency checks, but the README itself admits the image generator can still produce unusable results even when given correct character, position, and environment references.
What's the difference between Idea2Video, Script2Video, and Novel2Video?Idea2Video expands a short concept into full stories, characters, and shots; Script2Video renders an explicit screenplay while preserving its intent; Novel2Video adapts long-form fiction into episodic videos with narrative compression and character tracking.
What is AutoCameo?AutoCameo lets you upload a photo of a person or pet so ViMax inserts them as a consistent character inside the generated story.
Which image and video generation models does it support?ViMax integrates providers such as OpenRouter's GPT Image 2 for images and Seedance 2.0 Fast or Google Omni for video generation, alongside MiniMax as a chat model provider.
Does ViMax handle audio too?Yes, its Audio and Video Binding feature syncs character voice and sound effects with the visuals for immersive, synchronized storytelling.
What are similar projects?AdCraft, shuohao-skills, reelmimic, storycast, artcraft, dramaclaw, Pixelle-Video, OpenMontage
Which topics is it in?gpt-image, ai-video, openrouter, text-to-video, content-creation, workflow, video-generation, tui, orchestration, image-generation, agentic-ai, generative-ai
Creator
HKUDS
Size
182.3 MB
Stars
12.7K
Updated
CommitSep 30Releasev1.2.0Jul 20
Age
1y
Commits · 90 Days