177 ·GitHub
What is this?
Storyboard AI is an agentic pipeline that turns a single text prompt into a fully narrated whiteboard animation video: it researches the topic, writes a script, plans scenes, generates whiteboard-style artwork, animates the drawing, synthesizes voiceover, and burns subtitles.
Storyboard AI is an agentic pipeline that turns a single text prompt into a fully narrated whiteboard animation video: it researches the topic, writes a script, plans scenes, generates whiteboard-style artwork, animates the drawing, synthesizes voiceover, and burns subtitles.Is this a library I import or a standalone pipeline I run?It is an end-to-end Python pipeline run from a CLI, driven by a Director Agent that breaks the request into scenes and delegates to specialized sub-agents/tools, not a library you import into another app.
It is an end-to-end Python pipeline run from a CLI, driven by a Director Agent that breaks the request into scenes and delegates to specialized sub-agents/tools, not a library you import into another app.How do I get the pipeline running?Create a .env file in the genai-pipeline folder (copy .env.example) with your Gemini API key, install the root requirements.txt, then run the pipeline CLI which interactively asks for the topic, research mode, reference images, language and whether to use Veo video.
Create a .env file in the genai-pipeline folder (copy .env.example) with your Gemini API key, install the root requirements.txt, then run the pipeline CLI which interactively asks for the topic, research mode, reference images, language and whether to use Veo video.Do I need a Gemini API key before trying this?Yes, a Gemini API key is the only hard requirement; the Director Agent, image generation, and TTS all run on the Gemini stack, and the pipeline can run entirely in single-pass mode with just that key.
Yes, a Gemini API key is the only hard requirement; the Director Agent, image generation, and TTS all run on the Gemini stack, and the pipeline can run entirely in single-pass mode with just that key.What happens if I skip the SAM3 segmentation server?Without a SAMAPIURL pointing to a hosted SAM3 server, the pipeline simply skips object segmentation and draws the whiteboard animation in a simpler single-pass mode instead of the advanced multi-pass drawing.
Without a SAMAPIURL pointing to a hosted SAM3 server, the pipeline simply skips object segmentation and draws the whiteboard animation in a simpler single-pass mode instead of the advanced multi-pass drawing.How does it keep video generation costs low?Instead of generating long AI video clips, it stretches a small number of static line-art sketch sequences to match the narration length, e.g. only 32 seconds of raw visuals are stretched into a 2 minute 30 second final video.
Instead of generating long AI video clips, it stretches a small number of static line-art sketch sequences to match the narration length, e.g. only 32 seconds of raw visuals are stretched into a 2 minute 30 second final video.What languages can the narration be in?The Director, script narration, and subtitles can be generated dynamically in multiple languages such as Hindi, English, and Spanish, selected interactively when running the CLI.
The Director, script narration, and subtitles can be generated dynamically in multiple languages such as Hindi, English, and Spanish, selected interactively when running the CLI.What does the optional Veo integration add?Veo video generation can be enabled to insert rich AI-generated video segments into specific planned scenes instead of relying only on the hand-drawn whiteboard animation.
Veo video generation can be enabled to insert rich AI-generated video segments into specific planned scenes instead of relying only on the hand-drawn whiteboard animation.What are similar projects?CodeVideoModel, storycast, dramaclaw, Pixelle-Video, huobao-drama, Omni-Interaction-Agent, WanGP, generative-models
CodeVideoModel, storycast, dramaclaw, Pixelle-Video, huobao-drama, Omni-Interaction-Agent, WanGP, generative-modelsWhich topics is it in?gcp, speech-synthesis, subtitles, text-to-video, object-detection, agentic, fastapi, tts, video-processing, multimodal, video-generation, agent-orchestration
gcp, speech-synthesis, subtitles, text-to-video, object-detection, agentic, fastapi, tts, video-processing, multimodal, video-generation, agent-orchestration