SolarRenderer README

项目的 README,来自 github.com/Enderfga/SolarRenderer。合并为一个 markdown 文件:README.md。

SolarRenderer: Few-Step Generative Reference Rendering

🌐 Project Page • 🎮 Online Demo • 📄 Paper • 🤗 Weights • 🤗 Data • 💻 Data code

SolarRenderer turns a structure video from a game engine or an agent (grey box, semantic map, skeletons, depth, body mesh, or a sketch it never saw in training) into a finished video, with an optional first frame and a prompt deciding the look. It runs on MiniMax-H3 with two LoRAs and renders a clip in four steps. A block-causal pair of the same LoRAs streams the clip block by block.

News

Highlights

Structure video (left) and SolarRenderer (right).

Scene grey box Semantic segmentation
Object masks + skeletons SMPL body mesh
Near-field shaded geometry Pencil storyboard (unseen)
Point cloud (unseen) Child's crayon drawing (unseen)
Voxels (unseen) Wireframe (unseen)

More structure types, styles and comparisons are on the project page.

Quick Start

git clone https://github.com/Enderfga/SolarRenderer && cd SolarRenderer
pip install -e . && pip install flash-attn==2.7.4.post1 --no-build-isolation
python scripts/prepare_h3.py --dir ./MiniMax-H3
hf download Enderfga/SolarRenderer config.json solarrenderer_control_lora.safetensors solarrenderer_4step_lora.safetensors \
    --local-dir ./weights
python scripts/render.py --h3 ./MiniMax-H3 \
    --control weights/solarrenderer_control_lora.safetensors \
    --fewstep weights/solarrenderer_4step_lora.safetensors \
    --structure examples/sem-highway_synthwave/structure.mp4 \
    --first examples/sem-highway_synthwave/first.png \
    --prompt examples/sem-highway_synthwave/prompt.txt --out render.mp4

The example is a synthwave highway run (semantic map in, neon render out). Add --causal with the two causal LoRAs to stream the clip block by block (details).

Ways to run

For Start here
Online demo No install: upload a structure video, add or generate a first frame, render in the browser Hugging Face Space
Reference script The reference implementation: one GPU, a clip at a time scripts/render.py above
ComfyUI A browser page and node workflows with no code; the low-memory workflow fits a 16 GB GPU (e.g. RTX 4090) comfyui/
Streaming on Reactor The fastest path: a four-GPU streaming service with a browser demo, validated by the Reactor team on four B200 GPUs reactor/

The Reactor integration is a community contribution; thanks to Rising0321 and the Reactor team.

Citation

If you use SolarRenderer in your research, please cite our paper.

@misc{fang2026solarrenderer,
  title  = {SolarRenderer: Few-Step Generative Reference Rendering},
  author = {Fang, Guian and Qian, Shengju and Lu, Dongyue and
            Huang, Junchao and Jiang, Lutao and Wang, Xin and
            Song, Yiren and Jiang, Li and Shou, Mike Zheng},
  year   = {2026},
  eprint = {2610.28630},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2610.28630}
}
SolarWM citation

If you also use the SolarWM framework, data, or models, please cite SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models:

@misc{huang2026solarwmopendatascalable,
      title={SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models},
      author={Junchao Huang and Guian Fang and Shengju Qian and Xianghao Kong and Zhuoran Zhao and Wei Huang and Yihua Du and Zixin Zhang and Justin Cui and Yuchao Gu and Yukang Chen and Xinting Hu and Tianyu He and Shaoshuai Shi and Zhuotao Tian and Xin Wang and Mike Zheng Shou and Li Jiang},
      year={2026},
      eprint={2609.02886},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.02886},
}

Acknowledgement

SolarRenderer builds on MiniMax-H3, starts its four-step LoRA from DMAD, and shares its training framework with SolarWM.

License

Code: Apache 2.0. Weights: MiniMax H3 Community License, including its territorial restrictions (EU, UK, Republic of Korea and US excluded) and Acceptable Use Policy; the four-step LoRA is fine-tuned from DMAD under the same license.

SolarRenderer (Enderfga/SolarRenderer)

正在搜索数据库…

这是什么?SolarRenderer 接收来自游戏引擎或智能体的结构视频(灰盒、语义图、骨架、深度、人体网格或草图),并将其转换为成品渲染视频。可选地提供首帧,并由文本提示词决定最终效果。
这是完整的渲染引擎,还是我可以在现有视频上运行的模型?它是一个生成式模型,而不是游戏引擎。它基于 MiniMax-H3 视频模型微调的两个 LoRA,能在四个采样步骤内,将现有结构视频重新渲染为风格化的输出。
我该怎么试着渲染一个片段?你可以直接使用 Hugging Face Spaces 上的在线演示,无需安装;也可以通过 scripts/render.py 在本地运行参考实现;或者使用提供的 ComfyUI 节点工作流。
运行它需要什么硬件?参考脚本每个片段需要一块 GPU;低显存的 ComfyUI 工作流至少需要 16 GB 显存的 GPU,如 RTX 4090;训练需要一台 8 卡 GPU 节点;最快的流式方案则需要四块 B200 GPU。
发布的模型权重有使用限制吗?模型权重采用 MiniMax H3 社区许可证发布,其中包含地域限制条款和《可接受使用政策》;而代码采用的是 Apache 2.0 许可证。
它可以从哪些类型的结构输入进行渲染?它支持灰盒场景、语义分割、带骨架的物体掩码、SMPL 人体网格、点云、体素、线框,甚至是训练期间从未见过的手绘草图。
流式模式有什么作用?同一组 LoRA 构成的块因果(block-causal)配对会逐块渲染视频,每个片段一完成就立即显示。这正是低延迟 Reactor 流式服务的技术核心。
为什么只需要四个采样步骤?结构引导的后训练使 SolarRenderer 只需四步采样,而非通常的五十步,速度约快 12.3 倍,同时保持相同的结构保真度。
有没有用于评估渲染结果的基准?有。我们发布了一个开放基准,包含 208 个困难案例和 53 个留出片段,并附带评分代码、作者自己的渲染结果以及每一条评审结论。
作者
Enderfga
大小
52.2 MB
星标
396
更新
提交Oct 30
时长
1个月
提交 · 90 天