image-to-3d-pipeline README

The project's README from github.com/dreamers-laboratory/image-to-3d-pipeline. As one markdown file: README.md.

image-to-3d-pipeline

Turn a handful of images into a 3D mesh, then find out which model did it best.

Single-image-to-3D reconstruction has a dozen open-source models and no obvious winner; the honest way to choose is to run several on the same input and look hard at what comes out. This pipeline does exactly that. It takes AI-generated renders of a fictional submersible, reconstructs candidate meshes with several models, scores them against a fixed Blender inspection protocol, and serves the winner in a browser-based WebGL explorer.

For the curious, a live photogrammetry case study from this repo: 3d.thedreamers.us/about, and the explorer itself at 3d.thedreamers.us for live interaction.

See it

Eight generated viewpoints of the submersible, from one concept image

Eight viewpoints from one concept. Multiview reasoning happens before any geometry: port profile, three-quarter, bow-on, stern-on. Every reconstruction model in this repo starts from this sheet, so their outputs are comparable.

The reconstructed mesh in the browser viewer, with the six-stage build strip

Six moves, one browser object. The winning mesh in the WebGL viewer, and the pipeline it came through: mask, infer, bake, render, reject, export. The note under the strip is the honest limit of the whole method: a depth map estimates distance for visible pixels and cannot reveal what the camera never saw.

The build-story page at 3d.thedreamers.us

The build story, live. The public page walks from source imagery to a browser-delivered 3D world, step by step, in the open.

Built by Dreamers Inc.

Source imagery

All source images in this repository are AI-generated synthetic renders of a fictional vessel. No real vehicle, vessel, person, or client is depicted.

Repository layout

Large artifacts are excluded: Python environments, model checkpoints, intermediate PLY/GLB outputs, and the explorer's full runtime asset set. 05-web-explorer/README.md describes how to supply runtime assets, and examples/ provides a mesh you can use for that.

Review order

  1. 06-evaluation/EVALUATION.md for the ranking and its limitations.
  2. 06-evaluation/explorer-variant-comparison.jpg for the same browser scene with three candidate meshes.
  3. 05-web-explorer/README.md for local run instructions.
  4. examples/ for sample inputs and a reconstructed output.

Third-party reconstruction tools

Run ./setup.sh to clone the four reconstruction models at the exact commits used in this evaluation. They are cloned rather than vendored because each carries its own license, which you accept when you fetch it:

Tool License
TRELLIS MIT
TripoSR MIT
stable-fast-3d Stability AI Community License
Hunyuan3D-2 Tencent Hunyuan community license, territorially restricted

Review each license before use, particularly the Stability AI and Tencent terms, which restrict commercial use and territory respectively.

Pipeline stages

  1. Preprocess: tools/preprocess_object_masks.py produces transparent object-only masks from the source renders.
  2. Reconstruct: tools/run_trellis_multiview.py runs multi-image TRELLIS; 07-experiments/ holds runners for the other candidates.
  3. Evaluate: tools/run_reconstruction_eval.sh renders each GLB through the same fixed Blender orbit and builds contact sheets for the 0-2 visual rubric in 07-experiments/HUNYUAN_2MV_EXPERIMENT_PROTOCOL.md.
  4. Explore: 05-web-explorer/ loads the chosen GLB in a navigable underwater scene with BVH collision.

License

The code in this repository is licensed under the Apache License 2.0 (see LICENSE). The third-party tools fetched by setup.sh are excluded and remain under their own licenses.

image-to-3d-pipeline (dreamers-laboratory/image-to-3d-pipeline)

Searching Database…

278 ·GitHub
Image-to-3d-pipeline
这是什么？一条流水线：用几个开源模型从少量 AI 生成图像重建 3D 网格，按固定的 Blender 检查协议给结果打分，并在基于浏览器的 WebGL 查看器里展示胜出的网格。
这是现成的应用，还是一套我自己运行的脚本？它是按阶段组织的脚本和工具集合（tools/、05-web-explorer/、06-evaluation/、07-experiments/），而不是单一的可安装应用；预处理、重建、评估和查看器要作为独立步骤分别运行。
怎么让它跑起来？运行 ./setup.sh 克隆固定版本的重建工具，然后按流水线阶段进行：预处理物体掩膜，重建网格（例如用 TRELLIS），通过固定的 Blender 环绕路径评估 GLB，最后在 05-web-explorer/ 里加载胜出者。
重建模型需要我自己安装吗？需要——setup.sh 会按固定 commit 克隆 TRELLIS、TripoSR、stable-fast-3d 和 Hunyuan3D-2；仓库不附带 Python 环境、模型检查点或中间网格输出。
这种重建方法的根本限制是什么？它基于深度图，所以只能估计相机真正拍到的像素的距离——无法恢复源图像中从未露出的几何结构。
这里用到的模型都可以免费商用吗？不是——仓库自身的代码是 Apache-2.0，但四个克隆的重建工具中有两个带限制性许可：stable-fast-3d 采用 Stability AI Community License，Hunyuan3D-2 采用有地域限制的腾讯许可证，两者都限制商业使用。
流水线实际用什么图像作为输入？所有源图像都是 AI 生成的虚构潜水器合成渲染图——不涉及任何真实载具、船只或人物。
它比较多少个模型？怎么打分？它运行多个开源重建模型（experiments 文件夹里的 TRELLIS、TripoSR、stable-fast-3d 和 Hunyuan3D-2），用固定的 Blender 环绕渲染，按 0-2 分的视觉评分标准给每个模型的输出排名。
不自己跑流水线也能看到结果吗？可以，examples/ 文件夹附带三张示例源渲染图和一个重建好的 GLB 网格（约 1.5 MB），可以直接查看。
在哪里能看到成果的在线演示？在线摄影测量案例研究和交互式 WebGL 查看器托管在 3d.thedreamers.us/about 和 3d.thedreamers.us。
What are similar projects?Meshroom, cesium, spirula-studio, img2threejs, CodeVideoModel, paper-aquarium, pointdit, clay-safari
Which topics is it in?photogrammetry, gltf, blender, 3d-reconstruction, computer-graphics, threejs, webgl, rendering, evaluation, 3d, image-processing, computer-vision
Creator
dreamers-laboratory
Size
9.5 MB
Stars
278
Updated
CommitSep 02
Age
1mo
Commits · 90 Days