image-to-3d-pipeline README

The project's README from github.com/dreamers-laboratory/image-to-3d-pipeline. As one markdown file: README.md.

image-to-3d-pipeline

Turn a handful of images into a 3D mesh, then find out which model did it best.

Single-image-to-3D reconstruction has a dozen open-source models and no obvious winner; the honest way to choose is to run several on the same input and look hard at what comes out. This pipeline does exactly that. It takes AI-generated renders of a fictional submersible, reconstructs candidate meshes with several models, scores them against a fixed Blender inspection protocol, and serves the winner in a browser-based WebGL explorer.

For the curious, a live photogrammetry case study from this repo: 3d.thedreamers.us/about, and the explorer itself at 3d.thedreamers.us for live interaction.

See it

Eight generated viewpoints of the submersible, from one concept image

Eight viewpoints from one concept. Multiview reasoning happens before any geometry: port profile, three-quarter, bow-on, stern-on. Every reconstruction model in this repo starts from this sheet, so their outputs are comparable.

The reconstructed mesh in the browser viewer, with the six-stage build strip

Six moves, one browser object. The winning mesh in the WebGL viewer, and the pipeline it came through: mask, infer, bake, render, reject, export. The note under the strip is the honest limit of the whole method: a depth map estimates distance for visible pixels and cannot reveal what the camera never saw.

The build-story page at 3d.thedreamers.us

The build story, live. The public page walks from source imagery to a browser-delivered 3D world, step by step, in the open.

Built by Dreamers Inc.

Source imagery

All source images in this repository are AI-generated synthetic renders of a fictional vessel. No real vehicle, vessel, person, or client is depicted.

Repository layout

Large artifacts are excluded: Python environments, model checkpoints, intermediate PLY/GLB outputs, and the explorer's full runtime asset set. 05-web-explorer/README.md describes how to supply runtime assets, and examples/ provides a mesh you can use for that.

Review order

  1. 06-evaluation/EVALUATION.md for the ranking and its limitations.
  2. 06-evaluation/explorer-variant-comparison.jpg for the same browser scene with three candidate meshes.
  3. 05-web-explorer/README.md for local run instructions.
  4. examples/ for sample inputs and a reconstructed output.

Third-party reconstruction tools

Run ./setup.sh to clone the four reconstruction models at the exact commits used in this evaluation. They are cloned rather than vendored because each carries its own license, which you accept when you fetch it:

Tool License
TRELLIS MIT
TripoSR MIT
stable-fast-3d Stability AI Community License
Hunyuan3D-2 Tencent Hunyuan community license, territorially restricted

Review each license before use, particularly the Stability AI and Tencent terms, which restrict commercial use and territory respectively.

Pipeline stages

  1. Preprocess: tools/preprocess_object_masks.py produces transparent object-only masks from the source renders.
  2. Reconstruct: tools/run_trellis_multiview.py runs multi-image TRELLIS; 07-experiments/ holds runners for the other candidates.
  3. Evaluate: tools/run_reconstruction_eval.sh renders each GLB through the same fixed Blender orbit and builds contact sheets for the 0-2 visual rubric in 07-experiments/HUNYUAN_2MV_EXPERIMENT_PROTOCOL.md.
  4. Explore: 05-web-explorer/ loads the chosen GLB in a navigable underwater scene with BVH collision.

License

The code in this repository is licensed under the Apache License 2.0 (see LICENSE). The third-party tools fetched by setup.sh are excluded and remain under their own licenses.

image-to-3d-pipeline (dreamers-laboratory/image-to-3d-pipeline)

blender
Searching Database…

278 ·GitHub
Image-to-3d-pipeline
What is this?A pipeline that reconstructs 3D meshes from a handful of AI-generated images using several open-source models, scores the results against a fixed Blender inspection protocol, and serves the winning mesh in a browser-based WebGL explorer.
Is this a ready-made app or a set of scripts I run myself?It's a collection of scripts and tools organized by stage (tools/, 05-web-explorer/, 06-evaluation/, 07-experiments/) rather than a single installable app; you run preprocessing, reconstruction, evaluation, and the viewer as separate steps.
How do I get it running?Run ./setup.sh to clone the pinned reconstruction tools, then follow the pipeline stages: preprocess object masks, reconstruct a mesh (e.g. with TRELLIS), evaluate the GLBs through a fixed Blender orbit, and load the winner in 05-web-explorer/.
Do I need to install the reconstruction models myself?Yes — setup.sh clones TRELLIS, TripoSR, stable-fast-3d, and Hunyuan3D-2 at pinned commits; the repo doesn't ship Python environments, model checkpoints, or intermediate mesh outputs.
What's the fundamental limit of this reconstruction method?It's depth-map based, so it can only estimate distance for pixels the camera actually saw — it cannot recover geometry that was never visible in the source images.
Are all the models used here free to use commercially?No — the repo's own code is Apache-2.0, but two of the four cloned reconstruction tools carry restrictive licenses: stable-fast-3d uses the Stability AI Community License and Hunyuan3D-2 uses a Tencent license that is territorially restricted, both limiting commercial use.
What images does the pipeline actually use as input?All source images are AI-generated synthetic renders of a fictional submersible — no real vehicle, vessel, or person is depicted.
How many models does it compare and how are they scored?It runs several open-source reconstruction models (TRELLIS, TripoSR, stable-fast-3d, and Hunyuan3D-2 in the experiments folder) and ranks each one's output using a fixed Blender orbit render against a 0-2 visual rubric.
Can I see a result without running the pipeline myself?Yes, the examples/ folder ships three sample source renders plus a reconstructed GLB mesh (about 1.5 MB) you can inspect directly.
Where can I see a live demo of the output?A live photogrammetry case study and the interactive WebGL explorer are hosted at 3d.thedreamers.us/about and 3d.thedreamers.us.
What are similar projects?Meshroom, cesium, spirula-studio, img2threejs, CodeVideoModel, paper-aquarium, pointdit, clay-safari
Which topics is it in?photogrammetry, gltf, blender, 3d-reconstruction, computer-graphics, threejs, webgl, rendering, evaluation, 3d, image-processing, computer-vision
Creator
dreamers-laboratory
Size
9.5 MB
Stars
278
Updated
CommitSep 02
Age
1mo
Commits · 90 Days