211 ·GitHub
What is this?
Mira-Scene is a pipeline that reconstructs an editable 3D scene from a single image, combining interactive instance segmentation, depth estimation, canonical coordinate and voxel prediction, object mesh generation, scene assembly, and environment-map generation.
Mira-Scene is a pipeline that reconstructs an editable 3D scene from a single image, combining interactive instance segmentation, depth estimation, canonical coordinate and voxel prediction, object mesh generation, scene assembly, and environment-map generation.Is this a ready-to-use app or a research pipeline I run myself?It is an open-source research pipeline, not a packaged app: you get the inference code plus the Mira-CCM Stage 1 pretrained checkpoint and Stage 2 final checkpoint, which you run stage-by-stage yourself.
It is an open-source research pipeline, not a packaged app: you get the inference code plus the Mira-CCM Stage 1 pretrained checkpoint and Stage 2 final checkpoint, which you run stage-by-stage yourself.How do I run the inference pipeline?You follow the inference guide and run the stage-wise pipeline, which supports resumable execution, multiple depth and mesh backends, and case-level multi-GPU sharding.
You follow the inference guide and run the stage-wise pipeline, which supports resumable execution, multiple depth and mesh backends, and case-level multi-GPU sharding.Do I need pretrained checkpoints and a GPU to run this?Yes — you need the Stage 1 pretrained checkpoint and Stage 2 final checkpoint, downloadable from Hugging Face, and GPU resources since the pipeline is sharded across GPUs per case.
Yes — you need the Stage 1 pretrained checkpoint and Stage 2 final checkpoint, downloadable from Hugging Face, and GPU resources since the pipeline is sharded across GPUs per case.Is all the training data included in the release?The released Scene data and outpainted Objaverse data are provided for finetuning, but the large Objaverse rendering data used for pretraining is not included; it has to be regenerated from the source assets.
The released Scene data and outpainted Objaverse data are provided for finetuning, but the large Objaverse rendering data used for pretraining is not included; it has to be regenerated from the source assets.Is this meant for researchers rather than casual end users?Yes, it targets computer vision and generative-3D researchers who want to benchmark against or build on top of scene reconstruction methods, given the paper, benchmark tables, and training configs it ships with.
Yes, it targets computer vision and generative-3D researchers who want to benchmark against or build on top of scene reconstruction methods, given the paper, benchmark tables, and training configs it ships with.Does the MIT license cover everything in the repo?Only the Mira-Scene code itself is MIT-licensed; third-party components, pretrained models, and datasets keep their own separate licenses and terms.
Only the Mira-Scene code itself is MIT-licensed; third-party components, pretrained models, and datasets keep their own separate licenses and terms.What is Mira-CCM?Mira-CCM is the canonical-coordinate model checkpoint pretrained in Stage 1 of the pipeline; it's released on Hugging Face and can be used directly to launch Stage 2 finetuning.
Mira-CCM is the canonical-coordinate model checkpoint pretrained in Stage 1 of the pipeline; it's released on Hugging Face and can be used directly to launch Stage 2 finetuning.How does Mira-Scene compare to SAM3D?On the benchmark, Mira-Scene scores higher on 3D-IoU (0.727 vs 0.520), 2D-IoU, and ADD-S while using only 80k open-source 3D samples with a fully automatic data pipeline, versus SAM3D's closed-source million-plus samples requiring extensive manual layout annotation.
On the benchmark, Mira-Scene scores higher on 3D-IoU (0.727 vs 0.520), 2D-IoU, and ADD-S while using only 80k open-source 3D samples with a fully automatic data pipeline, versus SAM3D's closed-source million-plus samples requiring extensive manual layout annotation.How is training organized?Training has two stages: pretraining with exampletrain/configs/pretrainl1.yaml and finetuning with exampletrain/configs/finetune.yaml; a pretrained Mira-CCM checkpoint from Hugging Face lets you skip straight to the finetuning stage.
Training has two stages: pretraining with exampletrain/configs/pretrainl1.yaml and finetuning with exampletrain/configs/finetune.yaml; a pretrained Mira-CCM checkpoint from Hugging Face lets you skip straight to the finetuning stage.Where do the datasets come from?The Scene data and outpainted Objaverse data are published separately in the Mira-Scene Dataset repository; the large-scale raw Objaverse renders used for pretraining are not redistributed and must be regenerated from source assets.
The Scene data and outpainted Objaverse data are published separately in the Mira-Scene Dataset repository; the large-scale raw Objaverse renders used for pretraining are not redistributed and must be regenerated from source assets.What are similar projects?WorldSculpt, pointdit, tapnet, CodeVideoModel, InsertAny3D, UniMate, vggt, sam-3d-objects
WorldSculpt, pointdit, tapnet, CodeVideoModel, InsertAny3D, UniMate, vggt, sam-3d-objectsWhich topics is it in?3d-generation, 3d-reconstruction, segmentation, paper, training, dataset, evaluation, 3d, academic-research, artificial-intelligence, inference, generative-ai
3d-generation, 3d-reconstruction, segmentation, paper, training, dataset, evaluation, 3d, academic-research, artificial-intelligence, inference, generative-ai