342 ·GitHub
What is this?
WorldSculpt is the official implementation of a research method that turns grounded video into 3D scenes: from RGB images with instance masks and 3D boxes it builds a compositional mesh of very complex scenes with hundreds of individual objects.
WorldSculpt is the official implementation of a research method that turns grounded video into 3D scenes: from RGB images with instance masks and 3D boxes it builds a compositional mesh of very complex scenes with hundreds of individual objects.Is this an app I can just run, or research code?Research code, not an app: the official implementation of a paper. The Sep 2026 release includes Python inference scripts, a pretrained checkpoint on HuggingFace, the arXiv paper and a project page — everything runs from the command line.
Research code, not an app: the official implementation of a paper. The Sep 2026 release includes Python inference scripts, a pretrained checkpoint on HuggingFace, the arXiv paper and a project page — everything runs from the command line.How do I get it up and running?Five steps: install the TRELLIS.2 base environment, add the extra dependencies with NATTENCUDAARCH set to your GPU's compute capability, download the pretrained checkpoints and the data, then run the inference script.
Five steps: install the TRELLIS.2 base environment, add the extra dependencies with NATTENCUDAARCH set to your GPU's compute capability, download the pretrained checkpoints and the data, then run the inference script.What do I need before installing?An NVIDIA GPU with CUDA — you set NATTENCUDAARCH to your card's compute capability — plus the TRELLIS.2 environment installed first as the base, along with additional dependencies like NATTEN.
An NVIDIA GPU with CUDA — you set NATTENCUDAARCH to your card's compute capability — plus the TRELLIS.2 environment installed first as the base, along with additional dependencies like NATTEN.Anything tricky about running the inference?The inference script has no defaults: every option must be passed explicitly on the command line. Also, INPUTROOT must point at the extracted dataset subdirectory holding transforms.json, not at ./input.
The inference script has no defaults: every option must be passed explicitly on the command line. Also, INPUTROOT must point at the extracted dataset subdirectory holding transforms.json, not at ./input.How does it handle scenes with hundreds of objects?The core idea is reusing a generative prior trained for single objects on whole scenes: the authors finetuned Pixal3D to accept occluded multi-view input, then use it for compositional meshification of very complex scenes.
The core idea is reusing a generative prior trained for single objects on whole scenes: the authors finetuned Pixal3D to accept occluded multi-view input, then use it for compositional meshification of very complex scenes.Can I feed it a Marble world?Yes — one demonstrated application runs the same pipeline on a Marble 3DGS world and turns it into object-level compositional meshes.
Yes — one demonstrated application runs the same pipeline on a Marble 3DGS world and turns it into object-level compositional meshes.What is the UE-MeshyScene benchmark?A dataset released with the paper, providing per-frame, per-instance annotations for evaluating compositional mesh reconstruction on complex scenes.
A dataset released with the paper, providing per-frame, per-instance annotations for evaluating compositional mesh reconstruction on complex scenes.Can I train or finetune the model myself?The README only covers inference: the Sep 2026 release ships inference code and a pretrained checkpoint, with no training pipeline described.
The README only covers inference: the Sep 2026 release ships inference code and a pretrained checkpoint, with no training pipeline described.What are similar projects?pointdit, Mira-Scene, TRELLIS, ml-sharp, InsertAny3D, wuji-ego-mint, vggt, video-prediction-policy-2
pointdit, Mira-Scene, TRELLIS, ml-sharp, InsertAny3D, wuji-ego-mint, vggt, video-prediction-policy-2Which topics is it in?3d-gaussian-splatting, instance-segmentation, image-to-3d, 3d-generation, 3d-reconstruction, arxiv, pretrained-models, paper, dataset, 3d, video, inference
3d-gaussian-splatting, instance-segmentation, image-to-3d, 3d-generation, 3d-reconstruction, arxiv, pretrained-models, paper, dataset, 3d, video, inference