sam-3d-objects README

The project's README from github.com/facebookresearch/sam-3d-objects. As one markdown file: README.md.

SAM 3D

SAM 3D Objects is one part of SAM 3D, a pair of models for object and human mesh reconstruction. If you’re looking for SAM 3D Body, click here.

SAM 3D Objects

SAM 3D Team, Xingyu Chen*, Fu-Jen Chu*, Pierre Gleize*, Kevin J Liang*, Alexander Sax*, Hao Tang*, Weiyao Wang*, Michelle Guo, Thibaut Hardin, Xiang Li⚬, Aohan Lin, Jia-Wei Liu, Ziqi Ma⚬, Anushka Sagar, Bowen Song⚬, Xiaodong Wang, Jianing Yang⚬, Bowen Zhang⚬, Piotr Dollár†, Georgia Gkioxari†, Matt Feiszli†§, Jitendra Malik†§

Meta Superintelligence Labs

*Core contributor (Alphabetical, Equal Contribution), ⚬Intern, †Project leads, §Equal Contribution

[Paper] [Code] [Website] [Demo] [Blog] [BibTeX] [Roboflow]

SAM 3D Objects is a foundation model that reconstructs full 3D shape geometry, texture, and layout from a single image, excelling in real-world scenarios with occlusion and clutter by using progressive training and a data engine with human feedback. It outperforms prior 3D generation models in human preference tests on real-world objects and scenes. We released code, weights, online demo, and a new challenging benchmark.

Latest updates

Installation

Follow the setup steps before running the following.

Single or Multi-Object 3D Generation

SAM 3D Objects can convert masked objects in an image, into 3D models with pose, shape, texture, and layout. SAM 3D is designed to be robust in challenging natural images, handling small objects and occlusions, unusual poses, and difficult situations encountered in uncurated natural scenes like this kidsroom:

For a quick start, run python demo.py or use the the following lines of code:

import sys

# import inference code
sys.path.append("notebook")
from inference import Inference, load_image, load_single_mask

# load model
tag = "hf"
config_path = f"checkpoints/{tag}/pipeline.yaml"
inference = Inference(config_path, compile=False)

# load image and mask
image = load_image("notebook/images/shutterstock_stylish_kidsroom_1640806567/image.png")
mask = load_single_mask("notebook/images/shutterstock_stylish_kidsroom_1640806567", index=14)

# run model
output = inference(image, mask, seed=42)

# export gaussian splat
output["gs"].save_ply(f"splat.ply")

For more details and multi-object reconstruction, please take a look at out two jupyter notebooks:

SAM 3D Body

SAM 3D Body (3DB) is a robust promptable foundation model for single-image 3D human mesh recovery (HMR).

As a way to combine the strengths of both SAM 3D Objects and SAM 3D Body, we provide an example notebook that demonstrates how to combine the results of both models such that they are aligned in the same frame of reference. Check it out here.

License

The SAM 3D Objects model checkpoints and code are licensed under SAM License.

Contributing

See contributing and the code of conduct.

Contributors

The SAM 3D Objects project was made possible with the help of many contributors.

Robbie Adkins, Paris Baptiste, Karen Bergan, Kai Brown, Michelle Chan, Ida Cheng, Khadijat Durojaiye, Patrick Edwards, Daniella Factor, Facundo Figueroa, Rene de la Fuente, Eva Galper, Cem Gokmen, Alex He, Enmanuel Hernandez, Dex Honsa, Leonna Jones, Arpit Kalla, Kris Kitani, Helen Klein, Kei Koyama, Robert Kuo, Vivian Lee, Alex Lende, Jonny Li, Kehan Lyu, Faye Ma, Mallika Malhotra, Sasha Mitts, William Ngan, George Orlin, Peter Park, Don Pinkus, Roman Radle, Nikhila Ravi, Azita Shokrpour, Jasmine Shone, Zayida Suber, Phillip Thomas, Tatum Turner, Joseph Walker, Meng Wang, Claudette Ward, Andrew Westbury, Lea Wilken, Nan Yang, Yael Yungster

Citing SAM 3D Objects

If you use SAM 3D Objects in your research, please use the following BibTeX entry.

@article{sam3dteam2025sam3d3dfyimages,
      title={SAM 3D: 3Dfy Anything in Images}, 
      author={SAM 3D Team and Xingyu Chen and Fu-Jen Chu and Pierre Gleize and Kevin J Liang and Alexander Sax and Hao Tang and Weiyao Wang and Michelle Guo and Thibaut Hardin and Xiang Li and Aohan Lin and Jiawei Liu and Ziqi Ma and Anushka Sagar and Bowen Song and Xiaodong Wang and Jianing Yang and Bowen Zhang and Piotr Dollár and Georgia Gkioxari and Matt Feiszli and Jitendra Malik},
      year={2025},
      eprint={2511.16624},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2511.16624}, 
}

sam-3d-objects (facebookresearch/sam-3d-objects)

Searching Database…

7.5K ·GitHub
Sam-3d-objects
这是什么？SAM 3D Objects 是 Meta 推出的基础模型，能够从单张二维图像中重建完整的三维形状几何、纹理和物体布局，即使在杂乱的真实场景中存在物体遮挡，也能做到。
这是我自己运行的模型，还是只是一个在线演示？它是开源的研究代码和模型权重，你可以通过脚本或 Jupyter 笔记本自行运行，同时还附带了一个在线演示和基准测试。它是 SAM 3D 这一对模型中的一个，另一个是用于人体网格重建的 SAM 3D Body。
我怎样用自己的图片试一试？完成安装步骤后，运行 `python demo.py`，或者使用仓库中附带的单物体和多物体 Jupyter 笔记本，进行更详细的重建流程。
运行之前需要先安装哪些东西？README 指向了几项单独的安装步骤，必须在运行演示脚本或笔记本之前完成，但本段摘录中并未详细列出具体的依赖清单。
这份代码可以免费用于商业用途吗？模型权重和代码以定制的 'SAM License' 发布，而不是 MIT 或 Apache 这类标准开源许可证，因此使用条款与常见的宽松许可证有所不同。
这是为谁设计的？它面向需要单张图像三维物体重建的计算机视觉和机器人领域研究人员或工程师。从基准测试、排行榜以及大量学术引用列表中，可以看出这一定位。
SAM 3D Objects 和 SAM 3D Body 是什么关系？它们是同一项目中配套的一对模型：SAM 3D Objects 负责重建物体，SAM 3D Body 负责人体网格恢复。仓库中还包含一个 Jupyter 笔记本，演示如何将两者的输出对齐到同一个坐标系中。
SAM 3D Objects 比之前的三维生成模型好在哪里？它采用了渐进式训练和带有人类反馈的数据引擎。在人类偏好测试中，它尤其在真实世界的物体和场景上优于之前的三维生成模型，而不只是干净的棚拍图像。
一张图片里能处理多个物体吗？可以。除了单物体流程之外，还专门提供了一个多物体 Jupyter 笔记本，用于从单张图像中重建多个带掩码的物体。
有没有办法对结果进行基准测试或比较？项目发布了一个高难度的基准测试集，截至最新更新，还提供了一个 3D Artist Object Set，以及用于比较模型结果的 Hugging Face Leaderboard。
What are similar projects?ml-sharp, WorldSculpt, vggt, Mira-Scene, pointdit, wuji-ego-mint, InsertAny3D, shap-e
Which topics is it in?meta, image-to-3d, demo, 3d-reconstruction, foundation-models, jupyter-notebook, paper, robotics, 3d, academic-research, generative-ai, computer-vision
Creator
facebookresearch
Size
268 MB
Stars
7.5K
Updated
CommitJun 02
Age
1y
Commits · 90 Days