ml-sharp README

The project's README from github.com/apple-aiml-research/ml-sharp. As one markdown file: README.md.

Sharp Monocular View Synthesis in Less Than a Second

Project Page arXiv

This software project accompanies the research paper: Sharp Monocular View Synthesis in Less Than a Second by Lars Mescheder, Wei Dong, Shiwei Li, Xuyang Bai, Marcel Santos, Peiyun Hu, Bruno Lecouat, Mingmin Zhen, Amaël Delaunoy, Tian Fang, Yanghai Tsin, Stephan Richter and Vladlen Koltun.

We present SHARP, an approach to photorealistic view synthesis from a single image. Given a single photograph, SHARP regresses the parameters of a 3D Gaussian representation of the depicted scene. This is done in less than a second on a standard GPU via a single feedforward pass through a neural network. The 3D Gaussian representation produced by SHARP can then be rendered in real time, yielding high-resolution photorealistic images for nearby views. The representation is metric, with absolute scale, supporting metric camera movements. Experimental results demonstrate that SHARP delivers robust zero-shot generalization across datasets. It sets a new state of the art on multiple datasets, reducing LPIPS by 25–34% and DISTS by 21–43% versus the best prior model, while lowering the synthesis time by three orders of magnitude.

Getting started

We recommend to first create a python environment:

conda create -n sharp python=3.13

Afterwards, you can install the project using

pip install -r requirements.txt

To test the installation, run

sharp --help

Using the CLI

To run prediction:

sharp predict -i /path/to/input/images -o /path/to/output/gaussians

The model checkpoint will be downloaded automatically on first run and cached locally at ~/.cache/torch/hub/checkpoints/.

Alternatively, you can download the model directly:

wget https://ml-site.cdn-apple.com/models/sharp/sharp_2572gikvuh.pt

To use a manually downloaded checkpoint, specify it with the -c flag:

sharp predict -i /path/to/input/images -o /path/to/output/gaussians -c sharp_2572gikvuh.pt

The results will be 3D gaussian splats (3DGS) in the output folder. The 3DGS .ply files are compatible to various public 3DGS renderers. We follow the OpenCV coordinate convention (x right, y down, z forward). The 3DGS scene center is roughly at (0, 0, +z). When dealing with 3rdparty renderers, please scale and rotate to re-center the scene accordingly.

Rendering trajectories (CUDA GPU only)

Additionally you can render videos with a camera trajectory. While the gaussians prediction works for all CPU, CUDA, and MPS, rendering videos via the --render option currently requires a CUDA GPU. The gsplat renderer takes a while to initialize at the first launch.

sharp predict -i /path/to/input/images -o /path/to/output/gaussians --render

# Or from the intermediate gaussians:
sharp render -i /path/to/output/gaussians -o /path/to/output/renderings

Evaluation

Please refer to the paper for both quantitative and qualitative evaluations. Additionally, please check out this qualitative examples page containing several video comparisons against related work.

Citation

If you find our work useful, please cite the following paper:

@inproceedings{Sharp2025:arxiv,
  title      = {Sharp Monocular View Synthesis in Less Than a Second},
  author     = {Lars Mescheder and Wei Dong and Shiwei Li and Xuyang Bai and Marcel Santos and Peiyun Hu and Bruno Lecouat and Mingmin Zhen and Ama\"{e}l Delaunoy and Tian Fang and Yanghai Tsin and Stephan R. Richter and Vladlen Koltun},
  journal    = {arXiv preprint arXiv:2512.10685},
  year       = {2025},
  url        = {https://arxiv.org/abs/2512.10685},
}

Acknowledgements

Our codebase is built using multiple opensource contributions, please see ACKNOWLEDGEMENTS for more details.

License

Please check out the repository LICENSE before using the provided code and LICENSE_MODEL for the released models.

ml-sharp (apple-aiml-research/ml-sharp)

Searching Database…

9K ·GitHub
Ml-sharp
这是什么？SHARP（ml-sharp）是一套研究代码，可以把一张单独的照片转换成照片级真实感的 3D 场景。它只需一次前向推理，在普通 GPU 上不到一秒就能预测出场景的 3D Gaussian Splatting 表示，之后还能从附近的视角实时渲染这个场景。
这是现成的应用，还是带命令行的研究代码？这是配套某篇学术论文的研究代码，以命令行工具的形式提供，用于运行预测，并不是打包好的终端用户应用。
我该怎么把它跑起来？创建 Python 环境，安装项目，然后运行自带的测试来验证安装；首次运行时会自动下载预训练模型权重文件，并缓存到本地。
使用它一定需要 CUDA 显卡吗？从图像预测高斯表示可以在 CPU、CUDA 或 Apple MPS 上运行，但使用 --render 选项渲染相机轨迹视频目前只能在 CUDA GPU 上运行。
使用 --render 视频选项时有什么需要注意的坑吗？除了只支持 CUDA 之外，底层的 gsplat 渲染器在首次启动时初始化明显更慢。
我可以自由使用这些代码和已发布的模型吗？代码和模型分别有各自的许可证文件，所以在使用仓库中的任何内容之前，两者都要仔细检查。
和之前的方法相比，SHARP 有多快、多准？SHARP 树立了新的业界最优水平：相比此前最好的模型，LPIPS 降低了 25-34%，DISTS 降低了 21-43%，同时把合成时间缩短了三个数量级。
生成的 3D 场景是什么格式？输出的是一个 3D Gaussian Splatting 格式的 .ply 文件，保存在输出文件夹中。它兼容常见的公开 3DGS 渲染器，并采用 OpenCV 的相机坐标约定。
生成的场景是真实尺寸吗？是的。这个 3D Gaussian 表示带有绝对尺度，是公制的，因此可以支持真实的公制相机移动，而不是任意比例。
What are similar projects?nerfstudio, vggt, instant-ngp, InsertAny3D, WorldSculpt, vggt-omega, gaussian-splatting, sam-3d-objects
Which topics is it in?real-time-rendering, gaussian-splatting, image-to-3d, neural-rendering, 3d-reconstruction, paper, 3d, academic-research, inference, nvidia, computer-vision, cuda
Creator
apple-aiml-research
Size
185 MB
Stars
9K
Updated
CommitSep 11
Age
10mo
Commits · 90 Days