Code-as-World
Agentic Discovery of Executable World Representations for
Physical Reasoning
A world becomes intelligible to evolving intelligence when it can be represented, executed, and verified.
Overview
Pixels are evidence of the physical world, not its ontology. A pixel-level observation records how the world appears at a particular moment and from a particular viewpoint, but does not directly specify what exists within it, how it is structured, or what governs its evolution.
Code-as-World introduces code as executable representations for the physical world and an agentic process for discovering them through iterative simulation and verification. In doing so, it turns raw abundant observations into reusable physical data: explicit states, dynamics, and mechanisms that capture not only what was seen, but the underlying world that could have produced it. This provides scalable physical supervision, enabling our models to achieve state-of-the-art performance on quantitative physical reasoning.
News
Get started
The release includes local inference and QuantiPhy evaluation for
Code-as-World-VL-4B and
Code-as-World-VL-9B,
plus a video-driven abstraction example.
Installation
Use Python 3.10 or 3.11 on a CUDA host.
git clone https://github.com/MirroS-Lab/Code-as-World.git
cd Code-as-World
python -m venv .venv
source .venv/bin/activate
pip install -r requirements/inference.txt
hf download MirroS-Lab/Code-as-World-VL-4B --local-dir weights/4b
hf download MirroS-Lab/Code-as-World-VL-9B --local-dir weights/9b
Clone QuantiPhy and download its validation videos:
git clone https://github.com/Paulineli/QuantiPhy.git /path/to/QuantiPhy
hf download PaulineLi/QuantiPhy-validation \
--repo-type dataset \
--local-dir /path/to/QuantiPhy-validation
QuantiPhy evaluation
Run the released evaluation directly from the Code-as-World repository:
python -m code_as_world.evaluation 4b \
--input-csv /path/to/QuantiPhy/quantiphy_validation.csv \
--video-dir /path/to/QuantiPhy-validation/validation_videos
python -m code_as_world.evaluation 9b \
--input-csv /path/to/QuantiPhy/quantiphy_validation.csv \
--video-dir /path/to/QuantiPhy-validation/validation_videos
Each run writes an evaluator-compatible prediction CSV, a single-run metric summary,
and the raw generations to outputs/quantiphy/. To evaluate both CSV files
with the official QuantiPhy evaluator:
pip install pandas
python /path/to/QuantiPhy/evaluator.py \
outputs/quantiphy \
outputs/quantiphy_metrics \
--gt_file /path/to/QuantiPhy/quantiphy_validation.csv
OpenAI-compatible serving
The checkpoints can also be exposed through the standard vLLM API:
CUDA_VISIBLE_DEVICES=0 vllm serve weights/4b \
--served-model-name code-as-world-4b \
--chat-template code_as_world/templates/qwen3_5_no_think.jinja \
--chat-template-content-format openai \
--default-chat-template-kwargs '{"enable_thinking":false}' \
--max-model-len 4608 \
--gpu-memory-utilization 0.90 \
--media-io-kwargs '{"video":{"num_frames":16,"fps":-1,"video_backend":"openpangu"}}' \
--mm-processor-kwargs '{"do_sample_frames":false}' \
--mm-processor-cache-gb 0 \
--generation-config vllm
For the 9B checkpoint, replace weights/4b and code-as-world-4b with
weights/9b and code-as-world-9b.
Video-driven abstraction example
Install MuJoCo and run the bundled ballistic soccer case:
pip install -r requirements/simulation.txt
python -m code_as_world.simulation
The rendered video and trajectory are written to outputs/simulations/. Use
python -m code_as_world.simulation --no-render when only the trajectory is needed.
TODO
- Release Code-as-World-VL checkpoints and inference recipes
- Technical report, project page, and blog release
Acknowledgements
We sincerely thank the teams behind the following projects for making their work available to the community:
... and many other excellent open-source projects.
Citation
If you find Code-as-World useful, please cite:
@article{mirros2026codeasworld,
title = {Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning},
author = {{MirroS Team}},
journal = {arXiv preprint arXiv:2608.27549},
year = {2026}
}
Code-as-World 是一个研究项目，把物理世界表示为可执行代码，通过仿真与验证的智能体循环来发现，让模型获得可复用的物理数据（状态、动力学、机制），而不只是原始像素。