What is this?
RoboICL is a research framework that lets a frozen, general-purpose vision-language model (GPT-6 Astra) control a robot in simulation purely through in-context learning, without training a dedicated robot policy.
RoboICL is a research framework that lets a frozen, general-purpose vision-language model (GPT-6 Astra) control a robot in simulation purely through in-context learning, without training a dedicated robot policy.Is this a trained robot policy or something else?No model weights are trained here: it's a runner that packages demonstration trajectories and the model's own past actions as prompt context, then parses the VLM's predictions into robot actions.
No model weights are trained here: it's a runner that packages demonstration trajectories and the model's own past actions as prompt context, then parses the VLM's predictions into robot actions.How do I get this running?Clone the repo, point two separate Python environments at your Isaac Sim-based simulator and your multimodal policy process, install RoboICL's dependencies into each, then run the data scripts to fetch the RoboDojo asset tree and any one-shot reference bundles before launching a rollout.
Clone the repo, point two separate Python environments at your Isaac Sim-based simulator and your multimodal policy process, install RoboICL's dependencies into each, then run the data scripts to fetch the RoboDojo asset tree and any one-shot reference bundles before launching a rollout.What do I need installed before trying this?Git with Git LFS, an Isaac Sim 5.1-compatible simulator environment, and access to the GPT-6 Astra model API, on top of the two separate Python environments for simulator and policy.
Git with Git LFS, an Isaac Sim 5.1-compatible simulator environment, and access to the GPT-6 Astra model API, on top of the two separate Python environments for simulator and policy.Is setup heavy, or can I just pip install and go?Setup is heavy by design: exact pinned commits of RoboDojo, XPolicyLab, IsaacLab and cuRobo are recorded and verified before launch, and asset/reference bundles are checked by SHA256 rather than regenerated on the fly.
Setup is heavy by design: exact pinned commits of RoboDojo, XPolicyLab, IsaacLab and cuRobo are recorded and verified before launch, and asset/reference bundles are checked by SHA256 rather than regenerated on the fly.What platform does this run on?It's Linux-only, built around an NVIDIA/CUDA simulator stack; there's no mention of macOS or Windows support.
It's Linux-only, built around an NVIDIA/CUDA simulator stack; there's no mention of macOS or Windows support.What is GPT-6 Astra here?It's the frozen multimodal vision-language model RoboICL queries as a few-shot robot learner — the repo provides no model weights of its own, just the runner that prompts it.
It's the frozen multimodal vision-language model RoboICL queries as a few-shot robot learner — the repo provides no model weights of its own, just the runner that prompts it.What's the difference between demonstration context and interaction memory?Demonstration context supplies recorded robot trajectories as few-shot examples, while interaction memory retains the model's own actions, controller feedback, and observations from the current rollout for self-correction.
Demonstration context supplies recorded robot trajectories as few-shot examples, while interaction memory retains the model's own actions, controller feedback, and observations from the current rollout for self-correction.What tasks does it evaluate on?The paper benchmark covers 8 'Open' tasks tested zero-shot and 22 non-Open tasks given one demonstration each, backed by over 800 interactive evaluation rollouts released with the code.
The paper benchmark covers 8 'Open' tasks tested zero-shot and 22 non-Open tasks given one demonstration each, backed by over 800 interactive evaluation rollouts released with the code.What does the robot actually see and do?At each step the model views a triptych of left-wrist, head, and right-wrist camera images plus proprioception, then predicts a bounded sequence of dual-arm Cartesian actions executed in closed loop in the RoboDojo simulator.
At each step the model views a triptych of left-wrist, head, and right-wrist camera images plus proprioception, then predicts a bounded sequence of dual-arm Cartesian actions executed in closed loop in the RoboDojo simulator.What are similar projects?Zero-WAM, ZDTaichu5.0-9B, flex-pi, FluxVLA, StreamPI, Zetta-Embodiment, LoongForge, dreamforge
Zero-WAM, ZDTaichu5.0-9B, flex-pi, FluxVLA, StreamPI, Zetta-Embodiment, LoongForge, dreamforgeWhich topics is it in?in-context-learning, vision-language-action-model, vision-language, vla, robot, embodied-ai, vlm, arxiv, paper, robotics, simulation, gpt
in-context-learning, vision-language-action-model, vision-language, vla, robot, embodied-ai, vlm, arxiv, paper, robotics, simulation, gpt