What's new 🔥
- [2026/09/26] Purpose-built engines on Jetson Thor. FP8 pi05, LingBot-VLA, LingBot-VA and GR00T N1.7 now run on custom engines with fused Triton kernels, with no calibration pass or native build. Engines · Reproduce.
- [2026/09/17] RTX 5090 support. Deploy on your workstation with the same Runtime API used on Jetson Thor. Setup · Reproduce.
- [2026/09/16] RTX 4090 support. Desktop inference and WebSocket serving with dedicated installation profiles. Setup · Reproduce.
- [2026/09/15] Full-source release. Eight robotics model families, acceleration kernels, and Python / WebSocket serving through one Runtime. Get started.
- [2026/09/15] Jetson Thor benchmarks. Up to 33.78× speedup with LingBot-VA @2V/4A, using FP8 and fewer sampling steps. Results · Reproduce.
Results
Prediction p50 on Jetson Thor (ms), measured September 15, 2026.
We’ve seen up to 33.78× speedup with no observed loss in task performance in our real-robot tests.
| Model |
Acceleration line |
PyTorch |
InstinctFlash |
Speedup |
| LingBot-VA |
FP8 · 25V/50A |
15506.32 |
2891.74 |
5.36× |
| ↳ LingBot-VA |
FP8 · 2V/4A |
2071.29 |
459.10 |
4.51× |
| LingBot-VLA-4B |
FP8 |
624.22 |
221.53 |
2.82× |
| LingBot-VLA-V2-6B |
FP8 |
734.56 |
394.11 |
1.86× |
| Cosmos3 Edge DROID |
NUMERIC · UniPC4 / CFG3 |
3393.78 |
1048.01 |
3.24× |
| Cosmos3 Nano DROID |
NUMERIC · UniPC4 / CFG3 |
10184.68 |
4772.38 |
2.13× |
| pi05 |
FP8 |
408.58 |
51.85 |
7.88× |
| GR00T N1.7 |
BITEXACT |
139.50 |
117.30 |
1.19× |
| DreamZero DROID |
FP8 · 16 steps · dynamic cache |
23563.08 |
11899.42 |
1.98× |
VA measures early continuations; each row compares the same schedule.
The 33.78× headline includes 25V/50A → 2V/4A.
FP8 and sampling changes are optional.
The LingBot-VA, LingBot-VLA and pi05 FP8 rows and the Cosmos3 Edge row were measured on September 15 with the engines and kernels that InstinctFlash's own engines and Triton kernels have since replaced; the current source's clean-install checks run these modes but have not repeated this table's measurements.
Reproduction commands
Install
git clone https://github.com/General-Instinct/InstinctFlash && cd InstinctFlash
python3 -m venv .venv-core
source .venv-core/bin/activate
python -m pip install . uv==0.12.5
The Python 3.10+ core inspects checkpoints and plans without PyTorch or a GPU.
Inference uses a separate, pinned environment for each model family. For RTX 4090:
python3 scripts/bootstrap_vendor.py install pi05 --target rtx4090 \
--python python3.12 --root ~/ifl-pi05-4090 --ptxas /usr/local/cuda/bin/ptxas
source ~/ifl-pi05-4090/activate.sh
Use va, vla4, vla2, pi05, groot, edge, nano or dreamzero.
Edge and Nano use Python 3.13; the other families use Python 3.12.
The bootstrap installs the upstream source, compatibility patches, core and adapter.
Model weights are downloaded separately. See RTX 5090 setup,
RTX 4090 setup
or Jetson Thor setup, which selects --target jetson_thor.
The Thor FP8 engines are Python and Triton, compiled on first use; no native build is needed.
Load a model
Your fine-tuned checkpoint — the expected case. Point serve at the training output; it
detects the family, writes the small instinctflash.json declaration from what the checkpoint
itself proves, and starts serving. One command:
instinctflash serve /path/to/your/checkpoint
Anything the checkpoint cannot prove is asked for explicitly, never guessed. Once the
declaration exists (serve writes it on first run), the same directory also loads in Python:
from instinctflash import Runtime
runtime = Runtime.from_pretrained("/path/to/your/checkpoint")
A stock release — use its Hub id after installing the family's environment:
runtime = Runtime.from_pretrained("robbyant/lingbot-va-posttrain-robotwin")
| family |
model id |
| LingBot-VA (5B WAM) |
robbyant/lingbot-va-posttrain-robotwin |
| LingBot-VLA-4B |
robbyant/lingbot-vla-4b-posttrain-robotwin |
| LingBot-VLA-V2-6B |
robbyant/lingbot-vla-v2-6b-robotwin |
| pi0.5 |
lerobot/pi05_base · lerobot/pi05_libero_finetuned_v044 |
| GR00T-N1.7-3B |
nvidia/GR00T-N1.7-3B |
| Cosmos3 policies |
nvidia/Cosmos3-Edge-Policy-DROID · nvidia/Cosmos3-Nano-Policy-DROID |
| DreamZero |
GEAR-Dreams/DreamZero-DROID |
Fine-tunes reuse their family's adapter; quality is evaluated per checkpoint.
The same Runtime defaults to precision="native" with a BITEXACT transformation ceiling.
Use tier_ceiling="numeric" to allow numerical changes, or precision="fp8"
(CLI: --fp8) to explicitly enable FP8. Step schedules are selected separately.
See precision policy.
DreamZero's opt-in dynamic step cache requires
tier_ceiling="behavioral" with either precision.
Get actions
In process — this is the whole Python API:
with runtime.episode(prompt="put the bottle in the dustbin") as episode:
while not done:
result = episode.predict(observation)
action = result["action"]
observation is a dict in the model's own format; result["action"] contains its action array.
For LingBot-VA, pass executed_action=... when the controller changes a predicted action
chunk, so the next prediction uses the actions actually executed.
Over the network — the serve command above hosts the same runtime behind the
msgpack-over-websocket wire protocol the pi0/openpi ecosystem already speaks, so existing
robot-side clients connect unchanged (pip install openpi-client):
from openpi_client.websocket_client_policy import WebsocketClientPolicy
client = WebsocketClientPolicy("my-server", 8000)
result = client.infer(observation)
action = result["action"]
The prompt rides in the observation; a changed prompt starts a new episode, and a client can
say it explicitly with {"reset": True, ...}. Four flags cover the rest:
--serve.dry_run — preflight only: device, declaration, plan. No weights, no GPU.--serve.smoke — load, produce one action, exit.--serve.seed — seed native execution for paired comparisons; FP8 serving rejects this option.--serve.viz — stream observations, actions and latency to a Rerun viewer.
The second verb, instinctflash validate <dir>, checks a checkpoint is publishable; given
--validate.teacher_outcomes/.student_outcomes/.margin it also certifies non-inferiority and
stamps the certificate into the package.
Benchmark acceleration and quantization
After the vendor and auxiliary-asset preparation, reproduce paired
eager/default/selected Runtime measurements with the included inputs and fixed
checkpoint revision. The Thor FP8 engines need no native build.
Keep the model and asset environments activated. For RTX 4090:
python -I -m benchmarks.regression.reproduce prepare --target rtx4090 \
--model pi05 --mode fp8 --output pi05-inputs
python -I -m benchmarks.regression.reproduce run --prepared pi05-inputs --output pi05-results
python -I -m benchmarks.regression.serve_smoke --prepared pi05-inputs --output pi05-serving
run writes checked JSON/CSV reports and full action arrays. serve_smoke tests
the actual CLI and WebSocket pipeline across two episodes. Use --mode native
for default precision; FP8, numerical compilation and changed schedules are
explicit selections. Reproduction guide.
For additional framework comparisons, use the pinned comparison recipes.
Compare original and optimized models with instinctflash eval. Reports separate
latency, action agreement and simulator task success.
instinctflash eval adapters
instinctflash eval coverage --run /path/to/run
instinctflash eval --registry plan.registry.json report --run /path/to/run
See the evaluation guide to create and run
paired LIBERO / RoboTwin experiments, or benchmark details
for acceleration and quantization protocols.
The native qualification workflow
adds fresh-start admission, retained failures and checkpoint-specific evidence for each device.
LingBot-VA Hub IDs retain native step counts; 2V/4A requires an explicit nfe selection.
Shared BF16 fusion, a Triton fused linear + ReLU², provides an opt-in NUMERIC path for Cosmos Edge.
Shared tensor caching and prefill separation extend native Cosmos optimization to Edge and Nano; exact caching and NUMERIC compilation remain separate options.
Framework overview
InstinctFlash keeps model declarations, optimization planning, runtime execution, and evidence in
one inspectable path, whether it is called from Python or the command line.
Architecture
A checkpoint carries a short declaration of what it is. The runtime reads the declaration, decides
which optimizations are provably valid for those weights, applies them, and shows its work:
checkpoint ─▶ adapter ─▶ planner ─▶ engine passes ─▶ actions
declares what decides what apply and measure
the model is is valid (no GPU, each optimization
no weights needed)
Optimization is organized in six layers, by what each one changes:
| layer |
|
changes |
| 1 |
MODEL |
what is computed — distillation, step reduction, checkpoint compression (InstinctCompress, instinct-pdd) |
| 2 |
GRAPH |
when work is issued — prefill extraction, CUDA-graph capture, memory planning |
| 3 |
CACHE |
what is recomputed — KV reuse, cross-attention and episode caches |
| 4 |
ATTENTION |
how tokens mix — fused Triton attention, hybrid and linear attention |
| 5 |
KERNEL |
how a kernel is written — backend and layout dispatch, fusion (instinctflash/kernels) |
| 6 |
HARDWARE |
what it executes on — FP8, device-specific kernel targets, Jetson-class edge devices (instinctflash/engines) |
Layer 1 changes the weights and produces a checkpoint; it lives in the companion repos. Layers
2–6 change how the weights execute and produce a plan; they are the runtime in this repo. The
layers are not a priority order — the runtime measures where the time actually goes and starts
there.
Add a model
To add your own model family, declare an instinctflash.adapters entry point and pip install
your package — see examples/external_plugin/.
Roadmap
- Few-step distillation, when needed — only after native optimizations miss a declared
edge control budget; compare each student with its teacher and the matched untrained
schedule using paired closed-loop evaluation.
- LingBot-VA on the edge engine — native and FP8 serving on Jetson Thor,
with paired inference and WebSocket checks for full and 2V/4A schedules.
- Attention upgrades — a faster NUMERIC-tier attention arm beside the BITEXACT default
for pi05-class models; hybrid and linear attention for long-context world models.
- Cosmos3 and DreamZero on Thor — pinned installation, paired inference
and installed CLI/WebSocket checks; task quality remains a separate evaluation.
- Device-specific serving defaults — measure each family and operating point;
select a verified path within the caller's precision constraints. LingBot-VLA-V2
native Thor capture and LingBot-VA saturation profiling are complete; selective
VA action capture showed no speedup and stays experimental.
Execution-bound budget selection is available; the expanded V2 H100
evaluation remains a SCREEN.
Acknowledgements
We thank the following projects and their contributors for the code, models, tools,
and ideas that InstinctFlash builds on:
- Cosmos,
DreamZero,
Isaac GR00T,
LingBot-VA,
LingBot-VLA, and
LingBot-VLA-V2 — upstream model
implementations and checkpoints.
- FlashAttention and
NVIDIA CUTLASS — attention kernels and
matrix-multiplication templates used by the upstream models and frameworks.
- Hugging Face,
PyTorch, and
Triton — framework, compiler, and model tooling.
- LeRobot — model implementations,
preprocessing, and robotics tooling.
- LIBERO,
RoboLab, and
RoboTwin — simulation environments and
evaluation infrastructure.
- msgpack-numpy and
OpenPI — π0/π0.5 model code
and client serialization from OpenPI, whose serialization implementation adapts
msgpack-numpy.
- vLLM-Omni — design references for
KV-cache and conditioning-cache management.
Third-party code and model assets remain subject to their respective licenses.
See the license and attribution notices accompanying each component.
InstinctFlash 是一个面向机器人基础模型的高性能服务运行时——加载 GR00T、Cosmos3、pi0.5、LingBot-VA/VLA 和 DreamZero 等检查点，应用加速（FP8、融合 Triton 内核、缓存），并通过 Python 或 WebSocket 提供服务，在 Jetson Thor 上宣称最高 33.78 倍加速。