motion-bricks.cpp README

The project's README from github.com/localai-org/motion-bricks.cpp. As one markdown file: README.md.

motion-bricks.cpp

A C++23/GGML port of NVIDIA MotionBricks for CPU and Vulkan inference, with a stable C ABI suitable for PureGo.

The released batch-one G1 inference path is implemented end to end: strict GGUF loading, root/duration planning, pose-token prediction, VQ decoding, 418/414/413 feature conversion, style alignment, and skeletal animation output. CPU and Vulkan use the same public API and preserve the same duration and pose-token decisions in the reference suite.

The demo also offers optional native MuJoCo physics driven by GGML SONIC, including Kimodo animation playback, with target/reference/physical skeletons. The original G1 mode-0 policy passes independent CPU/Vulkan layer parity. See SONIC setup, results and limitations and the MuJoCo 3.12 upgrade notes. The browser sends WebSocket commands and renders buffered server-owned motion; see streaming and client QA.

Build

The normal build uses CMake and does not depend on Nix:

cmake --preset debug
cmake --build --preset debug
ctest --preset debug

Configuration downloads and SHA-256-verifies the published 0.73 GB G1 F32 GGUF and style bundles into generated/ when they are not already present. The same operation can be run explicitly:

python scripts/download_gguf_weights.py

For an offline or source-only build, preserve an existing local bundle or use cmake --preset debug -DMOTIONBRICKS_DOWNLOAD_MODELS=OFF. The repository and revision are configurable with MOTIONBRICKS_MODEL_REPOSITORY and MOTIONBRICKS_MODEL_REVISION.

On NixOS, enter the reproducible development shell first:

nix develop
cmake --preset debug
cmake --build --preset debug
ctest --preset debug

When the pinned ggml/ submodule is present, it is included automatically. The non-neural ABI and validation subset can also be built without GGML:

cmake -S . -B build/debug -G Ninja -DMOTIONBRICKS_ENABLE_GGML=OFF

The Go binding and demo use PureGo to load libmotionbricks at runtime; they do not use cgo or import "C". Once the native shared library has been built, the Go components therefore need no C compiler and can be built with cgo explicitly disabled:

cd demo
CGO_ENABLED=0 go build -o ../build/debug/bin/motionbricks-demo .

CGO_ENABLED=0 is optional but recommended for making this property explicit in builds and CI. It affects only the Go build—the native C++ library is still built separately with CMake.

The sanitizer lane is:

cmake --preset asan-ubsan
cmake --build --preset asan-ubsan
ctest --preset asan-ubsan

Current ABI

Start with the API selection guide: choose stateless MotionBricks inference, the optional animation controller, standalone SONIC inference, or the SONIC/MuJoCo physics controller. The inference-only C example uses caller-supplied poses without styles or an agent.

The installed C API uses only fixed-width scalars, pointers, and opaque heap handles. Callers never reproduce a C or C++ structure layout. All constructors have matching free functions, and no C++ exception crosses the ABI boundary.

The current CLI can report ABI information:

./build/debug/bin/motionbricks-cli abi

After producing the trusted safetensors intermediates described in reference/README.md, build and inspect an F32 runtime bundle with:

python scripts/convert_to_gguf.py \
  --safe-directory generated/safe \
  --output generated/g1-f32
./build/debug/bin/motionbricks-cli inspect generated/g1-f32

The released G1 inference path contains exactly 183,148,382 learned F32 parameters. The bundle loader validates the upstream revision, source checkpoint identities, component roles, tensor counts, parameter counts, anchor shapes, and the 34-joint parent topology before accepting a model.

Convert the 15 original demo styles with:

python scripts/convert_styles.py \
  --safe-directory generated/safe \
  --output generated/styles

At runtime the high-level flow is: load one immutable model, load one or more .mbstyle assets, create an agent, reset it from an initial style (or supply at least four frames of G1 context), set movement/facing/style on a command, then call mb_agent_plan. The returned motion owns row-major F32 root translations [frames,3] and local XYZW rotations [frames,34,4]. Call mb_agent_advance as playback progresses so replanning uses the generated motion as its next context.

The current implementation covers original preprocessed G1 styles. Direct Kimodo GLB-to-.mbstyle conversion remains subsequent integration work.

Weights

Ready-to-run native weights and all 15 upstream style primitives are published as MotionBricks-G1-GGML under the Hugging Face LocalAI-io organisation. NVIDIA currently distributes MotionBricks checkpoints through Git LFS in NVlabs/GR00T-WholeBodyControl, not a separate Hugging Face model repository, so the model card links to that pinned upstream revision. The downloader verifies the version-controlled distribution manifest before accepting any file.

The default build is pinned to Hugging Face commit cc2a47603dbc203a4f18f35dd06ed3611833f506 rather than the mutable main branch.

Interactive demo

The initial Go/Three.js demo renders the model's 34-joint skeleton alongside the four actual placed target-keyframe ghosts. It lets you steer relative to the current camera yaw with W/A/S/D, turn facing with the arrow keys, orbit/zoom the camera, and switch among the converted upstream styles. It uses the reusable PureGo binding and the same stateful native agent as other applications. See the demo guide for build, run, architecture, and headless-Chromium test instructions.

Pose tokens now use upstream-style Gumbel sampling by default. An explicit argmax mode remains available for diagnostics; sampling validation uses shared random draws rather than assuming identical framework RNG seeds.

The observational-parity tools can also turn the accepted upstream session into one immutable .mbreplay file consumed by the C++ CLI and the Three.js replay-only mode, plus a 345-frame MuJoCo reference MP4. See the reference guide.

The parity gate converts all 14 captured replans into an independent open-loop fixture, runs each through the real planner, writes aggregate, per-plan, and optional neural-boundary reports, and presents an upstream/native overlay with a joint-error scrubber. Strict CPU parity now passes all 14 plans against an upstream PyTorch CPU replay of the exact accepted sparse inputs. The original CUDA capture remains a separate cross-device diagnostic because near-tie pose argmax choices can differ across CPU and CUDA arithmetic. See open-loop native parity.

The opt-in CPU/Vulkan gate also passes all 14 plans on the recorded NVIDIA RTX 5070 Ti configuration. It uses exact durations and bounded public animation errors rather than requiring bit-identical floating-point intermediates.

Design

License

motion-bricks.cpp source code is licensed under the Apache License 2.0. NVIDIA's original model weights and converted GGUF/style distributions remain under the NVIDIA Open Model License reproduced with the published model. It permits derivative models and redistribution with conditions including retention of the agreement and attribution, Trustworthy AI terms, and trade compliance. Bundled third-party components retain their own licenses; the vendored Three.js files are covered by demo/web/vendor/THREE-LICENSE.txt.

motion-bricks.cpp (localai-org/motion-bricks.cpp)

physics
Searching Database…

304 ·GitHub
Motion-bricks.cpp
What is this?A C++23/GGML port of NVIDIA MotionBricks: a neural network that turns keyframe commands into skeletal character animations, for games and robots. It runs on CPU or Vulkan and exposes a stable C API.
So what do I actually get when I build it — a library, a CLI, or a full app?The core is a native C++ library with a stable C ABI: you pick stateless inference, an animation controller, standalone SONIC inference, or a SONIC/MuJoCo physics controller. A CLI, PureGo bindings, and a Go/Three.js demo sit on top.
How do I get it running?Configure and build with CMake — the configure step downloads and SHA-256-verifies the 0.73 GB G1 model bundle into generated/ (use -DMOTIONBRICKS_DOWNLOAD_MODELS=OFF to skip). Then build the Go demo.
What do I need installed before I can build it?CMake with a C++23 toolchain; Nix is only for NixOS. The Go binding and demo use PureGo, so once libmotionbricks is built they need no C compiler (CGO_ENABLED=0 works). The non-neural ABI subset can even build without GGML.
Can I animate my own characters, or only the bundled ones?Only the 15 bundled G1 styles for now — the current implementation covers original preprocessed G1 styles, and converting Kimodo GLB rigs to .mbstyle is explicitly listed as future integration work.
How do I pull animation frames out of it in my own code?Load the model and .mbstyle assets, create an agent, then call mbagentplan — it returns root translations [frames,3] and XYZW rotations [frames,34,4] for the 34-joint skeleton. Call mbagentadvance during playback so replanning uses the new motion as context.
Can I try it without writing any code first?Yes — the Go/Three.js web demo renders the 34-joint skeleton with keyframe ghosts: steer with W/A/S/D, turn facing with arrows, orbit/zoom the camera, and switch styles. A replay-only mode consumes .mbreplay files.
Is the whole thing under one license?It is split: the source code is Apache-2.0, but NVIDIA's model weights and the converted GGUF/style bundles stay under the NVIDIA Open Model License — redistribution allowed with attribution and agreement-retention conditions.
Will it run without an NVIDIA GPU?Yes — CPU inference works out of the box, and Vulkan gives a GPU path through the same API with identical duration and pose-token decisions; the published Vulkan parity runs used an RTX 5070 Ti.
Does the port faithfully match the original NVIDIA model?Yes — strict CPU parity passes all 14 captured replans against an upstream PyTorch CPU replay, and the opt-in CPU/Vulkan gate also passes all 14 plans. The original CUDA capture remains a separate cross-device diagnostic because near-tie pose choices can differ between CPU and CUDA arithmetic.
How big is the model and where do the weights come from?The G1 inference path has exactly 183,148,382 learned F32 parameters, published as a 0.73 GB GGUF bundle by the LocalAI-io organisation on Hugging Face. The build is pinned to a fixed upstream commit, and the loader verifies the distribution manifest before accepting any file.
Will I get the same animation every run?Not necessarily — pose tokens use upstream-style Gumbel sampling by default, so runs can differ. An explicit argmax mode remains available for diagnostics, and sampling validation uses shared random draws.
What is the MuJoCo physics part of the demo for?It adds optional native physics driven by GGML SONIC: Kimodo animation playback with target, reference, and physical skeletons, so generated motion can be inspected under real simulation rather than as raw keyframes.
What are similar projects?skin-tokens.cpp, kimodo.cpp, burn, kornia, kimodo, bullet3, Motion2MixamoRig, opencv_contrib
Which topics is it in?physics, cpu, gguf, threejs, vulkan, robotics, game-development, neural-network, 3d, animation, inference, c++
Creator
localai-org
Size
0.9 MB
Stars
304
Updated
CommitSep 21
Age
1mo
Commits · 90 Days