InWorld README

The project's README from github.com/Murea-Labs/InWorld. As one markdown file: README.md.

Murea-InWorld — From imagination to living worlds

Type what you do, and the world films it.

Open-source video generation with synchronized sound from text and reference images, and an agent that turns your actions into a continuing, filmed story.

Website Paper PDF GitHub code Hugging Face weights Code license

English | 中文

🎬 Demo · 🚀 Quick start · 🧭 Guides · 📚 Citation · ✉️ Contact

🎬 Demo

Text-driven storytelling with Murea-InWorld · 2 min 19 s · Download the original-quality video

https://github.com/user-attachments/assets/2617c0f3-ded2-4a8f-81bb-b77095c92b0e

[!NOTE] The Murea-Fast weights are available on Hugging Face. Download murea-fast.safetensors and follow the deployment guide to verify the checkpoint and set up the LTX-2.5 base components.

📄 Abstract

Adapted from our technical report. The report evaluates Murea-Pro and Murea-Fast; this repository provides Murea-Mind and the Murea-Fast inference service.

Interactive world models aim to simulate environments that evolve under user actions while retaining the consequences of past interactions. Extending such worlds to sight and sound requires persistent entities, changing scene states, and local audiovisual dynamics to remain coherent across turns. This paper presents Murea-InWorld, an interactive audiovisual world model that couples explicit world-state management with neural audiovisual generation. Its world agent, Murea-Mind, maintains characters, objects, locations, and narrative progress, interprets user actions in context, and compiles planned events into generation conditions. Observations of generated frames inform subsequent planning and selected state updates, closing the interaction loop.

Its audiovisual generator, Murea, realizes these events through Omni-Reference Generation, using visual references to anchor entity appearance and structured descriptions to specify actions and dialogue. Streaming generation uses modality-specific histories, with longer audio context for unfinished utterances. Degraded-history training prepares the model for imperfect rollout context, while sequential MeanFlow and distribution matching distillation enable few-step synthesis. Branching histories support alternative continuations, and co-op interaction brings multiple participants' actions into the same world.

We evaluate the generation components in complementary configurations. Murea-Pro achieves an Overall score of 81.13 on WBench Full-289, the highest among the compared systems. Murea-Fast achieves a Total score of 64.98 on OpenS2V-Eval and an aggregate score of 69.9 on OS2V-Pro, an extended evaluation of reference fidelity, attribute binding, rendering quality, and audio. Murea-Pro and Murea-Fast achieve state-of-the-art performance on their respective benchmarks, and both models surpass most evaluated commercial systems in their respective evaluations.

📣 News

✨ Highlights

🎮 What you can do

🖼️ Film a scene from references
Two characters and a place, one prompt, a clip with dialogue and sound. Murea-Fast
🔄 Continue a clip
Pick up from the last frames and keep the action going. Murea-Fast
🎭 Play a story
Type an action, watch it happen, choose what comes next. Murea-Mind
👥 Play with friends
A shared room, one clip per round, with optional AI teammates. Co-op mode

🧩 How it fits together

flowchart LR
    P([Player]) -->|action| A[Murea-Mind<br/>plans, checks, remembers]
    A <-->|text and images| L[(Language and image model APIs<br/>Gemini / OpenAI-compatible / DashScope)]
    A -->|prompt + reference images| F[Murea-Fast<br/>inference service on GPUs]
    F -->|video with sound| A
    A -->|clip + next choices| P

The two parts talk over a documented HTTP API, so the service can run on another machine or behind an SSH tunnel, and other clients can use it too.

🧭 Where to start

I want to… Start here
Play a story in the browser Murea-Mind quick start
Play with friends Co-op mode
Generate or continue a clip from the command line Murea-Fast quick start
Call the model from my own code API reference
Run the service on my GPUs Deployment
Know how fast it is and how much memory it needs Performance
Contribute CONTRIBUTING.md

⚙️ Components

Murea-Fast Murea-Mind
What it is Inference service for an 8-step distilled derivative of LTX-2.5, on a pinned SGLang Web app and Python package: a language-model director, deterministic checks, world memory and a critic
Runs on Linux and NVIDIA GPUs with CUDA. Verified on one H20 and on four H20s Any machine with Python 3.12; needs a language and image model API key and a Murea-Fast service
Interface HTTP API, command line Web UI, terminal, Python API

🚀 Quick start

1. Start Murea-Fast on a GPU machine. Install the environment and the LTX-2.5 base components as described in Deployment, download the murea-fast.safetensors checkpoint, and set the four paths in inference/configs/single-gpu.json. Set checkpoint to the absolute path of the downloaded file. Then:

cd inference
bash scripts/murea-fast prepare --config configs/single-gpu.json
bash scripts/murea-fast start --config configs/single-gpu.json
curl --fail http://127.0.0.1:30260/health

2. Run Murea-Mind on any machine. It needs uv and a key for text and images (Gemini, any OpenAI-compatible server, or DashScope):

git clone https://github.com/Murea-Labs/InWorld.git Murea-InWorld && cd Murea-InWorld/agent
cp .env.example .env          # set GEMINI_API_KEY and MUREA_URL=http://<gpu-host>:30260
./run.sh                      # then open http://127.0.0.1:8080

See the Murea-Mind README for other providers, Docker and SSH tunnels.

🗂️ Repository layout

Murea-InWorld/
├── LICENSE
├── CONTRIBUTING.md
├── SECURITY.md
├── inference/            Murea-Fast inference service
│   ├── configs/          service configurations and an example request
│   ├── docs/             deployment, API, performance, provenance
│   ├── scripts/          CLI wrapper, connector merge, acceptance run
│   ├── src/murea_fast/   launcher and client
│   ├── tests/
│   └── vendor/sglang/    pinned SGLang with the model implementation
└── agent/                Murea-Mind
    ├── murea_agent/      the agent (Python package)
    ├── app/              FastAPI server
    ├── frontend/         React UI (prebuilt in frontend/dist)
    ├── docs/             co-op mode design notes
    └── tests/

🔒 Security

Neither the inference service nor the agent has authentication. Read SECURITY.md before deploying them on a shared network.

🤝 Contributing

See CONTRIBUTING.md for the checks each part needs before a pull request.

📜 License

The code is released under the Apache License 2.0. The vendored SGLang in inference/vendor/ keeps its own copyright and license notices, including the files it carries under MIT and BSD-3-Clause. The Murea-Fast model weights are a derivative of LTX-2.5 and are governed by the LTX-2.x Community License Agreement; see Provenance for details. The agent calls third-party language and image model APIs, which have their own terms.

📚 Citation

@misc{murea_inworld2026,
  title        = {InWorld: From Imagination to Living Worlds with Agents and Fast Video Generation},
  author       = {{Murea Labs}},
  year         = {2026},
  url          = {https://murea.ai/page/paper/InWorld-technical-report.pdf}
}

✉️ Contact

Email: [email protected]

InWorld (Murea-Labs/InWorld)

storytelling
Searching Database…

3.4K ·GitHub
What is this?Murea-InWorld is an open-source system for generating video with synchronized sound from text and reference images, paired with an agent (Murea-Mind) that turns a player's typed actions into a continuing, filmed story.
Is this one tool or two separate systems?It is two linked components talking over an HTTP API: Murea-Fast, a GPU inference service that generates the actual video/audio, and Murea-Mind, a web app and Python package that acts as the story director and world-state manager.
How do I get this running?Start Murea-Fast on a GPU machine by installing the environment and LTX-2.5 base components per the deployment guide and downloading the murea-fast.safetensors checkpoint, then run Murea-Mind on any machine using uv.
What hardware and accounts do I need?Murea-Fast needs Linux with NVIDIA GPUs and CUDA (verified on one or four H20s); Murea-Mind just needs Python 3.12 and uv, plus an API key for a text/image model such as Gemini, an OpenAI-compatible server, or DashScope.
Any catches before deploying this?Neither Murea-Fast nor Murea-Mind has authentication built in, so you need to read SECURITY.md before putting either service on a shared network.
How fast is clip generation?With 8-step distilled sampling, a 5-second clip takes about 17 seconds on four H20 GPUs using sequence parallelism.
Can I play this with friends?Yes, co-op mode supports 2-4 participants (people or AI teammates) in a shared room, where each round is filmed as one shared clip.
How does the agent keep a story consistent across clips?Murea-Mind locks every character, object and place to a reference image with a fixed description, plans each story beat, checks it, films it, and reviews the result against the actual generated frames before continuing.
What does the output video look like?Clips are rendered at 1280x704 resolution and 24fps with synchronized audio, including spoken dialogue lines.
Creator
Murea-Labs
Size
237.8 MB
Stars
3.4K
Updated
CommitOct 20
Age
1mo
Commits · 90 Days