InWorld README

项目的 README,来自 github.com/Murea-Labs/InWorld。合并为一个 markdown 文件:README.md。

Murea-InWorld — From imagination to living worlds

Type what you do, and the world films it.

Open-source video generation with synchronized sound from text and reference images, and an agent that turns your actions into a continuing, filmed story.

Website Paper PDF GitHub code Hugging Face weights Code license

English | 中文

🎬 Demo · 🚀 Quick start · 🧭 Guides · 📚 Citation · ✉️ Contact

🎬 Demo

Text-driven storytelling with Murea-InWorld · 2 min 19 s · Download the original-quality video

https://github.com/user-attachments/assets/2617c0f3-ded2-4a8f-81bb-b77095c92b0e

[!NOTE] The Murea-Fast weights are available on Hugging Face. Download murea-fast.safetensors and follow the deployment guide to verify the checkpoint and set up the LTX-2.5 base components.

📄 Abstract

Adapted from our technical report. The report evaluates Murea-Pro and Murea-Fast; this repository provides Murea-Mind and the Murea-Fast inference service.

Interactive world models aim to simulate environments that evolve under user actions while retaining the consequences of past interactions. Extending such worlds to sight and sound requires persistent entities, changing scene states, and local audiovisual dynamics to remain coherent across turns. This paper presents Murea-InWorld, an interactive audiovisual world model that couples explicit world-state management with neural audiovisual generation. Its world agent, Murea-Mind, maintains characters, objects, locations, and narrative progress, interprets user actions in context, and compiles planned events into generation conditions. Observations of generated frames inform subsequent planning and selected state updates, closing the interaction loop.

Its audiovisual generator, Murea, realizes these events through Omni-Reference Generation, using visual references to anchor entity appearance and structured descriptions to specify actions and dialogue. Streaming generation uses modality-specific histories, with longer audio context for unfinished utterances. Degraded-history training prepares the model for imperfect rollout context, while sequential MeanFlow and distribution matching distillation enable few-step synthesis. Branching histories support alternative continuations, and co-op interaction brings multiple participants' actions into the same world.

We evaluate the generation components in complementary configurations. Murea-Pro achieves an Overall score of 81.13 on WBench Full-289, the highest among the compared systems. Murea-Fast achieves a Total score of 64.98 on OpenS2V-Eval and an aggregate score of 69.9 on OS2V-Pro, an extended evaluation of reference fidelity, attribute binding, rendering quality, and audio. Murea-Pro and Murea-Fast achieve state-of-the-art performance on their respective benchmarks, and both models surpass most evaluated commercial systems in their respective evaluations.

📣 News

✨ Highlights

🎮 What you can do

🖼️ Film a scene from references
Two characters and a place, one prompt, a clip with dialogue and sound. Murea-Fast
🔄 Continue a clip
Pick up from the last frames and keep the action going. Murea-Fast
🎭 Play a story
Type an action, watch it happen, choose what comes next. Murea-Mind
👥 Play with friends
A shared room, one clip per round, with optional AI teammates. Co-op mode

🧩 How it fits together

flowchart LR
    P([Player]) -->|action| A[Murea-Mind<br/>plans, checks, remembers]
    A <-->|text and images| L[(Language and image model APIs<br/>Gemini / OpenAI-compatible / DashScope)]
    A -->|prompt + reference images| F[Murea-Fast<br/>inference service on GPUs]
    F -->|video with sound| A
    A -->|clip + next choices| P

The two parts talk over a documented HTTP API, so the service can run on another machine or behind an SSH tunnel, and other clients can use it too.

🧭 Where to start

I want to… Start here
Play a story in the browser Murea-Mind quick start
Play with friends Co-op mode
Generate or continue a clip from the command line Murea-Fast quick start
Call the model from my own code API reference
Run the service on my GPUs Deployment
Know how fast it is and how much memory it needs Performance
Contribute CONTRIBUTING.md

⚙️ Components

Murea-Fast Murea-Mind
What it is Inference service for an 8-step distilled derivative of LTX-2.5, on a pinned SGLang Web app and Python package: a language-model director, deterministic checks, world memory and a critic
Runs on Linux and NVIDIA GPUs with CUDA. Verified on one H20 and on four H20s Any machine with Python 3.12; needs a language and image model API key and a Murea-Fast service
Interface HTTP API, command line Web UI, terminal, Python API

🚀 Quick start

1. Start Murea-Fast on a GPU machine. Install the environment and the LTX-2.5 base components as described in Deployment, download the murea-fast.safetensors checkpoint, and set the four paths in inference/configs/single-gpu.json. Set checkpoint to the absolute path of the downloaded file. Then:

cd inference
bash scripts/murea-fast prepare --config configs/single-gpu.json
bash scripts/murea-fast start --config configs/single-gpu.json
curl --fail http://127.0.0.1:30260/health

2. Run Murea-Mind on any machine. It needs uv and a key for text and images (Gemini, any OpenAI-compatible server, or DashScope):

git clone https://github.com/Murea-Labs/InWorld.git Murea-InWorld && cd Murea-InWorld/agent
cp .env.example .env          # set GEMINI_API_KEY and MUREA_URL=http://<gpu-host>:30260
./run.sh                      # then open http://127.0.0.1:8080

See the Murea-Mind README for other providers, Docker and SSH tunnels.

🗂️ Repository layout

Murea-InWorld/
├── LICENSE
├── CONTRIBUTING.md
├── SECURITY.md
├── inference/            Murea-Fast inference service
│   ├── configs/          service configurations and an example request
│   ├── docs/             deployment, API, performance, provenance
│   ├── scripts/          CLI wrapper, connector merge, acceptance run
│   ├── src/murea_fast/   launcher and client
│   ├── tests/
│   └── vendor/sglang/    pinned SGLang with the model implementation
└── agent/                Murea-Mind
    ├── murea_agent/      the agent (Python package)
    ├── app/              FastAPI server
    ├── frontend/         React UI (prebuilt in frontend/dist)
    ├── docs/             co-op mode design notes
    └── tests/

🔒 Security

Neither the inference service nor the agent has authentication. Read SECURITY.md before deploying them on a shared network.

🤝 Contributing

See CONTRIBUTING.md for the checks each part needs before a pull request.

📜 License

The code is released under the Apache License 2.0. The vendored SGLang in inference/vendor/ keeps its own copyright and license notices, including the files it carries under MIT and BSD-3-Clause. The Murea-Fast model weights are a derivative of LTX-2.5 and are governed by the LTX-2.x Community License Agreement; see Provenance for details. The agent calls third-party language and image model APIs, which have their own terms.

📚 Citation

@misc{murea_inworld2026,
  title        = {InWorld: From Imagination to Living Worlds with Agents and Fast Video Generation},
  author       = {{Murea Labs}},
  year         = {2026},
  url          = {https://murea.ai/page/paper/InWorld-technical-report.pdf}
}

✉️ Contact

Email: [email protected]

InWorld (Murea-Labs/InWorld)

正在搜索数据库…

3.4K ·GitHub
这是什么?Murea-InWorld 是一个开源系统,可以根据文本和参考图像生成带同步声音的视频。它还配有一个智能体(Murea-Mind),能把玩家输入的操作转化为连续的、被拍摄下来的故事。
这是一个工具,还是两个独立的系统?它由两个相互连接的组件构成,通过 HTTP API 通信。Murea-Fast 是一个 GPU 推理服务,负责生成实际的视频和音频;Murea-Mind 是一个网页应用和 Python 包,充当故事导演和世界状态管理器。
我要怎么把它跑起来?在 GPU 机器上启动 Murea-Fast:按照部署指南安装环境和 LTX-2.5 基础组件,并下载 murea-fast.safetensors 检查点。然后在任意机器上使用 uv 运行 Murea-Mind。
需要什么硬件和账号?Murea-Fast 需要配备 NVIDIA GPU 和 CUDA 的 Linux 系统(已在单块或四块 H20 上验证)。Murea-Mind 只需要 Python 3.12 和 uv,外加一个文本/图像模型的 API 密钥,例如 Gemini、兼容 OpenAI 的服务器或 DashScope。
部署前有什么要注意的吗?Murea-Fast 和 Murea-Mind 都没有内置身份验证,因此在把任何一个服务部署到共享网络之前,请先阅读 SECURITY.md。
片段生成有多快?使用 8 步蒸馏采样时,在四块 H20 GPU 上配合序列并行,一段 5 秒的片段大约需要 17 秒。
可以和朋友一起玩吗?可以。合作模式支持 2–4 名参与者(真人或 AI 队友)在同一个房间里游玩,每一轮都会被拍成一段共享片段。
智能体如何保证各片段之间的故事前后一致?Murea-Mind 会把每个角色、物品和场景都锁定到一张参考图像上,并配以固定的描述。它会规划每个故事节拍、进行检查、拍摄,然后在继续之前,对照实际生成的画面帧审查结果。
输出的视频是什么样的?片段以 1280x704 分辨率、24fps 渲染,并带有同步音频,包括角色的台词对白。
大小
237.8 MB
星标
3.4K
更新
提交Oct 20
时长
1个月
提交 · 90 天