InWorld README

README проекта с github.com/Murea-Labs/InWorld. Одним файлом markdown: README.md.

Murea-InWorld — From imagination to living worlds

Type what you do, and the world films it.

Open-source video generation with synchronized sound from text and reference images, and an agent that turns your actions into a continuing, filmed story.

Website Paper PDF GitHub code Hugging Face weights Code license

English | 中文

🎬 Demo · 🚀 Quick start · 🧭 Guides · 📚 Citation · ✉️ Contact

🎬 Demo

Text-driven storytelling with Murea-InWorld · 2 min 19 s · Download the original-quality video

https://github.com/user-attachments/assets/2617c0f3-ded2-4a8f-81bb-b77095c92b0e

[!NOTE] The Murea-Fast weights are available on Hugging Face. Download murea-fast.safetensors and follow the deployment guide to verify the checkpoint and set up the LTX-2.5 base components.

📄 Abstract

Adapted from our technical report. The report evaluates Murea-Pro and Murea-Fast; this repository provides Murea-Mind and the Murea-Fast inference service.

Interactive world models aim to simulate environments that evolve under user actions while retaining the consequences of past interactions. Extending such worlds to sight and sound requires persistent entities, changing scene states, and local audiovisual dynamics to remain coherent across turns. This paper presents Murea-InWorld, an interactive audiovisual world model that couples explicit world-state management with neural audiovisual generation. Its world agent, Murea-Mind, maintains characters, objects, locations, and narrative progress, interprets user actions in context, and compiles planned events into generation conditions. Observations of generated frames inform subsequent planning and selected state updates, closing the interaction loop.

Its audiovisual generator, Murea, realizes these events through Omni-Reference Generation, using visual references to anchor entity appearance and structured descriptions to specify actions and dialogue. Streaming generation uses modality-specific histories, with longer audio context for unfinished utterances. Degraded-history training prepares the model for imperfect rollout context, while sequential MeanFlow and distribution matching distillation enable few-step synthesis. Branching histories support alternative continuations, and co-op interaction brings multiple participants' actions into the same world.

We evaluate the generation components in complementary configurations. Murea-Pro achieves an Overall score of 81.13 on WBench Full-289, the highest among the compared systems. Murea-Fast achieves a Total score of 64.98 on OpenS2V-Eval and an aggregate score of 69.9 on OS2V-Pro, an extended evaluation of reference fidelity, attribute binding, rendering quality, and audio. Murea-Pro and Murea-Fast achieve state-of-the-art performance on their respective benchmarks, and both models surpass most evaluated commercial systems in their respective evaluations.

📣 News

✨ Highlights

🎮 What you can do

🖼️ Film a scene from references
Two characters and a place, one prompt, a clip with dialogue and sound. Murea-Fast
🔄 Continue a clip
Pick up from the last frames and keep the action going. Murea-Fast
🎭 Play a story
Type an action, watch it happen, choose what comes next. Murea-Mind
👥 Play with friends
A shared room, one clip per round, with optional AI teammates. Co-op mode

🧩 How it fits together

flowchart LR
    P([Player]) -->|action| A[Murea-Mind<br/>plans, checks, remembers]
    A <-->|text and images| L[(Language and image model APIs<br/>Gemini / OpenAI-compatible / DashScope)]
    A -->|prompt + reference images| F[Murea-Fast<br/>inference service on GPUs]
    F -->|video with sound| A
    A -->|clip + next choices| P

The two parts talk over a documented HTTP API, so the service can run on another machine or behind an SSH tunnel, and other clients can use it too.

🧭 Where to start

I want to… Start here
Play a story in the browser Murea-Mind quick start
Play with friends Co-op mode
Generate or continue a clip from the command line Murea-Fast quick start
Call the model from my own code API reference
Run the service on my GPUs Deployment
Know how fast it is and how much memory it needs Performance
Contribute CONTRIBUTING.md

⚙️ Components

Murea-Fast Murea-Mind
What it is Inference service for an 8-step distilled derivative of LTX-2.5, on a pinned SGLang Web app and Python package: a language-model director, deterministic checks, world memory and a critic
Runs on Linux and NVIDIA GPUs with CUDA. Verified on one H20 and on four H20s Any machine with Python 3.12; needs a language and image model API key and a Murea-Fast service
Interface HTTP API, command line Web UI, terminal, Python API

🚀 Quick start

1. Start Murea-Fast on a GPU machine. Install the environment and the LTX-2.5 base components as described in Deployment, download the murea-fast.safetensors checkpoint, and set the four paths in inference/configs/single-gpu.json. Set checkpoint to the absolute path of the downloaded file. Then:

cd inference
bash scripts/murea-fast prepare --config configs/single-gpu.json
bash scripts/murea-fast start --config configs/single-gpu.json
curl --fail http://127.0.0.1:30260/health

2. Run Murea-Mind on any machine. It needs uv and a key for text and images (Gemini, any OpenAI-compatible server, or DashScope):

git clone https://github.com/Murea-Labs/InWorld.git Murea-InWorld && cd Murea-InWorld/agent
cp .env.example .env          # set GEMINI_API_KEY and MUREA_URL=http://<gpu-host>:30260
./run.sh                      # then open http://127.0.0.1:8080

See the Murea-Mind README for other providers, Docker and SSH tunnels.

🗂️ Repository layout

Murea-InWorld/
├── LICENSE
├── CONTRIBUTING.md
├── SECURITY.md
├── inference/            Murea-Fast inference service
│   ├── configs/          service configurations and an example request
│   ├── docs/             deployment, API, performance, provenance
│   ├── scripts/          CLI wrapper, connector merge, acceptance run
│   ├── src/murea_fast/   launcher and client
│   ├── tests/
│   └── vendor/sglang/    pinned SGLang with the model implementation
└── agent/                Murea-Mind
    ├── murea_agent/      the agent (Python package)
    ├── app/              FastAPI server
    ├── frontend/         React UI (prebuilt in frontend/dist)
    ├── docs/             co-op mode design notes
    └── tests/

🔒 Security

Neither the inference service nor the agent has authentication. Read SECURITY.md before deploying them on a shared network.

🤝 Contributing

See CONTRIBUTING.md for the checks each part needs before a pull request.

📜 License

The code is released under the Apache License 2.0. The vendored SGLang in inference/vendor/ keeps its own copyright and license notices, including the files it carries under MIT and BSD-3-Clause. The Murea-Fast model weights are a derivative of LTX-2.5 and are governed by the LTX-2.x Community License Agreement; see Provenance for details. The agent calls third-party language and image model APIs, which have their own terms.

📚 Citation

@misc{murea_inworld2026,
  title        = {InWorld: From Imagination to Living Worlds with Agents and Fast Video Generation},
  author       = {{Murea Labs}},
  year         = {2026},
  url          = {https://murea.ai/page/paper/InWorld-technical-report.pdf}
}

✉️ Contact

Email: [email protected]

InWorld (Murea-Labs/InWorld)

Ищем в базе…

3.4K ·GitHub
Что это такое?Murea-InWorld — это система с открытым исходным кодом, которая генерирует видео с синхронизированным звуком по текстовому описанию и опорным изображениям. Вместе с ней работает агент Murea-Mind, который превращает введённые игроком действия в продолжающуюся историю с видеосъёмкой.
Это один инструмент или две отдельные системы?Это два связанных компонента, которые обмениваются данными через HTTP API: Murea-Fast — сервис генерации на GPU, который создаёт само видео и звук, и Murea-Mind — веб-приложение и Python-пакет, который выступает режиссёром истории и управляет состоянием мира.
Как это запустить?Запустите Murea-Fast на машине с GPU: установите окружение и базовые компоненты LTX-2.5 по руководству по развёртыванию и скачайте контрольную точку murea-fast.safetensors. Затем запустите Murea-Mind на любой машине с помощью uv.
Какое железо и какие аккаунты нужны?Murea-Fast требует Linux с GPU NVIDIA и CUDA (проверено на одной или четырёх H20). Murea-Mind нужны только Python 3.12 и uv, а также API-ключ для текстовой или графической модели, например Gemini, сервера с совместимым OpenAI API или DashScope.
Есть ли подводные камни перед развёртыванием?Ни Murea-Fast, ни Murea-Mind не имеют встроенной аутентификации, поэтому перед тем как выставлять любой из этих сервисов в общую сеть, прочитайте SECURITY.md.
Как быстро генерируется клип?При сэмплировании в 8 шагов (distilled) клип длительностью 5 секунд создаётся примерно за 17 секунд на четырёх H20 с использованием sequence parallelism.
Можно ли играть с друзьями?Да, кооперативный режим поддерживает от 2 до 4 участников (людей или ИИ-напарников) в общей комнате, где каждый раунд снимается как один общий клип.
Как агент сохраняет целостность истории между клипами?Murea-Mind закрепляет каждого персонажа, предмет и место за опорным изображением с фиксированным описанием, планирует каждый story beat, проверяет его, снимает и перед продолжением сверяет результат с реально сгенерированными кадрами.
Как выглядит итоговое видео?Клипы выводятся в разрешении 1280x704 и с частотой 24 кадра в секунду, со синхронизированным звуком, включая реплики персонажей.
Какие есть похожие проекты?Evoke, Pika-H3-Flex, EdenLive, awesome-minimax-h3-integration, CogVideo, LTX-Video, vllm-omni, WanGP
Автор
Murea-Labs
Размер
237.8 MB
Звёзды
3.4K
Обновлён
КоммитOct 20
Возраст
1 мес
Коммиты · 90 дней