Air-Lingjing README

The project's README from github.com/seanlxh/Air-Lingjing. As one markdown file: README.md.

🌐 LingJing Embodied Intelligence

An open simulation stack for LLM-driven embodied agents in large 3D environments.

English 简体中文

LingJing combines a FastAPI/LangGraph backend, runnable embodied-agent cases, and a Cesium-powered Unreal Engine 5.5 plugin in one repository. The backend handles task orchestration, LLM/VLM calls, Gym-style environment loops, engine communication, and data recording. The Unreal plugin provides the 3D world integration, heterogeneous equipment actors, cameras, UI, image capture, and telemetry.

Large Unreal assets are distributed separately through Hugging Face. This repository contains source code, configuration templates, tests, and compact example data only.

🗺️ Platform Overview

LingJing platform overview

LingJing connects heterogeneous simulators, geospatial assets, model services, and embodied agents through a shared urban physics environment. Each simulation step closes the perceive-think-act-communicate loop and records outputs for evaluation and visualization. Open the high-resolution PDF.

🧩 Runtime Framework

LingJing agent and simulation framework

The runtime separates agent policy, shared infrastructure, adjudication, command routing, evaluation, and simulation engines. This keeps policies independent from engine-specific protocols while preserving per-agent observations, message history, acknowledgements, and replayable trajectories. Open the high-resolution PDF.

✨ Highlights

Repository Layout

LingJing-Embodied-Intelligence/
├── backend/
│   ├── app/                 # FastAPI application and runtime modules
│   ├── examples/            # Runnable cases and UE reference client
│   ├── tests/               # API, environment, and realtime tests
│   ├── scripts/             # Deployment and maintenance helpers
│   └── .env.example         # Safe configuration template
├── unreal/
│   └── EmbodyIntelligence/  # Unreal Engine 5.5 runtime plugin
├── docs/                    # README figures and high-resolution PDFs
├── ASSETS.md                # Hugging Face asset instructions
└── README.zh-CN.md          # Chinese documentation

🚀 Backend Quick Start

Requirements: Python 3.10 or newer.

cd backend
python -m venv .venv
source .venv/bin/activate
pip install -r requirements-dev.txt
cp .env.example .env

Set your own provider values in .env:

AI_BASE_URL=https://api.example.com/v1
AI_API_KEY=replace-with-your-api-key
AI_CHAT_MODEL=replace-with-your-chat-model
AI_ANALYSIS_MODEL=replace-with-your-vision-model
AI_API_STYLE=openai
INTERNAL_AI_TOKEN=replace-with-a-long-random-token

Start the service:

uvicorn app.main:app --reload --host 0.0.0.0 --port 9909

Verify it:

curl http://127.0.0.1:9909/health

Run the end-to-end mock demo without Unreal Engine or an API key:

python examples/scenario_demo.py

🧠 Runnable Cases

Directory Scenario
examples/full_case/ Multi-drone urban delivery and centralized baseline
examples/fire_rescue/ UAV reconnaissance with UGV firefighting
examples/singledrone_fire/ Closed-loop visual fire search
examples/singledog/ Robot-dog visual navigation
examples/deliverytask/ Heterogeneous delivery and route inspection
examples/bridge/ Bridge inspection from top-down and front views
examples/multicars/ Multi-vehicle dispatch and route-cost analysis
examples/uavdog/ UAV-assisted robot-dog navigation
examples/multiagentstasks/ Mixed equipment scenario dispatch
examples/ue_client/ Python reference implementation of an engine client

See backend/examples/README.md for commands and runtime requirements.

🏙️ Unreal Engine Setup

Requirements:

Install the plugin at:

<HostProject>/Plugins/EmbodyIntelligence/

Download the asset repository:

Coming soon / 即将上传

Install the downloaded asset package in its two mount locations:

<HostProject>/Plugins/EmbodyIntelligence/Content/  # PluginContent
<HostProject>/Content/                            # HostContent

Program assets use the /EmbodyIntelligence/... plugin mount, while scene assets use the /Game/ArtRes/... host-project mount. See ASSETS.md and unreal/EmbodyIntelligence/README.md.

The plugin defaults to:

ws://127.0.0.1:9909/ws/LJ-ENGINE/image

For a remote backend, update WEBSOCKET_URL in Source/ue5project/Core/DataManager.cpp before building.

🧪 Validation

cd backend
python -m pytest -q
python -m ruff check app examples tests

The Unreal plugin must be compiled in Unreal Engine 5.5.4 with Cesium enabled. Generated Binaries/, Intermediate/, logs, databases, uploads, and result folders are intentionally excluded from Git.

Security

GitHub Metadata

Suggested description:

Open embodied-intelligence simulation stack with a FastAPI/LangGraph multi-agent backend, reproducible urban task cases, and a Cesium-powered Unreal Engine 5.5 plugin.

Suggested topics: embodied-ai, multi-agent, unreal-engine, ue5, cesium, simulation, robotics, langgraph, fastapi, digital-twin.

License

No license is selected in this release package. Add a LICENSE file only after confirming the redistribution terms for the source code and all Unreal/Hugging Face assets.

Air-Lingjing (seanlxh/Air-Lingjing)

langgraph
Searching Database…

194 ·GitHub
Air-Lingjing
What is this?LingJing is an open simulation stack for LLM-driven embodied agents in large 3D environments: a FastAPI/LangGraph backend plus a Cesium-powered Unreal Engine 5.5 plugin, with runnable example cases.
Is it one app, or what are the pieces?Two parts in one repo: a backend that handles task orchestration, LLM/VLM calls, Gym-style environment loops, engine communication and data recording, and an Unreal plugin that supplies the 3D world, actors, cameras and telemetry. Heavy assets come separately via Hugging Face.
How do I get it running?The backend needs Python 3.10+: set your provider values in .env, start the service and verify it in Swagger UI. For the full 3D stack you also install the plugin into Unreal Engine 5.5.4 with Cesium and the matching asset package from Hugging Face.
Do I need an API key or Unreal Engine to try it?Neither, to start: an end-to-end mock demo runs without Unreal Engine or an API key. For real models, providers are OpenAI-compatible and Anthropic-compatible and are configured in .env.
What is the catch with the Unreal side?The plugin only compiles in Unreal Engine 5.5.4 with Cesium enabled. Assets mount in two places (program under /EmbodyIntelligence/..., scenes under /Game/ArtRes/...), and for a remote backend you edit WEBSOCKETURL in DataManager.cpp before building.
Can I reuse this in my own project?Mind that no license is selected in this release package, so redistribution terms are not settled. The README says to add a LICENSE only after confirming terms for the source code and all Unreal/Hugging Face assets.
What scenarios can I run out of the box?Ten runnable cases under backend/examples/: multi-drone urban delivery, UAV reconnaissance with UGV firefighting, closed-loop visual fire search, robot-dog visual navigation, heterogeneous delivery and route inspection, bridge inspection, multi-vehicle dispatch, UAV-assisted robot-dog navigation, mixed equipment dispatch, and a Python reference engine client.
What kinds of agents live in the simulated world?A Cesium-based urban environment with drones, vehicles, robot dogs, ships and task actors. Each simulation step closes the perceive-think-act-communicate loop and records outputs for evaluation and visualization.
How do multiple agents coordinate?Through centralized, star and broadcast coordination patterns. The runtime keeps per-agent observations, message history and acknowledgements, and produces replayable trajectories for evaluation.
How does the backend talk to Unreal Engine?Over HTTP, WebSocket and UDP. Engine sessions are exposed by the backend service so you can check connectivity before launching the full Unreal client.
What are similar projects?FluxVLA, RoboICL, ZDTaichu5.0-9B, project_superdex, LoongForge, carla, AirSim, Embodied-AI-Guide
Which topics is it in?unreal-engine, udp, telemetry, geospatial, langgraph, embodied-ai, vlm, robotics, simulation, fastapi, 3d, websocket
Creator
seanlxh
Size
10.5 MB
Stars
194
Updated
CommitAug 14
Age
2mo
Commits · 90 Days