utopia README

The project's README from github.com/deeplethe/utopia. As one markdown file: README.md.

Utopia

Utopia

Philosophy · Quick start · Features · Roadmap

Stars License Rust

Official site Container Discussions Built by DeepLethe 中文

The enterprise world model built by DeepLethe. It is the first open substrate for knowledge engineering that learns passively and governs itself. Where a knowledge graph or a vector store works to hold present knowledge, Utopia puts time awareness and ontology in the base layer: the knowledge system evolves as material arrives, and conflict detection, reasoning and decision making all run against that ontology. It deploys offline, so a company can stand up a knowledge foundation, a decision core its agents can trust, and a compliance audit trail on hardware it controls.

Please note: we would rather this project were not framed as an open-source take on Palantir. It is a different route to enterprise intelligence, built bottom up from knowledge governance to trustworthy decisions and simulation.

https://github.com/user-attachments/assets/aa226443-75de-437e-bd80-88e592ed8457

Philosophy

We gave it a somewhat romantic name, Utopia. Ptolemy's geocentric model was taken for truth for a very long time, then falsified step by step by Copernicus, Kepler, Galileo and Newton. Looking back, what we keep is not only that heliocentrism turned out to be right; it is how that history unfolded.

Where existing vector stores and knowledge graphs work to get present knowledge right, one of Utopia's founding aims is to record the whole course of changing understanding. Engineered, that becomes a bitemporal knowledge graph. When a decision is reviewed later, the system can produce the full course it took and the grounds it rested on. To make this hold up in practice we have iterated at length against public corpora spanning enterprise records, education, finance, law and research. Temporality is only one facet; for how knowledge is taken in, how the future is reasoned about, and how logic bounds action, see utopia.bi/philosophy.

Features

One Rust binary and one Postgres. Full-text search is embedded in the binary, vectors go in pgvector, and the job queue is a table: nothing else to run.
A complete application A system console, a graph browser and an ontology workbench in one web UI. A product, not a library: install it and it works.
Knowledge ingest Upload PDF, DOCX, PPTX, XLSX, XLS, ODS, CSV, TSV, Markdown, HTML or plain text, with legacy encodings detected on the way in. Web pages, RSS, GitHub, Jira, Notion, WebDAV and S3-compatible buckets sync on a schedule; everything else comes in through the API.
Search and chat Full-text on Tantivy, vectors on pgvector, fused with RRF. Answers stream with inline citations that open the passage they came from. Any OpenAI-compatible endpoint works (DeepSeek, Qwen, GLM, Ollama, vLLM), so the whole system can run air-gapped.
Agent harness and agentic RAG The whole system can be driven through conversation. The built-in agent searches documents, walks the graph (an entity's facts as of any date, or what changed in a period) and queries a mounted database. The same read-only tools are exposed over MCP.
Ontology and cold start A new knowledge base has no vocabulary of its own; it starts from the packs you pick at creation. Five ship inside the binary: schema.org, W3C Org, PROV-O, FOAF and IOF Core (ask for your industry). Terms outside the packs are counted as they appear; confirm the common ones and they join the ontology.
Bitemporal graph Extraction turns documents into entities and facts, following an ontology you can edit. Every fact carries when it held and where it came from. Correcting a fact closes the old version and links the new one to it rather than overwriting, so the graph keeps two timelines: when something was true in the world, and when the system came to believe it.
Entity resolution and review Duplicates are resolved in three stages: exact name or alias, embedding similarity, then a model's call on the doubtful pairs. Every merge can be undone. Uncertain cases go to a review queue: low-confidence extractions, suspected duplicates and cardinality conflicts.
Reasoning and derivation Ontology axioms compile into rules: transitivity, symmetry, inverses and relation hierarchy derive new facts by forward chaining. Derivation is off by default, since a wrong axiom derives wrong facts. A derived fact is marked as such on the graph, carries validity and confidence like any other, and shows what it was derived from. When it contradicts an asserted fact, the asserted one stands.
Conflict detection Three kinds of conflict, three sets of choices. A new fact that clashes with an older one: close the old, keep both, or reject the new. Data that breaks an axiom (self-loop, asymmetry, transitive cycle, cardinality): retract the fact, relax the axiom, or accept both. The ontology itself is checked first, because violations of a self-contradictory ontology are noise.
Ontology-driven querying Mount a database on a base (Postgres, MySQL and the engines that speak its protocol, Trino for Iceberg / Delta Lake / Hive, Databricks, Snowflake) and chat can query it alongside the documents. The agent proposes how its tables map onto the ontology, and you confirm. The method behind it, Ontology2SQL, is state of the art on BIRD Mini-Dev for SQLite and PostgreSQL (submission).
Multi-user and permissions Each knowledge base has its own members and roles: owner, admin, editor and viewer. Open bases are readable by everyone in the deployment, restricted ones only by invitation. The first account registered becomes the system administrator.
Decision ledger Confirming or rejecting a fact, merging or reverting an entity, rebuilding the graph: each leaves a record of who, when, and what the object looked like at the time. The ledger is append-only, and a record outlives its object, even the base it belonged to.
Decision intelligence (in development) Record a decision, replay both what was understood and the course it took, and reason over overlaid scenarios.

Quick start

Requirements: Docker (local development also needs Rust 1.85+, Node 20+, pnpm).

Start from the prebuilt image:

git clone https://github.com/deeplethe/utopia.git
cd utopia
docker compose --profile app up -d

Open http://localhost:1516 and register. The first account automatically becomes the administrator, and a public knowledge base readable by everyone is created at the same time. Before extracting business documents, configure the model endpoints (chat and embedding) under Administration → Models.

The database password (UTOPIA_DB_PASSWORD in .env, default utopia) is applied when the data volume is first initialised. To change it on a running deployment, change it in the database as well — docker compose exec db psql -U utopia -c "ALTER USER utopia PASSWORD '<new password>'" — or start over with docker compose --profile app down -v, which deletes all data.

Or build from source:

docker compose -f docker-compose.yml -f docker-compose.build.yml --profile app up -d --build

Local development

# 1. Postgres with pgvector
docker compose up -d db

# 2. Backend on :1516, runs migrations on startup
cargo run -p utopia-server

# 3. Frontend on :5173, proxying /api to the backend
cd web && pnpm install && pnpm dev

Roadmap

Status

Utopia is still at v0.1. The database schema evolves between versions and migrations only roll forward, with no rollback. Pin a specific version with UTOPIA_IMAGE in production, and back up the database along with the data directory before upgrading.

Please read SECURITY.md before exposing it to the public internet.

Star History

Star History

Community

License

Apache-2.0

utopia (deeplethe/utopia)

full-text-search
Searching Database…

8.4K ·GitHub
Utopia
What is this?Utopia is an open-source enterprise world model from DeepLethe: a self-hosted knowledge system that turns documents into a bitemporal knowledge graph you can search, chat with and reason over, running entirely on hardware you control.
Is it a library I wire into my code, or a product I deploy?A complete product, not a library: one Rust binary and one Postgres, with a web UI bundling a system console, a graph browser and an ontology workbench. Install it and it works.
How do I get it running?Run the prebuilt Docker image with docker compose, open the web UI and register — the first account becomes the administrator and a public knowledge base is created. Before ingesting business documents, set your chat and embedding endpoints under Administration → Models.
What do I need besides Docker?Just Docker for deployment; local development also needs Rust 1.85+, Node 20+ and pnpm. For chat and embeddings you point it at any OpenAI-compatible endpoint — DeepSeek, Qwen, GLM, Ollama and vLLM all work — so no single AI vendor is required.
Is it production-ready today?Not fully — it is at v0.1: database migrations only roll forward with no rollback, so pin the image version in production and back up before upgrading, and read SECURITY.md before exposing it to the public internet.
Can it run fully offline, on my own hardware?Yes. It deploys offline and works with any OpenAI-compatible endpoint, so the whole system, LLM included, can run air-gapped on hardware you control — search, vectors and the job queue all live in that one binary plus Postgres.
What can I feed into it?Upload PDF, DOCX, PPTX, XLSX, CSV, Markdown, HTML or plain text, with legacy encodings detected on the way in. Web pages, RSS, GitHub, Jira, Notion, WebDAV and S3 buckets sync on a schedule, and everything else comes in through the API.
Can AI agents work with it, or is it just a web UI?The whole system can be driven through conversation: the built-in agent searches documents, walks the graph — including an entity's facts as of any date — and queries mounted databases. The same read-only tools are exposed over MCP, so external agents can plug in too.
Can it query my databases too, not just documents?Yes — mount Postgres, MySQL, Trino (for Iceberg, Delta Lake and Hive), Databricks or Snowflake, and chat can query them alongside documents. The agent proposes how tables map onto the ontology and you confirm; the method, Ontology2SQL, is state of the art on BIRD Mini-Dev for SQLite and PostgreSQL.
What happens when new facts contradict old ones?You choose per conflict: close the old fact, keep both, or reject the new. Data that breaks an ontology axiom can be retracted, the axiom relaxed, or both accepted — and the ontology itself is checked first, since violations of a self-contradictory ontology are just noise.
What are similar projects?m_flow, MemoryBear, LightRAG, semantica, WeKnora, zilan, QAnything, CodeMason
Which topics is it in?world-model, pgvector, graphrag, compliance, audit-log, full-text-search, vector-search, knowledge-graph, semantic-search, enterprise, agent-memory, openai-compatible
Creator
deeplethe
Size
12.5 MB
Stars
8.4K
Updated
CommitOct 11
Age
2mo
Commits · 90 Days