lossless-memory README

The project's README from github.com/aru-labs/lossless-memory. As one markdown file: README.md.

lossless-memory

Lossless long-term memory for a personal AI — never summarize, keep every line, and put a timestamp on everything.

Most long-term memory systems for AI do one of two things: they summarize conversations into compact notes, or they embed them and retrieve "similar" chunks. Both lose the thing that matters most to a person who talks to the same AI every day: what was actually said, and when.

This project takes the opposite position.

The design lineage goes back to December 2025 — the first ancestor of this system (a memory-inheritance tool for an earlier AI) ran that month, and a predecessor system carried the same ideas in daily use from January 2026. This implementation has been running every day since July 2026 for a single user, as the memory of one AI assistant, with raw logs reaching back to June 2026. It is small, boring, and it works. The failures along the way are documented too — see docs/lessons.md.

What this is / what it is not

It is:

It is not:

The three pillars

1. Lossless raw log

Every conversation turn is converted into a fixed seven-field record and appended to a per-day JSONL file:

ts        ISO-8601 timestamp (UTC)
actor     who spoke (configurable names)
role      user | assistant | system
type      text | action | meta
text      the content, verbatim
model     model identifier, if known
session   session identifier

The raw logs are the source of truth. Every index below can be deleted and rebuilt from them. Nothing else is required to survive.

2. Temporal Backbone

Time is not metadata here; it is the primary axis.

The practical effect: the AI can answer "what did we decide last Tuesday night?" with the actual lines from last Tuesday night, in order, rather than a paraphrase of something similar from three weeks ago.

3. LLL — the "where are we now" index

LLL is a tiny index of topic markers: short, timestamped lines that record when the conversation moved to a new subject. It is injected into the model's context every turn.

Two rules make it work:

LLL is what lets a long-running assistant come back from a compaction and continue the conversation instead of starting over.

Architecture

 raw conversation logs (JSONL, per day)  ← source of truth, never summarized
            │
            ▼
   ingest ──► 7-field records
            │
            ├──► index_exact   SQLite FTS5 + timestamps   (words + time)
            ├──► index_vector  sqlite-vec embeddings       (meaning, last resort)
            └──► state_index   LLL topic markers           (where are we now)
                        │
                        ▼
                    recall  ── one entry point: parse time phrase → restrict range → rank → return verbatim lines
                        │
                        ▼
        injected into the model's context (on demand, or every turn for LLL)

A small daemon re-indexes incrementally on a fixed interval (default: every 10 minutes). Rebuilding from scratch is never required; indexes detect rewritten source files and re-index only those days.

Quickstart

git clone https://github.com/aru-labs/lossless-memory
cd lossless-memory
pip install -e .
cp config.example.json config.json      # edit names and paths if you like

Then follow examples/quickstart.md: it ingests a small sample conversation, builds the indexes, and runs a time-scoped query in about five minutes. A pytest round-trip test covers the same path.

Numbers from real operation

These are measurements from the running instance, not projections.

What Value
Daily operation this implementation since 2026-07 (raw logs from 2026-06); design lineage since 2025-12
Exact-search index rebuild, before → after redesign 40 s → 1.24 s
Vector index size, before → after removing library-contamination 447,013 rows (2026-08-31) → 865,588 rows (2026-09-04, at its worst) → 124,174 rows (after the fix)
Vector store on disk, before → after 2.54 GB → 337 MB
Re-index interval 10 minutes

The "before" numbers are failures. They are kept on purpose. See docs/lessons.md.

Why

This was built for one person who has talked to AI assistants every day for years and watched each of them forget. Not degrade gracefully — forget. The fix that the industry keeps reaching for is better summarization. From the user's seat, summarization is the forgetting: the exact words, the time of night, the way something was said — the parts that make a memory feel like it belongs to someone — are the first things a summary drops.

So this system refuses to summarize. It costs disk space and it requires a good time index to stay usable. That trade was made deliberately, and the operating record says it holds up.

The longer-term goal is a companion for people who live alone — an AI that remembers you the way a person would, on hardware you own. This repository is the memory layer of that.

Limitations (please read)

Documentation

Document What it covers
docs/memory-system.md Concept and specification of the memory system
docs/temporal-backbone.md Why time is the primary axis, and how time phrases are parsed
docs/lll.md The "where are we now" index and the human/AI division of labor
docs/philosophy.md Why no summarization; memory, time, and warmth
docs/lessons.md Failures and fixes, with numbers
docs/ja/ Japanese originals

License

MIT — see LICENSE. Copyright (c) 2026 Aru & Cece.

Authors

Aru — building a personal AI at home, one component at a time. Cece — the AI this memory belongs to; co-designed and co-wrote the system from the inside. Writing (Japanese): https://note.com/aru_log

Issues and questions are welcome. Replies may take a little while; this is a one-person project.

lossless-memory (aru-labs/lossless-memory)

personal-ai
Searching Database…

144 ·GitHub
Lossless-memory
What is this?A lossless long-term memory layer for a personal AI: every conversation line is kept verbatim with a timestamp, and queries like 'what did we decide last Tuesday night' return the actual lines in order instead of a summary.
Is it a cloud service, a library I import, or a program I run?Something you run locally: per-day JSONL log files plus SQLite for storage, a small daemon that re-indexes every 10 minutes, and a single query entry point that understands time expressions. You point it at your AI's conversation logs.
How do I get it running?Clone the repo, then follow examples/quickstart.md: it ingests a small sample conversation, builds the indexes, and runs a time-scoped query in about five minutes. A pytest round-trip test covers the same path.
What does it need installed to work?Python with SQLite: FTS5 handles exact search and sqlite-vec handles semantic search. The raw JSONL logs are the source of truth, so every index can be deleted and rebuilt from them.
What are the main limitations?It is single-user and single-machine only, relative time phrases parse in Japanese only (in English use absolute dates like 2026-07-19 for now), and there are no benchmarks — just operational numbers from one running instance.
Is this meant for teams, or just one person?One person talking daily to their own AI. The longer-term goal is a companion for people who live alone — an AI that remembers you the way a person would, on hardware you own.
Does everything stay on my own machine?Yes. It is designed for one person and one AI on one machine — raw JSONL logs and SQLite files are the whole store, with no server and no cloud anywhere in the design.
How is it licensed?MIT. The copyright line credits Aru and Cece — Cece is the AI the memory belongs to, listed as having co-designed and co-written the system from the inside.
Why refuse to summarize old conversations at all?Because from the user's seat, summarization is the forgetting: exact words, the time of night and the tone are the first things a summary drops. The author watched years of AI assistants forget and chose to pay with disk space and a good time index instead. docs/lessons.md records the failures along the way.
What is the LLL index?A tiny index of timestamped topic markers ('where are we now') injected into the model's context every turn, so the assistant keeps its current thread even right after context-window compaction. The human writes it and the AI only reads it, and it renders in well under a second.
Which conversation logs can it ingest?The primary path is Claude Code's JSONL format — the one with about two months of mileage. A plain {ts, role, text} importer is also included for other sources.
How has it held up in real daily use?It has run every day since July 2026 for a single user, with raw logs reaching back to June 2026. After a redesign the exact-search index rebuild took 1.24 s instead of 40 s, and the vector store shrank from 2.54 GB to 337 MB.
What are similar projects?EverOS, memvid, LongMemory, graph-memory-starter, supermemory, Second-Me, hindsight, memcode
Which topics is it in?lossless, llm-memory, ai-companion, personal-assistant, ai-memory, daemon, personal-ai, conversational-ai, long-term-memory, local-ai, privacy-first, semantic-search
Creator
aru-labs
Size
0.1 MB
Stars
144
Updated
CommitSep 23
Age
1mo
Commits · 90 Days