backpass
Gradient descent for your agent memory.
Your AGENTS.md is a set of weights. Every agent session is a forward pass. The
transcript that session leaves on disk is the loss signal - and today nothing reads it.
The loop only closes when a human happens to remember a failure and edits the file by hand.
backpass closes it. It finds the agent sessions that actually ran in your repo, reads
what happened in them, and proposes evidence-backed edits to your memory surface - the
memory file and project skills - under a token budget, gated by you.
- Local-first - Reads the transcript stores of seven agent harnesses directly from disk.
No API, no upload; transcripts never leave your machine except into an agent you already
authenticated, and obvious secrets are redacted before they do.
- Evidence-gated - Every proposed edit carries verbatim quotes from real sessions,
and every
add, rewrite, or remove edit needs evidence from at least two distinct
sessions. Small, noisy, bounded steps - not a rewrite.
- Human in the loop - Analysis never writes.
backpass apply is the only writing
command, and it shows each edit with its evidence for you to accept or reject.
AGENTS.md / CLAUDE.md + skills (the weights)
→ agent session (forward pass)
→ transcript on disk (loss signal)
→ backpass: collect samples, distill, calculate loss, aggregate gradients
→ backpass: gradient descent (diffs + skill extractions)
→ you accept or reject (the human gate)
→ back to the weights
One run is one bounded gradient step.
Quick Start
npm install -g backpass
# or run it without installing
npx backpass
Requires Node >= 22.5 and acpx on your PATH.
backpass has no API keys of its own. Every model call goes through acpx to a harness
you have already authenticated.
cd your-repo
backpass init # write .backpassrc.json, exclude .backpass/ via .git/info/exclude
backpass # collect samples → calculate loss → aggregate gradients → gradient descent (never writes)
backpass apply # review each edit, accept or reject, then write
User-level memory
A run is one scope. The default is the checkout you are in. backpass --scope user
trains the always-loaded user file and user-level skills from Claude Code and Codex
sessions across projects, and writes only those files. A project-scoped run never
writes a user-level file.
Canonical user memory is the first existing file in this order: ~/.agents/AGENTS.md,
$CLAUDE_CONFIG_DIR/CLAUDE.md (default ~/.claude/CLAUDE.md), and
$CODEX_HOME/AGENTS.md (default ~/.codex/AGENTS.md). User-level skill extractions
default to ~/.agents/skills, with a warning if Claude's active skills path is a
real directory rather than the usual symlink. See Configuration for
using an existing harness-loaded directory instead.
In user scope every add, rewrite, or remove edit also clears minGapProjects
(default 1): the distinct projects behind its own quotes, counted from the gap
clusters it cites and from the session-to-project map behind the instruction evidence
rows. extract and move edits remain exempt.
State lives in $XDG_CONFIG_HOME/backpass/user/ (default
~/.config/backpass/user/) with mode 0700, isolated from every project's
.backpass/. User-scope evidence, ledgers, proposals, and apply surfaces stay in
that one directory.
Harness load paths, verified for v1:
- Claude Code loads
CLAUDE.md from CLAUDE_CONFIG_DIR (default ~/.claude)
and inlines @ imports, including ~/ and absolute paths. A CLAUDE.md containing
only an import that resolves to the canonical user memory is a valid pointer, such
as @~/.agents/AGENTS.md with the default paths.
- Codex loads
AGENTS.md from CODEX_HOME (default ~/.codex). It follows the
AGENTS.md convention; @ import is not assumed.
A target that resolves into a read-only store (nix, home-manager) is refused by name
rather than written. The whole path is resolved, so the link may be the file itself
(<path> is a symlink to <real>, which is not writable; edit the source that generates it) or a directory on the way to it (<path> resolves to <real>, which is not writable; ...). The test is whether the directory holding the resolved location can be
written; both messages name that resolved location, so you know which source to edit.
backpass init --scope user
backpass --scope user
backpass apply --scope user
One file instead of the whole surface
--target narrows a run to one configured memory file or one skill, named exactly: a
memoryFiles entry, or a skill's name:. Nothing else resolves - not a basename, a
directory, a path to a SKILL.md, or an existing file the config does not name - and an
unknown name fails, listing the valid ones, instead of falling back to the whole surface. A
configured file that contains only an @ import is rejected rather than rewritten or silently
mapped to its import; the error names the imported memory file, which must itself be configured
to be targeted. A correctly named skill whose file backpass cannot write - one resolving into a
location nothing may write, or in project scope one resolving outside the repository - is refused
by name too: backpass loads and bills that skill, but a targeted run against it could only end in
a refused write.
backpass --target AGENTS.md # this memory file; existing skills are read-only
backpass --target db # this skill only; the memory file and other skills are read-only
backpass --scope user --target db # the same, against the user-level surface
A memory-file target may still extract a new skill (that is how the file shrinks). A
skill target writes only that SKILL.md, and its budget is still the whole always-loaded
surface: the memory file plus every description line, moved by the description-line delta
of the edit. Analysis, evidence, and state are those of the whole surface; only staging and
the proposal gate narrow. --target applies to the default run, analyze, propose, and
apply; on apply it is optional, but when supplied it must match the saved proposal's target.
It cannot be combined with
--memory-file, and a targeted run never bootstraps a missing memory file.
How It Works
1. Collect samples - which sessions belong to this repo
backpass reads the local transcript stores of seven harnesses directly. No API, no upload.
| Harness |
Store |
Repo tie |
| claude |
~/.claude/projects/<munged-cwd>/<uuid>.jsonl |
per-line cwd |
| codex |
~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl |
cwd + recorded git.repository_url |
| pi |
standalone and BB-managed Pi JSONL stores |
session-header cwd |
| opencode |
~/.local/share/opencode/opencode.db (sqlite) |
session.directory |
| grok |
~/.grok/sessions/<encoded-cwd>/<uuid>/ |
summary.json cwd + git_remotes |
| cursor CLI |
~/.cursor/chats/<md5(cwd)>/<uuid>/ |
meta.json cwd |
| hermes |
~/.hermes/state.db (sqlite) |
session cwd, with CLI prompt / ACP config fallbacks |
Claude collection covers $CLAUDE_CONFIG_DIR/projects alongside the default store, so a
relocated config dir does not hide its sessions. The variable is read from backpass's own
environment: if you reach that profile through an alias that only prefixes claude, set it
for the backpass run too (CLAUDE_CONFIG_DIR=~/.claude-work backpass, or export it).
Pi collection covers standalone sessions under ~/.pi/agent/sessions/ and BB-managed Pi
sessions under ~/.bb/pi-bridge-sessions/. It also honors PI_CODING_AGENT_DIR,
PI_CODING_AGENT_SESSION_DIR, BB_DATA_DIR, and BB_PI_BRIDGE_SESSION_DIR when they are
set in backpass's environment. When roots overlap, backpass scans every applicable layout
and reads each JSONL file once.
Hermes collection includes CLI and ACP sessions only. Gateway, cron, and WhatsApp sessions
are excluded because their recorded cwd belongs to the shared gateway process, not a project.
Association runs in four tiers:
- Tier 1 - deterministic. The session's cwd is (or sits inside) one of this repo's
worktrees.
- Tier 1.5 - deterministic, sibling clone. The cwd is a live local clone (or one of
its worktrees) that shares a git remote with this repo.
git worktree list only sees
worktrees from one clone, and Claude records no remote, so without this tier a sibling
clone's interactive history is invisible. Backpass searches the parent
of each of this repo's worktrees, plus any discovery.cloneRoots you configure, and
only reads git identity there. Each clone root may be a checkout or a directory whose
immediate children are checkouts; relative paths resolve from the repo root. --strict
keeps this tier.
- Tier 2 - deterministic, survives deletion. A git remote recorded in the transcript
matches one of the repo's remotes. This is how codex and grok stay attributable long
after the worktree is gone.
- Tier 3 - best-effort. A dead path whose last segment is the repo's directory name,
or one matching a glob you configured. Labelled as such, and excluded by
--strict.
Collection is incremental. Codex alone can hold 10,000+ rollouts, so verdicts are cached in
.backpass/scan-cache.json by path, mtime and size - re-scans cost only the new files.
A harness whose store is missing or has drifted into an unrecognised shape produces a
warning and is skipped; the run continues. backpass's own loss and gradient-descent calls land
in these same stores under the repo's cwd; every prompt it sends is tagged, and tagged
sessions are excluded from the corpus (the SELF column in backpass scan).
Every remaining session is labelled interactive or non-interactive (src/interaction.js).
Codex codex exec / originator: codex_exec, Claude SDK, GitHub, action, and CI
entrypoints, OpenCode child sessions (parent_id), and a cwd with a .no-mistakes path
segment are non-interactive. Hermes gateway, cron, and WhatsApp sessions are classified the
same way if they leak past collection's source filter. A no-mistakes pipeline run is just one
kind of non-interactive session, not its own category. Missing harness metadata defaults to
interactive. backpass scan, the proposal, and apply all print the mix so relevance is never
silently computed against a robot-skewed pool.
backpass scan --since 7d --strict
2. Distillation - cheap first
Raw transcripts are mostly tool-call noise; a megabyte of session is a few thousand tokens
of actual signal. Before any model sees anything, backpass reduces each session
deterministically: user and assistant turns verbatim, each tool call collapsed to one line
(tool: Bash "npm test" -> 1 failing), tool output truncated, injected harness scaffolding
dropped, secrets redacted. Typical reduction is 96-99%.
The distilled trace ends with the path to the raw transcript, so the analysis agent can
open the original when - and only when - a specific claim needs it.
3. Calculate loss - one cheap call per transcript
Calculating loss costs one call per transcript, so the set is capped first: past maxTranscripts
(default 100, --max-transcripts) a recency-weighted sample is analyzed instead of
everything. Each transcript's weight halves every sampleHalfLife (default 14d), and the
sample is drawn without replacement, so recent sessions are almost always kept and old
ones stay represented in proportion. When both interactive and non-interactive sessions
are present, slots are allocated in proportion to the corpus with a 20% floor per category,
subject to available sessions and the cap. This keeps a scarce category in the sample
without forcing a genuinely mixed corpus to 50/50. When this happens the run says so on
stderr
(discovered 340 transcript(s), analyzing a recency-weighted sample of 100).
The draw is deterministic and sticky: it is derived from each transcript's own durable
identity, not from a fresh random seed, so rerunning does not trigger a fresh draw that
randomly reshuffles the sample and wastes model calls. Existing transcripts keep their
draws as the corpus grows, while new sessions compete for room on the same footing. Recency
weights still evolve as transcripts age, so the selected set can change over time. Pass
--max-transcripts all to analyze every transcript, or --seed <n> to draw a different,
equally reproducible sample.
Each distilled trace goes to a cheap model with the memory file, the project skill index,
and a rubric. It returns strict JSON: which instructions helped, which were violated, and
what mistakes no current instruction covers. Every negative carries a class - harm (following the instruction
caused damage), non-compliance (the agent ignored it), or irrelevant - because those
argue for opposite fates: harm argues against an instruction, non-compliance argues for
reinforcing it. Every gap carries a domain - orchestration when the mistake was caused
not by this repository but by an external agent harness or tooling that orchestrated the
task, project for every other mistake (including when this repository is the
orchestrating tool) - and the
analysis is shown the ledger's open gaps so it can cite an existing gap id instead of
coining a paraphrase of it.
Every claim must carry a verbatim quote. Quoteless items are discarded - the single
most important defence against a model confabulating influence. Negative evidence is
weighted highest, but its class determines what it supports: non-compliance supports
reinforcement, while only harm supports removal.
To avoid smearing evidence across a large prose blob, backpass splits eligible prose
paragraphs above 120 tokens at high-confidence sentence boundaries for attribution only.
Ambiguous boundaries stay unsplit, and the paragraph remains one line-oriented edit unit.
When its sentence parts
draw repeated non-compliance, synthesis is steered to restructure the paragraph into list
items instead of adding a cosmetic label.
Results are cached per transcript, keyed to the transcript's content, the effective
memory-surface hash, and the analysis-index version. The surface hash covers the memory-file
set plus every project skill's name and description. Edit a memory file or skill description
and the evidence correctly re-computes; edit only a skill body and the cache remains valid
because bodies are inspected only for failed-trigger confirmation. A repo without skills
keeps the prior memory-set hash. A surface edit therefore reanalyzes without --force - that
is not a cache miss, it is the cache doing its job - and the run says so on stderr, naming the
old and new hash, so a "0 reused" line reads as "the surface changed" rather than "reuse is
broken." Evidence files that are not refreshed remain on disk but are excluded while their
full cache key is stale. They become eligible again only when that complete key is current
and the selected-sample and interaction-stamp rules below apply; evidence for transcripts
included in the new analysis is replaced with fresh judgments.
4. Aggregate gradients - and one judged consolidation call
Evidence is grouped by instruction, giving each one a positive/negative count, a count of
distinct sessions with harm-class negatives, and a relevance figure: the share of
analyzed sessions in which it mattered at all. Relevance is reported both overall and
separately for interactive and non-interactive sessions. The fold also reports memory-file
units that substantially overlap a project skill. An overlap with a skill description duplicates
always-loaded tokens and points the shrink at the memory-file copy; an overlap with a
triggered skill body is placement evidence only, since the memory copy may be the only
always-loaded coverage. Neither kind is deleted automatically. Duplicate gaps across
sessions are clustered, and only clusters seen in at least minGapEvidence sessions
(default 2) are eligible for synthesis. Mixed-domain clusters below that floor remain
visible as report-only diagnostics. One bad session never rewrites the weights.
Whether two sightings are one gap is a judgment call, not a word-overlap score - models
paraphrase, and a paraphrase that fails a lexical match would hide real recurrence.
Identity is judged twice: the quote-anchored analysis turn cites an existing gap id when
it sees a gap already on the books, and, when at least two open entries exist, one bounded
consolidation call sees the full open gap set and merges entries that describe the same
mistake. That second judgment is what lets two sightings of a brand-new gap in the same
run's parallel fan-out corroborate. A failed consolidation call degrades the run to
lexical identity and says so; it never aborts. All sightings cluster before their domain
is decided. Each sighting votes project or orchestration; only a majority-orchestration
cluster is excluded from synthesis, while a tie stays eligible. Mixed clusters are always
reported with their orchestration count, even below the evidence floor. A corroborated
majority-excluded cluster, including a pure-orchestration cluster, remains clearly labeled
as a report-only diagnostic rather than becoming an instruction in the project's memory
file; an uncorroborated pure-orchestration singleton stays hidden.
Only evidence with the current transcript, memory-surface, and analysis-index cache key,
stamped with one of the two interaction categories, and belonging to this run's selected
sample is folded into a proposal. A transcript that fell outside the time window or
maxTranscripts cap, disappeared, or still has legacy evidence can leave an evidence file
on disk. That file is left untouched, but it does not count toward this run's session total
or instruction scores, or add a gap observation, until ordinary discovery and analysis
select and refresh it.
Gap sightings persist across runs in .backpass/gap-ledger.json by gap and session, but a
run only counts observations whose sessions belong to its selected sample. This prevents
older observations outside the cap from undoing the sample mix while still allow
Backpass is a CLI tool that reads the agent session transcripts that actually ran in your repo (from Claude Code, Codex and other harnesses) and proposes evidence-backed edits to your AGENTS.md/CLAUDE.md memory file and project skills, instead of a human manually editing it after noticing a failure.