rrsi README

The project's README from github.com/google-research/rrsi. As one markdown file: README.md.

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

Check out our paper and project page for more details.

🔥 Updates

🧬 Overview

RRSI overview: proposal-side and selection-side regularization of the harness search

An LLM agent's capability is largely set by its harness: the prompts, control flow, tools, memory and context management around a frozen model. Evolving the harness against a fixed evolve set is effective but overfits: the harness memorizes the training tasks, and large in-distribution gains shrink or vanish out of distribution. RRSI keeps the harness edit space open and regularizes the search trajectory through it instead.

On the proposal side, an annealed budget caps how many independent edits one candidate may bundle, the proposer is conditioned on the full edit history so a falsified hypothesis is not redrawn, and a stalled run is redirected toward components it has never exercised. On the selection side, a critic screens every candidate for suite-specific logic before it is evaluated, a noise-adjusted floor blocks gains within evaluation variance, a cost rule requires added inference tokens to be paid for by measured gain, and components that stop helping are pruned.

✨ Key features

🧩 Method to code

Paper Code
Empirical score and cost estimate rrsi/evaluate.py: aggregate (weighted per-trial rewards; a missing trial counts 0 with the full denominator)
Annealed edit budget b_t rrsi/schedule.py: edit_budget, enforced in the proposer's done()
Edit history L_t, tried set T_t, recent yield g_t rrsi/history.py: History (one JSONL record per edit; a = 1 only for the edits of the candidate that became H_{t+1})
Stall flag, untried components, exploration directives rrsi/history.py: stall_flag, exploration; reserved slots enforced in rrsi/propose.py
Analyze(H_t, D) rrsi/analyst.py dispatching rrsi/digester.py
Proposer with (component, hypothesis, diff) tags rrsi/propose.py; tags validated against the diff by rrsi/components.py
Critic (leakage screen before evaluation) rrsi/critic.py (domain regex denylist plus LLM review, bounded repair)
Evaluate in parallel rrsi/evaluate.py, Run.round thread pool
Noise-adjusted floor, cost rule, within-band rule, argmax rrsi/selection.py
Novelty nu_t (structural component types never in a winning edit) rrsi/components.py: novelty over K_str = client_tool, skill, memory, subagent
Prune set B_t History.prune_set, handed to the proposer with the accepted machinery to remove
Noise band delta fixed per instance in rrsi.json (0.017 / 0.004 / 0.020); rrsi/calibrate.py re-estimates it when delta is null (bootstrap over trials of the base evaluation, or repeated base evaluations)
Non-compensatory domain criteria Domain.guards (engineering: valid-rate drop, no-submission rise)

⚡️ Quickstart

0. Install

git clone https://github.com/google-research/rrsi.git && cd rrsi
pip install -e ".[dev]"            # the search core (Python 3.10 or newer)
python3 -m pytest tests

The benchmark runners live in their own environments: harbor for the coding instance (domains/coding/.venv), and a Python 3.11 environment with pip install -e ".[agentic]" for the workspace and engineering instances (RRSI_AGENT_PYTHON).

1. LLM configuration

The proposer, the analyst, the critic and the frozen policy are Claude Opus 4.8 on Vertex AI (policy_model in domains/coding/rrsi.json, ORCHESTRATOR_MODEL for the other two instances; any LiteLLM model string works). The Harvey LAB judge is Gemini 3.5 Flash.

gcloud auth application-default login
export VERTEX_PROJECT="your-project-id" VERTEXAI_PROJECT="your-project-id"
export VERTEX_LOCATION=global VERTEXAI_LOCATION=global
export RRSI_VERTEX_PROJECTS="your-project-id"      # comma-separated list: the search roles spread their calls round-robin

2. Run an instance

Every instance follows the same shape:

python3 rrsi.py --domain <coding|workspace|eng> smoke     # liveness: compile, construct, a couple of tasks
python3 rrsi.py --domain <name> baseline                   # Evaluate(H_0), seed runs/<name>/frontier.json
python3 rrsi.py --domain <name> run                        # rounds 0..T-1, resumable; touch runs/<name>/STOP to stop
python3 rrsi.py --domain <name> status

Each round drafts two candidates in their own git worktrees, screens them, evaluates both on the full evolve set and fast-forwards evolve/<name> to the winner. runs/<name>/ holds the frontier, the edit history and the raw trials. Hyperparameters live in domains/<name>/rrsi.json and can be overridden on the command line (--T, --k, --delta, --beta1, ...); readjudicate --t <t> re-applies Algorithm 2 to a stored round and reevaluate --t <t> re-measures one after an infrastructure failure.

Open the README of the instance you want to run for its environment, its evaluation protocol and the out-of-distribution runs:

The short version of each:

# coding: Docker + harbor
python3 -m venv domains/coding/.venv && domains/coding/.venv/bin/pip install "harbor>=0.18"
python3 rrsi.py --domain coding baseline && python3 rrsi.py --domain coding run
bash domains/coding/scripts/swe_eval.sh                       # H_0 and the incumbent on SWE-bench Verified

# workspace: a Harvey LAB checkout at the pinned commit; the split is generated from it on first use
git clone https://github.com/harveyai/harvey-labs.git && (cd harvey-labs && git checkout 1da4750 && uv sync)
export HARVEY_LAB_ROOT=$PWD/harvey-labs RRSI_AGENT_PYTHON=~/venvs/rrsi-agentic/bin/python
python3 rrsi.py --domain workspace baseline && python3 rrsi.py --domain workspace run
python3 rrsi.py --domain workspace heldout --label champ      # the 40 held-out tasks; ood/run_{jobbench,gdpval,apex}.sh for the rest

# eng: the official EngDesign tasks in the verifier layout, a grading venv, a jailed tool gateway
git clone https://github.com/AGI4Engineering/EngDesign.git
python3 domains/eng/scripts/engdesign/build_engdesign_bench.py --engdesign-open EngDesign/EngDesign-Open --out domains/eng/engdesign_bench
python3 -m venv domains/eng/.venvs/engdesign && domains/eng/.venvs/engdesign/bin/pip install -r domains/eng/scripts/engdesign/requirements.txt
bash domains/eng/scripts/preflight.sh
python3 rrsi.py --domain eng baseline && python3 rrsi.py --domain eng run
bash domains/eng/scripts/final_eval.sh frontier               # Frontier-Eng, from a Frontier-Engineering checkout

📊 Results

Numbers from the paper, with Claude Opus 4.8 as the frozen policy in every instance and every number measured against the unevolved harness H_0 in the same window. "Evolve" is the split the harness was searched on; the other rows never entered selection. Terminal-Bench, SWE-bench, JobBench, GDPval, APEX-Agents and EngDesign report pass rate, Harvey LAB the fraction of rubric criteria passed and Frontier-Eng Medal points.

Domain Benchmark Role H_0 RRSI Δ
Coding Terminal-Bench 2.1 evolve 74.2 80.2 +6.0
Coding SWE-bench Verified OOD 82.0 83.8 +1.8
Agentic workspace Harvey LAB evolve 89.4 90.5 +1.1
Agentic workspace Harvey LAB ID held-out 86.9 89.2 +2.3
Agentic workspace JobBench OOD 36.0 40.7 +4.7
Agentic workspace GDPval OOD 48.8 52.3 +3.5
Agentic workspace APEX-Agents OOD 34.2 37.9 +3.7
Engineering design EngDesign evolve 50.0 54.9 +4.9
Engineering design Frontier-Eng OOD 17.7 22.0 +4.3

The search is not tied to one policy family: with Gemini 3.5 Flash as the frozen policy, the same coding instance goes from 64.6 to 78.7 on Terminal-Bench 2.1 and from 76.8 to 79.0 on SWE-bench Verified.

🧱 Adding a domain

A domain is one module, domains/<name>/adapter.py, exporting DOMAIN, an instance of rrsi.domain.Domain that implements:

plus harness_path (the evolvable directory), SKILL.md and PATTERNS.md (the proposer's constitution) and rrsi.json (hyperparameters). The core never reads a trajectory format or a benchmark directory itself.

🧪 Tests

python3 -m pytest tests            # or: python3 tests/test_core.py

🙏 Acknowledgements

The starting harnesses are the Terminus-2 agent from harbor and the react_toolbelt agent and runner from archipelago. The instances evaluate on Terminal-Bench, SWE-bench Verified, Harvey LAB, JobBench, GDPval, APEX-Agents, EngDesign and Frontier-Eng.

💬 Citation

@article{xia2026rrsi,
  title={RRSI: Regularized Recursive Self-Improvement of Agent Harnesses},
  author={Xia, Peng and Han, Rujun and Wang, Zifeng and Chen, Yanfei and Zhang, Yufan and Lee, Yoonho and Huang, Chengsong and Yu, Han and CuiZhu, Zhongying and Ming, Yifei and Yao, Huaxiu and Gokturk, Burak and Pfister, Tomas and Lee, Chen-Yu},
  journal={arXiv preprint arXiv:2609.24972},
  year={2026}
}

Contributing

See CONTRIBUTING.md.

License

Apache 2.0; see LICENSE. Third-party code under third_party/ carries its own license.

Disclaimer

This is not an officially supported Google product. This project is not eligible for the Google Open Source Software Vulnerability Rewards Program.

rrsi (google-research/rrsi)

Searching Database…

1.5K ·GitHub
Rrsi
Что это такое?RRSI (Regularized Recursive Self-Improvement) — исследовательский фреймворк от Google, который автоматически улучшает обвязку (harness) LLM-агента — промпты, логику управления, инструменты, память, управление контекстом вокруг замороженной модели. Он ищет варианты изменений обвязки и оставляет только те, что дают измеримый и проверяемый прирост.
Так это готовый агент, с которым можно общаться, или исследовательский код для улучшения агентов?Исследовательский код, а не готовый продукт: основное поисковое ядро (proposer, analyst, critic, evaluator) плюс адаптеры Domain. В комплекте три инстанса — агент для кодинга в терминале, агент для работы с документами и агент для инженерного проектирования; каждый — это адаптер Domain плюс своя стартовая обвязка.
Как запустить один из инстансов?Создайте окружение Python 3.11 и выполните pip install -e '.[agentic]' для инстансов workspace и engineering (инстанс кодинга вместо этого использует harbor в domains/coding/.venv), настройте свою модель, затем запустите инстанс — в каждом раунде в отдельных git worktree создаются, отбираются и оцениваются два кандидата.
Какие LLM для этого нужны — получится только с Claude?По умолчанию proposer, analyst, critic и замороженная политика работают на Claude Opus 4.8 через Vertex AI, но подойдёт любая модель в виде строки LiteLLM. Судья Harvey LAB — это Gemini 3.5 Flash, а раннер инстанса кодинга живёт в собственном окружении harbor.
Держится ли выигрыш на задачах, которых поиск никогда не видел?Отчасти — прирост внутри распределения уменьшается вне его, и именно это RRSI регуляризует. В статье Terminal-Bench 2.1 вырос с 74.2 до 80.2 на наборе evolve, а SWE-bench Verified, который поиск не видел, — только с 82.0 до 83.8; прирост вне распределения остался положительным на JobBench, GDPval, APEX-Agents и Frontier-Eng.
Какие результаты привела статья?С Claude Opus 4.8 в роли замороженной политики: Terminal-Bench 2.1 — с 74.2 до 80.2 (набор evolve), SWE-bench Verified — с 82.0 до 83.8 (вне распределения), Harvey LAB — с 89.4 до 90.5, JobBench — с 36.0 до 40.7 (вне распределения), EngDesign — с 50.0 до 54.9, Frontier-Eng — с 17.7 до 22.0 (вне распределения). С Gemini 3.5 Flash в роли политики тот же инстанс кодинга вырос с 64.6 до 78.7 на Terminal-Bench 2.1.
Что мешает ему обманывать бенчмарк?Критик проверяет каждого кандидата на логику, заточенную под конкретный набор задач, перед оценкой (регулярное выражение с блок-листом для домена плюс проверка LLM с ограниченными исправлениями), шумовой порог отсекает прирост в пределах разброса оценок, а компоненты, которые перестают помогать, удаляются.
На какие бенчмарки нацелены три инстанса?domains/coding: Terminal-Bench 2.1, затем SWE-bench Verified. domains/workspace: Harvey LAB, затем JobBench, GDPval и APEX-Agents. domains/eng: EngDesign, затем EngDesign v1 и Frontier-Eng.
Это официальный продукт Google?Нет. Это исследовательский релиз под Apache 2.0, и в README сказано, что это не официально поддерживаемый продукт Google и он не подпадает под программу Google OSS Vulnerability Rewards Program. Стартовые обвязки — это агент Terminus-2 из harbor и агент reacttoolbelt из archipelago.
What are similar projects?harness-sdk, awesome-rsi, imp, Raven, hive, adk-python, Recuris, autoagent
Which topics is it in?vertex-ai, benchmarks, self-improvement, self-improving, self-improving-agents, context-management, evals, research-tools, self-evolving, papers, agent-harness, agentic
Creator
google-research
Size
2.4 MB
Stars
1.5K
Updated
CommitSep 23
Age
1mo
Commits · 90 Days