FlashRSI README

The project's README from github.com/RUC-NLPIR/FlashRSI. As one markdown file: README.md.

FlashRSI: blue-white electric arcs trace a lightning logo and the large white wordmark on a deep black background.

FlashRSI: A Unified Framework for Agent Recursive Self-Improvement

One click to reproduce, compare, and develop RSI methods across agents and tasks.

FlashRSI provides one experiment interface for 13 RSI algorithms, 30 benchmarks, and native integrations for MiniAgent, Claude Code, Codex, Pi, and Hermes. Each run records the resources that changed, trajectories that produced the score, and the Snapshot selected at the end.

English · 简体中文

Installation / Features / Quick start / Design Logic / Supporting Methods / Supporting Benchmarks

Benchmark, RSI algorithm, and harness reels settle on a combination, followed by illustrative Training, Held-out, Near-domain, and OOD-domain curves.

Features

🧩 Configurable Experiment Framework: Combines an RSI method, Target harness, Worker, benchmark, and independent model settings in one YAML/CLI experiment. Resource scopes and execution roles can change without writing a new runner.
🧪 30 Benchmarks and 13 RSI Methods: Covers question answering, reasoning, mathematics, coding, web research, documents, tool use, and interactive tasks, with 34 pinned Hugging Face prepared configurations for 29 suites, plus runnable adaptations of OPRO, GEPA, EvoPrompt, ACE, SkillOpt, Evo-Harness, AHE, ADAS, DarwinX, Meta-Harness, ModularRSI, RRSI, and Arbor.
🧠 Native Harnesses and Reproducible Evidence: MiniAgent, Claude Code, Codex, Pi, Hermes, OpenClaw, and OpenCode map prompts, skills, tools, configuration, hooks, plugins, memory, and runtime resources into versioned Snapshots. Each run keeps immutable candidates, redacted configuration, Target/Worker trajectories, metrics, usage, and task artifacts together.
🚀 Simple Workflow and Extensible Docs: Use flashrsi run for evolution and flashrsi evaluate for saved Snapshots, with method examples, a searchable documentation site, and GitHub Pages support. OpenCode is available through the pure-Python Snapshot runtime; it does not require Bun, Node, npm, or the OpenCode CLI.

See the documentation home for the guide, integrations, and development notes.

Installation

To install the framework and all non-Harbor benchmark integrations, use the single all extra:

git clone https://github.com/RUC-NLPIR/FlashRSI.git
cd FlashRSI
uv venv --python 3.12
source .venv/bin/activate
uv sync --extra all

or using pip:

python -m pip install -e ".[all]"

Harbor is kept as a separate extra. Install it together with the complete environment when you need Harbor tasks:

uv sync --extra all --extra harbor

or using pip:

python -m pip install -e ".[all,harbor]"

For Docker-backed tasks, install Docker Desktop or Docker Engine.

Quick start

Run a small RSI experiment with explicit algorithm, benchmark, and target harness choices:

export TARGET_API_KEY="..."
export WORKER_API_KEY="..."

flashrsi run \
  --algorithm skillopt \
  --benchmark searchqa \
  --agent-harness miniagent \
  --model deepseek-v4-pro \
  --worker-model deepseek-v4.1-flash \
  --set agent.llm.api_key="$TARGET_API_KEY" \
  --set algorithm.worker.llm.api_key="$WORKER_API_KEY"

The same experiment can be started entirely from the checked-in YAML file:

flashrsi run --config examples/quickstart.yaml

Run options

The CLI keeps the common choices short and accepts every nested configuration field through a dotted override. --algorithm chooses the RSI search method, while --agent-harness chooses the harness whose resources are evolved. Use flashrsi run --help for the complete parser help.

Option	Purpose
`--config PATH`	Load an existing YAML experiment, including its data, split, and role settings. Start with `examples/quickstart.yaml`.
`--algorithm NAME`, `--algo NAME`	Select the RSI method that proposes and evaluates candidate edits. See Supporting Methods.
`--benchmark NAME`	Select the task suite and scoring protocol used to evaluate candidates. See Supporting Benchmarks.
`--agent-harness NAME`, `--agent NAME`	Select the Target harness to evolve and execute. See Supporting Harnesses.
`--model NAME`	Set the Target model used to solve benchmark tasks.
`--worker-model NAME`	Set the independent Worker model used to propose edits; Agent Workers can be selected through `--set algorithm.worker.type=agent`.
`--output-dir PATH`	Choose the parent directory for run records, Snapshots, trajectories, and artifacts.
`--set PATH=VALUE`	Override any nested setting, such as `--set algorithm.max_steps=3` or `--set algorithm.worker.name=codex`.
`--check-only`	Check dependencies and data preparation without calling models or creating an experiment result.
`--prepare-only`	Write a redacted configuration directory without checking runtime readiness.

After the run completes, the following artifacts are produced:

runs/<algorithm>-<agent>-<benchmark>-<timestamp>/
├── config.yaml           # Redacted configuration
├── snapshots/            # Candidate resources and lineage
├── trajectories/         # Target rollouts, Worker calls, and decisions
├── artifacts/harbor/     # Collected files and native logs for Harbor tasks
├── result.json           # Selected Snapshot and final test result
├── end-summary.json      # Machine-readable summary
└── end-summary.md        # Human-readable summary

Evaluate a Snapshot

Use evaluate to test how an evolved harness performs on another benchmark without running evolution. Any JSON file under a run's snapshots/ directory can be used, including an intermediate candidate:

flashrsi evaluate \
  --config examples/quickstart.yaml \
  --snapshot <run-dir>/snapshots/<snapshot-id>.json

Design Logic

FlashRSI represents an agent harness as a small, tagged file system. A Snapshot contains the files that define the agent's behavior, while credentials, downloaded data, temporary logs, and task state remain outside the Snapshot. Every file is assigned a role such as prompt, skill, tool, runtime, or config; an algorithm declares which roles it may observe and edit.

AgentSnapshot (tagged files)
        │
        ├── selected scope ──► RSI algorithm edits and versions a candidate
        │
        └── complete candidate ──► Harness executes ──► Benchmark scores trajectories

This boundary lets algorithms share the same checkout, execution, and scoring operations while keeping their search policies independent. The harness always executes a complete candidate, so editing one skill still preserves the instructions, tools, and runtime that make that skill effective.

Harness mapping strategies

Harness type	How FlashRSI maps it	Current example
Closed or interface-first	Map the stable interfaces exposed by the product to files. The Snapshot contains the resources that can be edited; proprietary internals stay in the native harness.	Claude Code: instructions, skills, agents, hooks, MCP/LSP registrations, and settings.
Open source with a large runtime	Reduce the execution surface against a pinned baseline, document supported behavior and migration limits, then align that implementation with the Snapshot file layout.	Codex: a Python semantic port of supported paths from a pinned Rust baseline; selected resources execute from the Snapshot in an isolated process.
Clean open source harness	Map the source tree directly and tag each editable file by role. The native entry point continues to execute the selected Snapshot.	MiniAgent: prompts, skills, tools, configuration, and declared runtime code.

See Agent integration, Codex integration, and the usage guide for implementation details.

Supporting Methods

FlashRSI implements 13 RSI methods:

Method	Scope	Search organization	Feedback used
OPRO	Instruction search	Scored history · reflection · population	Train scores, outputs
GEPA	Instruction search	Candidate pool and reflection	Scores, traces
EvoPrompt	Instruction search	GA / DE population	Dev scores
SkillOpt	Skill evolution	Fast / slow updates	Train traces, dev scores
Evo-Harness	Skill evolution	Task-stream accumulation	Failure traces
ACE	Context evolution	Playbook accumulation	Task feedback, reflections
AHE	Iterative harness refinement	Successive edits	Traces, diagnoses, edit audits
Meta-Harness	Iterative harness refinement	File-based history	Code, scores, traces
RRSI	Iterative harness refinement	History-guided revisions	Traces, scores, token costs
ADAS	Archive search	Design archive	Design accounts, dev scores
DarwinX	Branch evolution	Branches and recombination	Repeated outcomes, traces
ModularRSI	Branch evolution	Independent branches; integration	Success / failure traces
Arbor	Branch evolution	Hypothesis tree and insight propagation	Train traces, dev scores

Arbor is newly integrated; the original diagram and experiment results below cover the twelve-method study. Arbor has no real-model results yet.

Overall experiment results

The table below reports source-task performance and frozen transfer performance for the twelve methods evaluated on Claude Code and Mini-Agent. It covers four source benchmarks, five near-domain transfers, and two far-OOD transfers. Scores are percentages; each result cell shows the final score followed by its change from the corresponding initial harness in percentage points, and bold marks the highest final score within each benchmark and harness, including ties.

Method	Target harness	Source _In-domain	Near-domain _{Frozen transfer}	Far-OOD _{Frozen transfer}
SearchQA _In-domain	HotpotQA _In-domain	Terminal-Bench 2.0 _In-domain	AppWorld _In-domain	SearchQA → NQ _Near-domain	SearchQA → TriviaQA _Near-domain	HotpotQA → MuSiQue _Near-domain	Terminal-Bench 2.0 → SWE-bench _Near-domain	AppWorld → BFCL Multi-Turn _Near-domain	SearchQA → OfficeQA Pro _Far-OOD	HotpotQA → DeepSearchQA _Far-OOD
Initial harness	Claude Code	73.93	57.32	63.33	64.88	56.67	68.67	67.50	76.67	65.00	44.19	66.00
Mini-Agent	76.43	58.54	66.67	65.48	59.33	74.33	65.83	80.00	70.00	27.91	52.00
SkillOpt	Claude Code	83.14 _{+9.21 pp}	60.98 _{+3.66 pp}	70.00 _{+6.67 pp}	84.52 _{+19.64 pp}	69.00 _{+12.33 pp}	82.00 _{+13.33 pp}	76.67 _{+9.17 pp}	80.00 _{+3.33 pp}	68.75 _{+3.75 pp}	46.51 _{+2.32 pp}	58.00 _{-8.00 pp}
Mini-Agent	82.93 _{+6.50 pp}	59.76 _{+1.22 pp}	66.67 _{0.00 pp}	91.67 _{+26.19 pp}	62.00 _{+2.67 pp}	84.33 _{+10.00 pp}	74.17 _{+8.34 pp}	86.67 _{+6.67 pp}	68.75 _{-1.25 pp}	37.21 _{+9.30 pp}	46.00 _{-6.00 pp}
OPRO	Claude Code	81.43 _{+7.50 pp}	62.20 _{+4.88 pp}	63.33 _{0.00 pp}	66.07 _{+1.19 pp}	69.67 _{+13.00 pp}	80.33 _{+11.66 pp}	73.33 _{+5.83 pp}	80.00 _{+3.33 pp}	66.25 _{+1.25 pp}	51.16 _{+6.97 pp}	60.00 _{-6.00 pp}
Mini-Agent	80.71 _{+4.28 pp}	60.98 _{+2.44 pp}	66.67 _{0.00 pp}	59.52 _{-5.96 pp}	64.67 _{+5.34 pp}	81.33 _{+7.00 pp}	78.33 _{+12.50 pp}	80.00 _{0.00 pp}	68.75 _{-1.25 pp}	41.86 _{+13.95 pp}	44.00 _{-8.00 pp}
AHE	Claude Code	79.36 _{+5.43 pp}	60.98 _{+3.66 pp}	70.00 _{+6.67 pp}	91.67 _{+26.79 pp}	66.00 _{+9.33 pp}	81.67 _{+13.00 pp}	74.17 _{+6.67 pp}	80.00 _{+3.33 pp}	71.25 _{+6.25 pp}	46.51 _{+2.32 pp}	60.00 _{-6.00 pp}
Mini-Agent	78.57 _{+2.14 pp}	57.32 _{-1.22 pp}	60.00 _{-6.67 pp}	71.43 _{+5.95 pp}	66.33 _{+7.00 pp}	79.00 _{+4.67 pp}	73.33 _{+7.50 pp}	83.33 _{+3.33 pp}	72.50 _{+2.50 pp}	34.88 _{+6.97 pp}	46.00 _{-6.00 pp}
ACE	Claude Code	83.93 _{+10.00 pp}	63.41 _{+6.09 pp}	66.67 _{+3.34 pp}	91.07 _{+26.19 pp}	64.00 _{+7.33 pp}	85.67 _{+17.00 pp}	72.50 _{+5.00 pp}	73.33 _{-3.34 pp}	73.75 _{+8.75 pp}	41.86 _{-2.33 pp}	54.00 _{-12.00 pp}
Mini-Agent	84.71 _{+8.28 pp}	63.41 _{+4.87 pp}	70.00 _{+3.33 pp}	91.67 _{+26.19 pp}	62.67 _{+3.34 pp}	85.33 _{+11.00 pp}	75.83 _{+10.00 pp}	70.00 _{-10.00 pp}	67.50 _{-2.50 pp}	30.23 _{+2.32 pp}	54.00 _{+2.00 pp}
EvoPrompt	Claude Code	74.07 _{+0.14 pp}	58.54 _{+1.22 pp}	63.33 _{0.00 pp}	61.90 _{-2.98 pp}	59.67 _{+3.00 pp}	69.00 _{+0.33 pp}	70.00 _{+2.50 pp}	83.33 _{+6.66 pp}	66.25 _{+1.25 pp}	44.19 _{0.00 pp}	64.00 _{-2.00 pp}
Mini-Agent	76.71 _{+0.28 pp}	59.76 _{+1.22 pp}	76.67 _{+10.00 pp}	64.29 _{-1.19 pp}	58.67 _{-0.66 pp}	74.00 _{-0.33 pp}	67.50 _{+1.67 pp}	86.67 _{+6.67 pp}	63.75 _{-6.25 pp}	34.88 _{+6.97 pp}	40.00 _{-12.00 pp}
GEPA	Claude Code	82.43 _{+8.50 pp}	62.20 _{+4.88 pp}	66.67 _{+3.34 pp}	67.86 _{+2.98 pp}	65.33 _{+8.66 pp}	83.67 _{+15.00 pp}	78.33 _{+10.83 pp}	76.67 _{0.00 pp}	70.00 _{+5.00 pp}	41.86 _{-2.33 pp}	52.00 _{-14.00 pp}
Mini-Agent	84.64 _{+8.21 pp}	59.76 _{+1.22 pp}	70.00 _{+3.33 pp}	75.00 _{+9.52 pp}	61.67 _{+2.34 pp}	86.33 _{+12.00 pp}	75.83 _{+10.00 pp}	70.00 _{-10.00 pp}	62.50 _{-7.50 pp}	51.16 _{+23.25 pp}	34.00 _{-18.00 pp}
ADAS	Claude Code	81.36 _{+7.43 pp}	60.98 _{+3.66 pp}	73.33 _{+10.00 pp}	89.88 _{+25.00 pp}	63.67 _{+7.00 pp}	83.33 _{+14.66 pp}	77.50 _{+10.00 pp}	73.33 _{-3.34 pp}	70.00 _{+5.00 pp}	51.16 _{+6.97 pp}	54.00 _{-12.00 pp}
Mini-Agent	84.86 _{+8.43 pp}	60.98 _{+2.44 pp}	66.67 _{0.00 pp}	79.76 _{+14.28 pp}	62.33 _{+3.00 pp}	84.33 _{+10.00 pp}	74.17 _{+8.34 pp}	73.33 _{-6.67 pp}	70.00 _{0.00 pp}	32.56 _{+4.65 pp}	34.00 _{-18.00 pp}
DarwinX	Claude Code	80.50 _{+6.57 pp}	60.98 _{+3.66 pp}	63.33 _{0.00 pp}	70.83 _{+5.95 pp}	65.67 _{+9.00 pp}	82.00 _{+13.33 pp}	77.50 _{+10.00 pp}	66.67 _{-10.00 pp}	72.50 _{+7.50 pp}	48.84 _{+4.65 pp}	60.00 _{-6.00 pp}
Mini-Agent	79.29 _{+2.86 pp}	60.98 _{+2.44 pp}	66.67 _{0.00 pp}	69.05 _{+3.57 pp}	64.67 _{+5.34 pp}	83.00 _{+8.67 pp}	75.83 _{+10.00 pp}	83.33 _{+3.33 pp}	67.50 _{-2.50 pp}	41.86 _{+13.95 pp}	34.00 _{-18.00 pp}
Meta-Harness	Claude Code	85.07 _{+11.14 pp}	62.20 _{+4.88 pp}	70.00 _{+6.67 pp}	90.48 _{+25.60 pp}	64.00 _{+7.33 pp}	86.33 _{+17.66 pp}	80.00 _{+12.50 pp}	80.00 _{+3.33 pp}	66.25 _{+1.25 pp}	53.49 _{+9.30 pp}	68.00 _{+2.00 pp}
Mini-Agent	79.14 _{+2.71 pp}	59.76 _{+1.22 pp}	73.33 _{+6.66 pp}	88.10 _{+22.62 pp}	65.67 _{+6.34 pp}	81.00 _{+6.67 pp}	72.50 _{+6.67 pp}	76.67 _{-3.33 pp}	73.75 _{+3.75 pp}	41.86 _{+13.95 pp}	38.00 _{-14.00 pp}
ModularRSI	Claude Code	81.07 _{+7.14 pp}	59.76 _{+2.44 pp}	60.00 _{-3.33 pp}	56.55 _{-8.33 pp}	63.00 _{+6.33 pp}	84.00 _{+15.33 pp}	72.50 _{+5.00 pp}	80.00 _{+3.33 pp}	65.00 _{0.00 pp}	44.19 _{0.00 pp}	52.00 _{-14.00 pp}
Mini-Agent	78.64 _{+2.21 pp}	59.76 _{+1.22 pp}	70.00 _{+3.33 pp}	69.05 _{+3.57 pp}	67.00 _{+7.67 pp}	81.00 _{+6.67 pp}	65.83 _{0.00 pp}	73.33 _{-6.67 pp}	75.00 _{+5.00 pp}	53.49 _{+25.58 pp}	40.00 _{-12.00 pp}
RRSI	Claude Code	82.79 _{+8.86 pp}	60.98 _{+3.66 pp}	76.67 _{+13.34 pp}	92.86 _{+27.98 pp}	64.00 _{+7.33 pp}	83.67 _{+15.00 pp}	79.17 _{+11.67 pp}	80.00 _{+3.33 pp}	72.50 _{+7.50 pp}	53.49 _{+9.30 pp}	58.00 _{-8.00 pp}
Mini-Agent	81.64 _{+5.21 pp}	59.76 _{+1.22 pp}	76.67 _{+10.00 pp}	84.52 _{+19.04 pp}	68.67 _{+9.34 pp}	83.00 _{+8.67 pp}	76.67 _{+10.84 pp}	76.67 _{-3.33 pp}	66.25 _{-3.75 pp}	53.49 _{+25.58 pp}	36.00 _{-16.00 pp}
Evo-Harness	Claude Code	75.36 _{+1.43 pp}	60.98 _{+3.66 pp}	66.67 _{+3.34 pp}	81.55 _{+16.67 pp}	58.67 _{+2.00 pp}	71.33 _{+2.66 pp}	77.50 _{+10.00 pp}	80.00 _{+3.33 pp}	65.00 _{0.00 pp}	34.88 _{-9.31 pp}	60.00 _{-6.00 pp}
Mini-Agent	80.64 _{+4.21 pp}	59.76 _{+1.22 pp}	66.67 _{0.00 pp}	77.38 _{+11.90 pp}	65.33 _{+6.00 pp}	80.00 _{+5.67 pp}	75.00 _{+9.17 pp}	76.67 _{-3.33 pp}	72.50 _{+2.50 pp}	48.84 _{+20.93 pp}	58.00 _{+6.00 pp}

Supporting Harnesses

FlashRSI currently supports seven harness integrations; additional integrations are planned.

Harness	Editable resources	Integration details	Status
MiniAgent	Instructions, skills, tools, runtime code, configuration	Native Python entry point; direct file mapping and role tags.	Available
Claude Code	Instructions, skills, agents, hooks, MCP/LSP, settings	Closed SDK integration; stable exposed interfaces are mapped into the Snapshot.	Available
Codex	Instructions, skills, agents, hooks, plugins, memory, rules, configuration	Snapshot-local Python port based on `rust-v0.147.0`, with documented coverage limits.	Available
Pi	Pure-Python Snapshot runtime, instructions, skills, prompt templates, and settings	Snapshot-local `runtime/pi_native` follows Pi v1.4.0's headless loop and Pi-shaped JSON events; no Node/npm dependency.	Available
Hermes	Python Snapshot runtime, SOUL instructions, skills, tool settings	Supported headless Hermes v0.22.5 path with upstream Python helpers and local tools; no installed Hermes CLI required.	Available
OpenClaw	Pure-Python Snapshot runtime, bootstrap files, skills, tools, policy, session and memory settings	Snapshot-local `runtime/openclaw_native` follows the pinned embedded turn path across Chat Completions, Responses and Anthropic Messages; no Node/npm/Gateway dependency.	Available
OpenCode	Pure-Python Snapshot runtime, instructions, tools, permissions, sessions and settings	Snapshot-local `runtime/opencode_native` follows the pinned v1.19.3 embedded turn path across Chat Completions, Responses and Anthropic Messages; no Bun/Node/CLI dependency.	Available

Supporting Benchmarks

FlashRSI supports 30 benchmark suites across question answering, reasoning, mathematics, coding, web research, documents, tool use, and interactive tasks. The table below summarizes each suite's task, and primary metric. 34 pinned Hugging Face configurations are available for 29 suites, while native data sources remain supported.

Domain	Benchmark	Task	Primary metric
QA	SearchQA	Search-snippet short-answer QA	Exact Match
QA	HotpotQA	Multi-hop Wikipedia QA	Joint F1
QA	MuSiQue	Compositional multi-hop QA	Answer F1
QA	TriviaQA	Wikipedia trivia QA	Exact Match
QA	Natural Questions	Short-answer and yes/no QA	Exact Match
QA	2WikiMultiHopQA	Multi-hop QA with context	Exact Match
Reasoning	BBEH	BIG-Bench Extra Hard tasks	Accuracy
Reasoning	SuperGPQA	Graduate-level multiple choice	Accuracy
Reasoning	HLE	Text-only Humanity's Last Exam	Accuracy
Math	MATH	Competition mathematics	Accuracy
Math	AIME	AIME 2025 / 2026	Accuracy
Math	LiveMathBench	Repeated-sampling mathematics	mG-Pass@k
Coding	SWE-bench Verified	Repository issue repair	Resolved rate
Coding	SWE-bench Pro V2	Repository repair	Resolved rate
Coding	DeepSWE	DeepSWE repository tasks	Reward
Coding	Terminal-Bench 2.0 / 2.1	Linux terminal tasks	Reward
Web research	BrowseComp	Broad web research	Accuracy
Web research	DeepSearchQA	Single- and set-answer search	F1
Web research	WideSearch	Tabular broad search	Score
Documents	OfficeQA-Pro	Document research	Accuracy
Documents	Harvey LAB	Legal document work	All-pass rate
Documents	JobBench	Professional file delivery	Reward
Documents	SpreadsheetBench 2	Spreadsheet debugging and modeling	Reward
Tool use	AppWorld	Application and API automation	Reward
Tool use	BFCL V4	Multi-turn function calling	Reward
Tool use	WorkBench	Workplace database tasks	Correctness
Tool use	TravelPlanner	Constraint-aware itineraries	Final pass rate
Interactive	τ²-bench	Airline, retail, telecom dialogue	Reward
Interactive	τ³ Banking	Multi-turn banking support	Reward
Interactive	ALFWorld	Text-only embodied tasks	Success rate

Frequently Asked Questions

How do I add a new benchmark?

Implement a Benchmark (or NativeBenchmark) that yields Task objects and grades Trajectory objects, register it in flashrsi/benchmarks/__init__.py, add a configuration example, and document its source, revision, split policy, metric, and environment requirements. Use the benchmark index and the benchmark interface as the checklist.

How do I add a new algorithm?

Subclass Algorithm, declare the observation and editing scope, and keep proposal, selection, and memory logic inside evolve. Use the shared Snapshot checkout, rollout, and scoring operations; do not move algorithm policy into the runner. Start with the existing algorithm implementations and the usage guide.

How do I add a new harness?

Declare editable resources in a components.yaml, implement an Agent adapter whose _run calls the native entry point, and return a Trajectory. Map the harness according to its source model: interface files for a closed product, a minimal faithful runtime for a large open system, or direct tagged files for a clean open harness. See Agent integration.

What is a Snapshot?

A Snapshot is an immutable, executable version of the harness resources selected for an experiment. It contains the files that define behavior and records parentage and changes; credentials, task data, logs, and other runtime state stay outside it. Any Snapshot can be checked out, evaluated, compared, or promoted without changing the original version.

The persisted Snapshot is one JSON file, usually written as runs/<run-id>/snapshots/<snapshot-id>.json. Its top-level fields include format_version, agent_type, component_rules, id, root_id, parent_id, and files. Each entry in files stores a normalized relative path, the component role, an executable flag, and the file's exact bytes encoded as canonical base64. The Snapshot ID is a SHA-256 hash of the canonical file contents and version metadata, so changing one declared resource produces a new content-addressed version.

When a candidate is evaluated, FlashRSI materializes those bytes into a temporary checkout and the native harness runs from that complete tree. An algorithm's read/write scope is a temporary permission view; it is not serialized into the Snapshot ID or JSON. Saving uses a temporary file, fsync, and an atomic link, so an existing Snapshot file is never silently overwritten.

Citation

Please cite our paper if FlashRSI helps your research:

@misc{jin2026flashrsi,
  title  = {FlashRSI: A Unified Framework for Agent Self-Improvement},
  author = {Jiajie Jin and Yuze Li and Yuyang Hu and Guanting Dong and
            Xiaoxi Li and Xinyu Yang and Jiejun Tan and Zhicheng Dou and Yutao Zhu},
  year   = {2026},
}

License

FlashRSI is released under the Apache License 2.0. Individual benchmark datasets, model APIs, and upstream harnesses may have additional terms; follow the license and usage conditions of each source.

Gaoling School of Artificial Intelligence · Renmin University of China

Searching Database…

172 ·GitHub

FlashRSI

FlashRSI: A Unified Framework for Agent Recursive Self-Improvement

Creator: RUC-NLPIR
Size: 16 MB
Stars: 172
Updated: CommitNov 02 Releasev0.1.0Nov 02
Age: 1mo
Commits · 90 Days

FlashRSI README

FlashRSI: A Unified Framework for Agent Recursive Self-Improvement

Features

Installation

Quick start

Run options

Evaluate a Snapshot

Design Logic

Harness mapping strategies

Supporting Methods

Overall experiment results

Supporting Harnesses

Supporting Benchmarks

How do I add a new benchmark?

How do I add a new algorithm?

How do I add a new harness?

What is a Snapshot?

Citation

License

FlashRSI (RUC-NLPIR/FlashRSI)