toolrush README

The project's README from github.com/OnlyTerp/toolrush. As one markdown file: README.md.

ToolRush — kill the tool-call tax

shipped and live v2.0 206 tests passed Windows / MSYS

Modern agent models stream tokens faster than their harness can read a file. The bottleneck stopped being tokens/sec — it became the tool-call tax: every read_file, search_files, and terminal call paying process spawns, shell round-trips, wrapper layers, and full re-dispatch for work that costs microseconds.

ToolRush kills the tax. It is a low-overhead execution layer for Hermes Agent: same tools, same output envelopes, same safety gates — radically cheaper transport, plus real batched parallelism and survival across harness updates.

Benchmark bars: stock shell path vs ToolRush v2 on the real installed harness

Results (measured on the real installed harness — no mocks, no fixtures)

Lane Before After Win
Native file reads 255.23 ms 4.44 ms 57.5x
Warm terminal (persistent shell) 285 ms 12.1 ms 23.6x
Search transport (direct rg) 183–455 ms 27–97 ms 4.7–6.8x
Batched parallel RPC 108 ms seq 53 ms batched 2.1x (3.3x controlled overlap)

These are tool-operation wall times, not model-inclusive turn speed — the honest framing: tool-heavy turns get dramatically faster, chat-heavy turns barely move. Full samples, p95s, methodology, and one disclosed regression (trivial native reads don't benefit from threading) in v2/README.md.

Architecture

Five lanes: native read, direct rg transport, warm terminal, batched parallel RPC, update survival

  1. One search engine, accelerated transport. No second, less-correct reimplementation. Direct rg.exe execution preserves real ignore files, regex grammar, context flags, and configuration; native Windows reads reuse the upstream bounded reader, access guards, binary/document routing, and output assembler.
  2. Correctness before speed. Fixed JSON-breaking trailing text in search results; pagination now has stable content order and a more-results sentinel; regex backslashes and leading hyphens stay literal; CRLF and unterminated final lines handled consistently.
  3. Real programmatic parallelism. from hermes_tools import parallel — a batch of 1–16 read operations runs on up to 4 workers through one RPC, returns input order, and keeps authentication, tool allowlists, call budgets, and cell retirement fully enforced. Whole invalid batches are rejected before dispatch. No writes, no terminal.
  4. Correct warm-shell transport. One persistent bash, streaming through an OS pipe with bounded parser memory, a filtered atomic snapshot commit, preserved exit status/cwd/exports, and command-tree kill on cancellation. Never retries a submitted command.
  5. Hardened scheduler admission. Fail-closed classifier refuses hidden writes (wget, curl -o, sed w, branch creation, env-wrapped scripts, shared cwd mutations). Admission is not approval: anything refused still runs — just sequentially, the regular way.
  6. Update survival. Hash-verified helper sources and 25 function-scoped compatibility patches live outside the upstream checkout; after a harness update the plugin restores them in memory, preserving imported references. Unknown upstream drift degrades loudly instead of overwriting new code.
  7. Diagnostics and rollback. doctor.py --smoke, per-lane kill-switches (TOOLRUSH_*=0), a master toolrush.enabled: false, source preimages, payload hashes, and a documented runbook.

Verification

206 passed 5 negative controls live activation verified

Repo layout

Path What
v2/ The shipped implementation — full report, intent recovery, design, MANIFEST (sha256), plugin, installed-source snapshot, evidence
toolrush.py v1 lab runtime (fast_read / batch_read, persistent pool, session cache)
toolrush_search.py · toolrush_exec.py wave-3 in-process search · wave-2 persistent-shell executor
bench_*.py, dissect_*.py the profiling and benchmarking that named the tax
validation-contract*.md VAL- contracts, one per wave (contract-first)
results.md, *.json measured evidence — no invented numbers anywhere

The origin story: lab waves

Before v2 shipped into the live tree, the tax was found and killed one wave at a time in this repo. Kept for the receipts:

Wave Target Result
1 read_file dispatch 1460 ms → 1.18 ms (1237x) — root cause: up to 5 shell commands per read (stat + head|base64 + sed|cut + wc -l + tail); the bytes cost 0.25 ms, the wrapping cost 1460 ms
2 terminal spawn tax echo 285 ms → 12.1 ms (23.6x) via one persistent bash; win decomposed by negative control into ~8x wrap-trim + ~3x shell persistence
3 content search 900 ms → 42 ms (21.3x) in-process walk + memoized real-guard verdicts; match sets identical 40/40
4 dispatch pipeline measured, then STOP — registry 0.005 ms, full path 4.44 ms; the remaining "tax" is load-bearing safety rails. No theater.

Wave 4's verdict is the project's favorite line: know when to stop. v2 then rebuilt the lab wins as production code — one engine, correct semantics, hardened admission, update survival — because a 21x search that drops .gitignore semantics is a bug with a speedup.

Laws

Built for Hermes Agent by Nous Research · shipped and running live since 2026-09-05.

toolrush (OnlyTerp/toolrush)

Searching Database…

157 ·GitHub
Toolrush
What is this?ToolRush is a low-overhead execution layer plugin for Hermes Agent (Nous Research's agent harness) that eliminates the latency overhead of tool calls like file reads, searches, and terminal commands while keeping the same tools, outputs, and safety gates.
Is this a standalone tool or a plugin for something else?It's a plugin that installs into an existing Hermes Agent harness and accelerates its transport layer; it deliberately avoids reimplementing the search engine or file reader, instead speeding up the real ones.
How do I verify it's working after install?After installing the plugin into the harness, you run doctor.py --smoke for diagnostics, and can disable any accelerated lane individually via a TOOLRUSH=0 kill-switch or the master toolrush.enabled flag.
Do I need Hermes Agent installed first?Yes, ToolRush is a plugin that only functions inside an installed Hermes Agent harness, and its fast search lane shells out directly to rg.exe (ripgrep on Windows) while reusing the harness's native Windows file reader.
Does it speed up every kind of agent turn?No — the speedups apply to tool-operation wall time, not whole model turns, so tool-heavy turns get dramatically faster while chat-heavy turns barely change; trivial native reads also don't benefit from the threading used for batched reads.
Who is this built for?It's built specifically for users and operators of Hermes Agent, the harness made by Nous Research, not as a general-purpose tool-calling accelerator.
How much faster does ToolRush make native file reads?Native file reads drop from 255.23ms to 4.44ms, a measured 57.5x speedup, benchmarked on the real installed harness with no mocks or fixtures.
What happens if a command is flagged as unsafe to accelerate?ToolRush's fail-closed admission classifier refuses to accelerate hidden-write operations like wget, curl -o, or branch creation, but those calls still run — just through the normal slower sequential path instead of being blocked.
How does ToolRush keep working after Hermes Agent updates?It stores hash-verified helper sources and 25 function-scoped compatibility patches outside the upstream checkout, then restores them in memory after an update, and degrades loudly rather than silently breaking if upstream code has drifted unexpectedly.
What does the parallel batch reading feature do?Through hermestools.parallel, 1-16 read operations can run across up to 4 workers in one RPC call, returning results in input order while still enforcing authentication, allowlists, and call budgets; invalid batches are rejected entirely before dispatch.
What are similar projects?hermes-agent-self-evolution, pi-autoresearch, billion-context, agentbox, hermes-agent, botmaker, spinel, Proteus
Which topics is it in?nous-research, parallel, agent-tools, rpc, hermes-agent, optimization, caching, concurrency, harness, testing, benchmark, plugin
Creator
OnlyTerp
Size
0.6 MB
Stars
157
Updated
CommitSep 05
Age
1mo
Commits · 90 Days