benjamin-plus-skill README

The project's README from github.com/JetBrains/benjamin-plus-skill. As one markdown file: README.md.

Benji — the benjamin-plus mascot

“Beware of little expenses; a small leak will sink a great ship.”
— Benjamin Franklin

benjamin-plus

A token-efficiency skill for coding agents.
It changes how the agent looks things up and waits — never what it builds.

Measured: up to −18 % cost and −22 % tokens per task, quality unchanged.

What the skill teaches

An agent pays twice for every clumsy lookup: once for the step itself, and again every time the growing conversation gets re-read. So the skill teaches five habits:

  1. Recon in one pass. Gather the facts up front in one combined step instead of poking at the repo five separate times. And before copying a format or convention, look at two real examples, not one.
  2. Keyhole reads. When the agent only needs to see something, it reads 50 lines, not the whole file. Data it will actually transform is never truncated.
  3. Probe the environment once. Check every dependency in one command and install whatever is missing in one go, instead of discovering them one crash at a time.
  4. Green means the task's own check. If the task says how to verify, that command is the definition of done. A missing compiler is still the agent's problem to fix, and a check that fails twice means the approach is wrong, not the symptom. When it passes: stop.
  5. Polling is a step. A build that hasn't finished has nothing new to say. Check on it every 30 seconds, not every second. On some agent platforms, polling alone turned out to be nearly half of all steps.

The skill's full text: RULESET.md (~745 tokens injected).

What to expect

Median paired savings across six metrics

Token consumption by arm

Install — inject it, don't "install" it

Same skill, two delivery methods, tested head-to-head: injected, it saves (−17.9 % cost median on the charts above; −4.4 % even on the harder Java/Codex setup) — as a discoverable skill folder, it saves nothing (−0.5 %, n.s.; agents burned steps just finding SKILL.md). So: inject.

git clone https://github.com/JetBrains/benjamin-plus-skill ~/.benjamin-plus

Claude Code — add to ~/.claude/settings.json (verify with /hooks, or just ask Claude Code to add it):

{ "hooks": { "SessionStart": [ { "matcher": "startup|resume|clear|compact",
  "hooks": [ { "type": "command", "command": "cat ~/.benjamin-plus/injected-instruction.md" } ] } ] } }

…or per-project, zero config: cat ~/.benjamin-plus/injected-instruction.md >> CLAUDE.md

Codex CLI — AGENTS.md is loaded into every session; no hook needed:

cat ~/.benjamin-plus/injected-instruction.md >> ~/.codex/AGENTS.md   # or >> AGENTS.md in a repo

Any other agent — append injected-instruction.md to the system prompt. That's the whole integration (~3 KB).

How it was measured

This skill wasn't written once and shipped. It came out of auto research: an agent working in a loop against the benchmark. Mine ~1,200 old agent traces for where the money actually goes, draft rules, run a paired A/B, read the failing trajectories, revise, run again. Six versions later, only the rules that survived the evidence were left. Everything that traded quality for savings got deleted along the way, which turned out to be most of the clever ideas.

The measurement itself is a paired A/B. Same agent, same model, same tasks, same container images; the only difference between the arms is the injected skill text. 80 paired SkillsBench tasks (Claude Code 2.1.201 in Docker sandboxes, Sonnet 5, low effort), Wilcoxon on the paired deltas, a sign test on rewards, and a per-trial adoption check: the payload reached the model in 80 of 80 treated runs and 0 of 80 controls. Trials that failed on one side only were retried before anything was counted. Every number and caveat lives in EXPECTED-RESULTS.md.

Feedback

Found a regression, a workload where it loses money, or a rule that misfires on your stack? Open an issue — ideally with the paired numbers (with/without) and, if you can, a trace. Results from other benchmarks and harnesses are especially welcome; that's how v6 got its polling rule.

License

MIT.

benjamin-plus-skill (JetBrains/benjamin-plus-skill)

Searching Database…

332 ·GitHub
Benjamin-plus-skill
What is this?Benjamin-Plus is a token-efficiency skill for coding agents: it teaches five habits around looking things up and waiting (not what the agent builds), measured to cut cost by up to 18% with quality unchanged.
Is this a plugin I install, or text I inject into the agent's prompt?It is not a package you install — it is a short rules file (RULESET.md, about 745 tokens) that you inject directly into the agent's system prompt or hook; packaging it as a discoverable skill folder saved nothing in testing because agents burned steps just finding SKILL.md.
How do I add it to my agent?For Claude Code, append the ruleset to CLAUDE.md (for example with 'cat ~/.benjamin-plus/injected-instruction.md >> CLAUDE.md') or wire it via a hook in ~/.claude/settings.json; for Codex CLI append it to AGENTS.md since that file loads every session; for any other agent, append injected-instruction.md to its system prompt.
Which coding agents can use it?It was benchmarked on Claude Code 2.1.201 (Sonnet 5) and Codex CLI (gpt-5.6-luna), but it works with any coding agent whose system prompt or context you can edit — no API keys or extra tooling required.
Does it always save the same amount of cost?Savings scale with how bloated your baseline sessions already are, roughly −10% to −18% cost; an identical run against a leaner-running baseline a day earlier only measured −10%, and the quality comparison (7 better/5 worse/68 ties) wasn't statistically powered to rule out small effects either way.
What are the five habits the skill teaches?Gather facts in one combined recon pass instead of many small lookups, read only the needed lines of a file (keyhole reads), probe every dependency in one go, treat the task's own verification command as the definition of done, and count polling as a step by checking every 30 seconds instead of every second.
How was Benjamin-Plus developed and tested?An agent mined about 1,200 old agent traces in a loop, drafted rules, and ran paired A/B tests on 80 SkillsBench tasks (Claude Code 2.1.201, Sonnet 5) across six revisions, keeping only the rules that survived the evidence.
Does it only work on Claude Code?No — it was also measured on Codex CLI with gpt-5.6-luna on a Java SWE-bench set (675 paired replicas), showing −4.4% cost with an unchanged solve rate, so it generalizes across harnesses.
How big is the injected payload?The whole integration is about 3 KB of text, with the core ruleset itself only around 745 tokens once injected into the system prompt.
What are similar projects?SkillOpt, Intent-Router, ralph, pstack, headroom, HERO-Anti-OverDefense, SwiftUI-Agent-Skill, SwiftUI-Agent-Skill
Which topics is it in?optimization, prompt-engineering, shell, benchmark, coding-agent, claude, agent-skills, performance, codex, research, agent, claude-code
Creator
JetBrains
Size
1 MB
Stars
332
Updated
CommitAug 25
Age
2mo
Commits · 90 Days