benjamin-plus-skill README

The project's README from github.com/JetBrains/benjamin-plus-skill. As one markdown file: README.md.

Benji — the benjamin-plus mascot

“Beware of little expenses; a small leak will sink a great ship.”
— Benjamin Franklin

benjamin-plus

A token-efficiency skill for coding agents.
It changes how the agent looks things up and waits — never what it builds.

Measured: up to −18 % cost and −22 % tokens per task, quality unchanged.

What the skill teaches

An agent pays twice for every clumsy lookup: once for the step itself, and again every time the growing conversation gets re-read. So the skill teaches five habits:

  1. Recon in one pass. Gather the facts up front in one combined step instead of poking at the repo five separate times. And before copying a format or convention, look at two real examples, not one.
  2. Keyhole reads. When the agent only needs to see something, it reads 50 lines, not the whole file. Data it will actually transform is never truncated.
  3. Probe the environment once. Check every dependency in one command and install whatever is missing in one go, instead of discovering them one crash at a time.
  4. Green means the task's own check. If the task says how to verify, that command is the definition of done. A missing compiler is still the agent's problem to fix, and a check that fails twice means the approach is wrong, not the symptom. When it passes: stop.
  5. Polling is a step. A build that hasn't finished has nothing new to say. Check on it every 30 seconds, not every second. On some agent platforms, polling alone turned out to be nearly half of all steps.

The skill's full text: RULESET.md (~745 tokens injected).

What to expect

Median paired savings across six metrics

Token consumption by arm

Install — inject it, don't "install" it

Same skill, two delivery methods, tested head-to-head: injected, it saves (−17.9 % cost median on the charts above; −4.4 % even on the harder Java/Codex setup) — as a discoverable skill folder, it saves nothing (−0.5 %, n.s.; agents burned steps just finding SKILL.md). So: inject.

git clone https://github.com/JetBrains/benjamin-plus-skill ~/.benjamin-plus

Claude Code — add to ~/.claude/settings.json (verify with /hooks, or just ask Claude Code to add it):

{ "hooks": { "SessionStart": [ { "matcher": "startup|resume|clear|compact",
  "hooks": [ { "type": "command", "command": "cat ~/.benjamin-plus/injected-instruction.md" } ] } ] } }

…or per-project, zero config: cat ~/.benjamin-plus/injected-instruction.md >> CLAUDE.md

Codex CLI — AGENTS.md is loaded into every session; no hook needed:

cat ~/.benjamin-plus/injected-instruction.md >> ~/.codex/AGENTS.md   # or >> AGENTS.md in a repo

Any other agent — append injected-instruction.md to the system prompt. That's the whole integration (~3 KB).

How it was measured

This skill wasn't written once and shipped. It came out of auto research: an agent working in a loop against the benchmark. Mine ~1,200 old agent traces for where the money actually goes, draft rules, run a paired A/B, read the failing trajectories, revise, run again. Six versions later, only the rules that survived the evidence were left. Everything that traded quality for savings got deleted along the way, which turned out to be most of the clever ideas.

The measurement itself is a paired A/B. Same agent, same model, same tasks, same container images; the only difference between the arms is the injected skill text. 80 paired SkillsBench tasks (Claude Code 2.1.201 in Docker sandboxes, Sonnet 5, low effort), Wilcoxon on the paired deltas, a sign test on rewards, and a per-trial adoption check: the payload reached the model in 80 of 80 treated runs and 0 of 80 controls. Trials that failed on one side only were retried before anything was counted. Every number and caveat lives in EXPECTED-RESULTS.md.

Feedback

Found a regression, a workload where it loses money, or a rule that misfires on your stack? Open an issue — ideally with the paired numbers (with/without) and, if you can, a trace. Results from other benchmarks and harnesses are especially welcome; that's how v6 got its polling rule.

License

MIT.

benjamin-plus-skill (JetBrains/benjamin-plus-skill)

Searching Database…

329 ·GitHub
Benjamin-plus-skill
这是什么？Benjamin-Plus 是一个提升 token 效率的编程智能体技能：它传授五个围绕查询和等待的习惯（不涉及智能体构建什么），经实测最多可削减 18% 的成本且质量不变。
这是我安装的插件，还是注入到智能体提示词里的文本？它不是你要安装的包——而是一个简短的规则文件（RULESET.md，约 745 个 token），你直接把它注入智能体的系统提示词或钩子即可；测试表明，把它打包成可被发现的技能文件夹毫无收益，因为智能体光是寻找 SKILL.md 就浪费了不少步骤。
怎么把它加到我的智能体上？在 Claude Code 中，把规则集追加到 CLAUDE.md（例如用“cat ~/.benjamin-plus/injected-instruction.md >> CLAUDE.md”），或通过 ~/.claude/settings.json 里的钩子接入；在 Codex CLI 中，把它追加到 AGENTS.md，因为该文件每次会话都会加载；对其他任何智能体，把 injected-instruction.md 追加到其系统提示词即可。
哪些编程智能体能使用它？它在 Claude Code 2.1.201（Sonnet 5）和 Codex CLI（gpt-5.6-luna）上做过基准测试，但任何系统提示词或上下文可编辑的编程智能体都能使用——不需要 API 密钥或额外工具。
它每次省下的成本都一样多吗？节省幅度取决于你原本的会话有多臃肿，成本大致降低 10% 到 18%；一天前对一个运行得更精简的基线所做的完全相同的测试只测得 −10%，而且质量对比（7 项更好/5 项更差/68 项持平）在统计上也不足以排除任一方向的微小效应。
这个技能传授的五个习惯是什么？在一次合并的侦察中收集事实，而不是多次零散查询；只读取文件中需要的那几行（锁孔式读取）；一次性探测所有依赖；把任务自身的验证命令当作完成的定义；把轮询也计为一步——每 30 秒检查一次，而不是每秒一次。
Benjamin-Plus 是如何开发和测试的？由一个智能体循环挖掘约 1,200 条旧的智能体执行轨迹、起草规则，并在 80 个 SkillsBench 任务（Claude Code 2.1.201、Sonnet 5）上进行成对 A/B 测试，历经六个修订版本，只保留经证据检验成立的规则。
它只在 Claude Code 上有效吗？不是——它还在 Codex CLI 上用 gpt-5.6-luna 在一套 Java SWE-bench 题目（675 组成对副本）上做过测量，结果显示成本降低 4.4% 而解题率不变，因此它能跨不同框架推广。
注入的内容有多大？整个集成内容约 3 KB 文本，其中核心规则集本身注入系统提示词后仅约 745 个 token。
What are similar projects?SkillOpt, Intent-Router, ralph, pstack, headroom, HERO-Anti-OverDefense, SwiftUI-Agent-Skill, SwiftUI-Agent-Skill
Which topics is it in?optimization, prompt-engineering, shell, benchmark, coding-agent, claude, agent-skills, performance, codex, research, agent, claude-code
Creator
JetBrains
Size
1 MB
Stars
329
Updated
CommitAug 25
Age
2mo
Commits · 90 Days