penguin-harness README

项目的 README,来自 github.com/Prism-Shadow/penguin-harness。合并为一个 markdown 文件:README.md。

PenguinHarness logo

PenguinHarness

Unified and Stable RSI Framework
Fully automatic optimization: any model · any algorithm · any benchmark

Download App

npm version CI Deploy Site License: Apache-2.0 Node >= 24

Website Docs Blog

Discord X (Twitter) WeChat

PenguinHarness - Let Agents Autonomously Build Better Agents for $0.02 | Product Hunt

English | 简体中文

The PenguinHarness start page: agents, RSI algorithms and Benchmarks, and the examples

Why PenguinHarness

PenguinHarness is an open-source framework for agent self-evolution (recursive self-improvement, RSI). The model stays fixed; the harness evolves: prompts, Skills, tools and hooks. It runs on your computer or server as a desktop app, a Web App, a CLI and an SDK.

Self-evolution

Pick an algorithm, pick a Benchmark, and an agent improves another agent round by round. Every candidate is a new version, snapshotted before it changes, and every score lands in the Evaluation Center.

Toolkit Algorithm
default-rsi-toolkit Trial, reflect, improve: one hypothesis and one change per round
opro-rsi-toolkit OPRO: new instructions from the history of scored ones
ape-rsi-toolkit APE: candidate instructions induced from examples
ace-rsi-toolkit ACE: lessons from Traces curated into a playbook
awm-rsi-toolkit AWM: workflows induced from successful runs
skilladam-rsi-toolkit SkillAdam: evolving issue tracker and adaptive Skill edit budget
skillopt-rsi-toolkit SkillOpt: bounded Skill edits, score-based selection and cross-epoch reflection
openskill-rsi-toolkit OpenSkill: external research and independent virtual tests before frozen-State evaluation
trace-rsi-toolkit TRACE: skill-bank evolution from contrasting trajectories, with consistency measurement
rrsi-rsi-toolkit RRSI: annealed edit budget, a leakage Critic before any measurement, a Pruner for what stopped earning, and a noise-floored selection rule

Every new Project includes seventeen Benchmarks, run as Harbor tasks in Docker: PenguinHarness Benchmark Sec A to Sec E (from rag-bench-essential, DeepSWE, AutomationBench, Terminal-Bench-Science and Terminal-Bench) and a Train/Test pair for each of GDPEVO Essential's six scenarios. More are in penguin-harness-benchmark.

A Benchmark's score chart: three agents climbing across versions

Demo: a decision agent is evaluated, revises its working method once, and is retested on the same model. Its score goes from 84 to 100, and the cost of every run is on screen.

https://github.com/user-attachments/assets/7925b4b5-db78-4e00-b209-89deca394e34

One sentence, one app

Build a legal review agent with a simple workspace where users upload contracts, track the review, and download a report with risky clauses, explanations, and suggested revisions.

The result: a legal review app. Users upload contracts, follow each file's review, and download a report of risky clauses with explanations and suggested revisions. Five sample reviews cost $0.0423 in model calls.

https://github.com/user-attachments/assets/28c0ef95-1dfa-47be-bad0-7326ccdf4e7c

What agents can do

Browser automation
Agents drive your own Chrome through the PenguinHarness Browser extension, or the desktop app's built-in browser. They read pages as text and act through short scripts, so a step costs few tokens. Tutorial

An agent reads the PenguinHarness blog in the Browser panel
Workbenches
Every agent can have its own hot-reloaded workbench: ask for one and the agent writes the page and its server code. The workbench appears as a tab beside the agent's chat. Tutorial

A legal agent's Contract desk page
Remote machines
Add a machine the server reaches over ssh. Conversations in a Workspace on that machine run on it, in the same window. Tutorial

A conversation running on the machine devbox
Agent API
One unified API connects any agent to any external app: send input over HTTP and read the run back as a stream of AMSP events. Tutorial

An agent's Try it panel after one run

Replies can also carry UI: choices, forms, steps, Mermaid diagrams and live widgets (A2UI).

Results

Each harness on the model it is usually paired with, on the same tasks:

Benchmark: PenguinHarness leads the data-analysis suite and ties OpenAI Codex on coding, at a small fraction of both rivals' cost

Built-in plugins

Agents can also write and optimize their own Skills (docs).

Category Plugins
Agent Self-Evolution default-rsi-toolkit, opro-rsi-toolkit, ape-rsi-toolkit, ace-rsi-toolkit, awm-rsi-toolkit, skilladam-rsi-toolkit, skillopt-rsi-toolkit, openskill-rsi-toolkit, trace-rsi-toolkit, rrsi-rsi-toolkit
Office Productivity a2ui, data-analysis, use-firecrawl, browser-automation, use-bento-slides, humanizer, goal, continual-learning, trace-lookup, agent-company, penguin-support
Software Development software-development, use-claude-code
AI App Development agent-development, agent-evaluation, model-development, skill-porting

Supported models

These are the model families we focus on supporting. The app's Models page lists each variant and its providers.

Closed-source models

Model Providers
GPT 6.1 OpenAI, OpenRouter
Claude 5.5 Anthropic, OpenRouter, OpenCode Go
Gemini 4 Google Gemini, OpenRouter, Penguin Go, OpenCode Go
Grok 4.7 OpenRouter, OpenCode Go
Qwen3.8 Max Qwen Token Plan, Qwen Pay-As-You-Go, OpenRouter, TokenDance, OpenCode Go

Open-source models

Model Providers
DeepSeek V4.1 DeepSeek, OpenRouter, Fireworks AI, Atlas Cloud, TokenDance, Penguin Go, OpenCode Go, Qwen Token Plan, Qwen Pay-As-You-Go, ModelScope
Kimi K3 Moonshot AI, OpenRouter, Fireworks AI, TokenDance, OpenCode Go, Qwen Pay-As-You-Go
GLM 5.3 Z.AI, OpenRouter, TokenDance, OpenCode Go
Hunyuan 4 OpenRouter, TokenDance, OpenCode Go
MiMo v2.6 OpenRouter, TokenDance, OpenCode Go

OpenCode Go offers Claude Haiku 5.5; Fable 5.1, Opus 5.5 and Sonnet 5.5 are available through Anthropic and OpenRouter. Hunyuan 4 uses hy4-preview.

Custom OpenAI-protocol endpoints also work, covering 1000+ online and local models.

Install

Desktop app

Download the installer from penguin.ooo/download: macOS 11+ (dmg), Windows 10+ (exe) or Linux x64 (AppImage, deb). The app opens signed in and needs no terminal.

CLI and server

Platform Install command
Linux / macOS curl -fsSL https://penguin.ooo/install.sh | sh
Windows (PowerShell) irm https://penguin.ooo/install.ps1 | iex
npm (Node >= 24) npm install -g @prismshadow/penguin-cli

Then run penguin web. It serves the Web App at http://127.0.0.1:7364 and prints a first-login link until a password is set; models are configured on the Models page. The desktop app and the CLI share ~/.penguin/data.

Docker

docker run -d --name penguin -p 127.0.0.1:7364:7364 -v penguin-data:/data hiyouga/penguinharness:0.2.18
docker logs penguin       # the first-login link, openable as it stands on this machine

The image serves on port 7364 with its data on the /data volume, for linux/amd64 and linux/arm64; latest follows changes to the image's runtime or build inputs on main, and each release is also tagged with its version. The example listens on loopback only. For a network-facing deployment, Compose and upgrades, see the Docker quickstart.

Offline install (air-gapped machines)

Every GitHub Release attaches exactly one package per target — Linux and macOS in x64 / arm64, Windows in x64, plus a runtime-less universal package. The same file serves online and offline installation. Each package seals the program payload, its SHA256 checksum and the platform's installer. Download the one file on a networked machine, copy it to the target, extract once and run the bundled installer — no network, no separate checksum file to carry (the sealed SHA256 is always verified).

Linux (on arm64, use penguin-linux-arm64.tar.gz):

mkdir penguin-install
tar -xzf penguin-linux-x64.tar.gz -C penguin-install
./penguin-install/install.sh

macOS (Apple silicon shown; on Intel, use penguin-darwin-x64.tar.gz):

mkdir penguin-install
tar -xzf penguin-darwin-arm64.tar.gz -C penguin-install
./penguin-install/install.sh

Windows (unzip, then double-click install.cmd — or run it in PowerShell):

Expand-Archive penguin-win32-x64.zip -DestinationPath penguin-install
cd penguin-install
.\install.cmd

Chrome extension (optional)

PenguinHarness Browser lets agents drive the tabs you hand them in your own Chrome.

CLI and SDK

The same engine, scriptable, and made to be driven by agents:

penguin config model add --provider deepseek --model-id deepseek-flash --api-key sk-... --set-default
penguin run -m "Create hello.txt containing Hello, Penguin"   # one-shot task
penguin chat       # interactive REPL (/compact, /clear, /exit, Ctrl-C to interrupt)
penguin server     # headless service (same API the Web App uses)
import { createAgent, isCompleteModelMessage, userText } from "@prismshadow/penguin-core";

const agent = await createAgent({ agentId: "default_agent" });
const session = await agent.createSession({ workspaceDir: process.cwd() });

for await (const output of session.run([userText("Create hello.txt containing hi")], {
  approve: async () => "allow", // per-tool-call approval
})) {
  if (isCompleteModelMessage(output) && output.payload.type === "text") {
    console.log(output.payload.text);
  }
}

Roadmap

Development

pnpm install && pnpm build   # build first: core's exports point at dist/
pnpm dev                     # backend + web app together (prefixed logs, deps built once)

Working on this repository with a coding agent: its development Skills are in .agents/skills/ (penguin-harness-dev, penguin-harness-frontend, penguin-harness-manual-test, penguin-harness-writing). Claude Code loads them with no install, through the .claude → .agents symlink; for another agent, link or copy .agents/skills/* into the directory it reads Skills from.

See CONTRIBUTING.md for the full workspace guide: dev commands, quality gates, repo layout, and the changelog rule.

Contributors

Thanks to everyone who has contributed to PenguinHarness!

PenguinHarness contributors

Citation

If you use PenguinHarness in your research, please cite:

@software{penguinharness2026,
  author  = {{PrismShadow Team}},
  title   = {PenguinHarness: Efficient Self-Improving Harness for Everyone},
  year    = {2026},
  url     = {https://github.com/Prism-Shadow/penguin-harness},
  license = {Apache-2.0}
}

Acknowledgements

This repo benefits from:

See THIRD-PARTY-NOTICES.md for their licenses.

License

Apache-2.0 © 2026 Prism Shadow

Maintenance team

Organizations: PrismShadow AI, Qinghe Dynamic

Maintainers:

Built with ❤️

penguin-harness (Prism-Shadow/penguin-harness)

正在搜索数据库…

这是什么?PenguinHarness 是一个开源框架,用于智能体自我进化(递归自我改进,RSI)。模型本身保持不变,而它外围的框架(harness),包括提示词、Skills、工具和钩子,会一轮接一轮地被自动改进。
这是框架、桌面应用还是 CLI 工具?这三者它都是:同一个引擎同时提供桌面应用、Web 应用、CLI 和 SDK,你既可以通过图形界面点选操作,也可以用脚本自行调用。
我该如何让它运行起来?从 penguin.ooo/download 下载适用于 macOS、Windows 或 Linux 的桌面安装包;或者用 curl/irm 脚本或 `npm install -g @prismshadow/penguin-cli` 安装 CLI/服务端,然后运行 `penguin web` 启动 Web 应用。
需要哪些模型和 API 密钥?需要你自备服务商密钥。闭源模型包括 GPT、Claude、Gemini、Grok 和 Qwen(可通过其官方 API 或 OpenRouter 调用);开源模型包括 DeepSeek、Kimi、GLM 和混元(Hunyuan)。也支持自定义 OpenAI 协议的接口,可接入 1000 多个在线及本地模型。
路线图上的功能都已经做完了吗?还没有完全做完。公开基准测试套件、桌面应用和 Windows 支持已经发布,但路线图上的企业级自我进化,以及 OpenShell(权限受控的 shell)集成,仍未勾选完成。
这是为谁设计的?为想要低成本构建或自动化智能体应用和工具的开发者设计。他们可以使用经过自我进化调优的开源模型,而不必为每个任务都依赖昂贵的前沿模型调用。
包含哪些自我进化算法?PenguinHarness 内置五套 RSI 工具包:Default、OPRO、APE、ACE 和 AWM。它们全部运行在同一套内置的 Benchmarks 与 Evaluation Center 上,因此得分可以直接比较。
比其他智能体框架便宜多少?项目在自家的数据分析基准测试套件上报告称,其最佳准确率所需的成本,约为 Claude Code 成本的七十分之一。
智能体能控制我的浏览器吗?可以。智能体可以通过 PenguinHarness Browser 扩展,或桌面应用内置的浏览器,操作你自己的 Chrome。它以文本形式读取页面,并通过简短脚本执行操作,从而降低 Token 用量。
能把它连接到远程机器吗?可以。你可以添加一台服务器能通过 SSH 访问的机器,此后该机器上 Workspace 中的对话会直接在它上面运行,并且都在同一个窗口内完成。
大小
114.8 MB
星标
3.3K
时长
4个月
提交 · 90 天