Awesome Harness Engineering 
A curated list of articles, playbooks, benchmarks, specifications, and open-source projects for harness engineering: the practice of shaping the environment around AI agents so they can work reliably.
Harness engineering sits at the intersection of context engineering, evaluation, observability, orchestration, safe autonomy, and software architecture. This list focuses on resources that make agents more dependable in real workflows, especially long-running coding and research tasks.
Generic agent tooling is out of scope unless the page directly covers harness design, context management, evaluation, runtime control, or other reliability-critical harness primitives.
Contents
Courses & Learning Resources
- walkinglabs/learn-harness-engineering - A project-based course repository on making Codex and Claude Code more reliable, centered on an Electron personal knowledge base app with lecture handouts, example artifacts, and practical harness projects.
- Phelan164/codex-howto - A Codex-focused engineering curriculum with installable skills, repository instructions, scoped permissions, testing, review, orchestration, and reproducible token measurements for building an inspectable coding-agent harness.
- hardness1020/awesome-agent-architecture - Trilingual architecture notes and runnable demos covering agent loops, tool execution, memory, permissions, context delivery, and orchestration.
Foundations
- Harness engineering: leveraging Codex in an agent-first world - OpenAI's flagship field report on building a large application with Codex using architectural constraints, repo-local instructions, browser validation, and telemetry.
- Effective harnesses for long-running agents - Anthropic's core article on initializer agents, feature lists,
init.sh, self-verification, and handoff artifacts across many context windows.
- Harness design for long-running application development - Anthropic follow-up focused on improving long-running app generation with better task state and evaluator design.
- The Anatomy of an Agent Harness - LangChain's concise framing of an agent as model plus harness, with prompts, tools, middleware, orchestration, and runtime infrastructure.
- Harness Engineering - Thoughtworks' framing of harness work into context engineering, architectural constraints, and "garbage collection" against entropy.
- Building effective agents - Anthropic's broader guide to workflows, agents, tools, and when structured systems outperform raw prompting.
- Skill Issue: Harness Engineering for Coding Agents - A practical argument that weak results from coding agents are often harness problems rather than model problems.
- Your Agent Needs a Harness, Not a Framework - Inngest's case for treating state, retries, traces, and concurrency as first-class infrastructure.
- Greenfield AI, Brownfield AI, and the Vibecode You Just Inherited - A three-way taxonomy of codebases agents encounter — agent-native greenfield, true legacy brownfield, and recently-vibecoded inheritance — with playbooks for installing layered
CLAUDE.md rules, ratcheted pre-commit hooks, baselined lint violations, and feature-folder refactors so the codebase itself stops being the harness bottleneck.
- Harness Engineering for Language Agents: The Harness Layer as Control, Agency, and Runtime - A position paper that treats the harness layer as a first-class research object, proposes the control–agency–runtime (CAR) decomposition, and introduces HarnessCard for structured reporting of harness design and evaluation.
- Many Hands Engineering - A handbook framing the layer above the per-agent harness: how multiple harnessed agents share a commons, where decisions belong on a planned / emergent spectrum, and how human stewardship operates at a different cadence than agent execution. Treats harness engineering as a critical layer of "terrain" the framework sits on top of.
- Agent Frameworks, Runtimes, and Harnesses, Oh My! - LangChain's decomposition of what belongs in a framework, a runtime, and a harness.
Context, Memory & Working State
Context Design & Delivery
- Effective context engineering for AI agents - Anthropic's guidance on managing the context window as a working memory budget rather than a dumping ground.
- Context Engineering for AI Agents: Lessons from Building Manus - Manus' detailed playbook on KV-cache locality, tool masking, filesystem memory, and keeping useful failures in-context.
- Context Engineering for Coding Agents - Thoughtworks guidance on shaping the task environment so coding agents can stay grounded and productive.
- Advanced Context Engineering for Coding Agents - HumanLayer patterns for reducing context drift and making coding sessions easier to resume.
- Context-Efficient Backpressure for Coding Agents - HumanLayer's ideas for preventing agents from burning context on noisy or low-value work.
- OpenHands Context Condensensation for More Efficient AI Agents - OpenHands' design for bounded conversation memory that preserves goals, progress, critical files, and failing tests while keeping long-running coding sessions efficient.
- Writing a good CLAUDE.md - A practical guide to creating durable, repo-local instructions that agents can repeatedly follow.
- Deterministic Context Routing - A context-management methodology that routes only the necessary-and-sufficient context to an agent through a deterministic chain (module registry → wiki → session state), cutting overread and context loss on large, under-documented multi-module codebases.
- DevProjex - GUI, TUI, and CLI tooling for selecting and exporting structured codebase context with folder trees, token estimates, ignore rules, and previews.
Memory & Knowledge Systems
- wiki - Indexed Markdown knowledge bases that give agents incremental project context through deterministic indexes, scoped CLI access, and merge handling for parallel edits.
- Data Olympus - Git-native project knowledge base and MCP server with governed proposal-to-acceptance workflows, validity windows, supersession chains, and retrieval of in-force engineering guidance.
- OpenViking - Context database that unifies agent memory, knowledge retrieval, and skills behind an MCP-accessible storage layer.
Constraints, Guardrails & Safe Autonomy
Security, Authorization & Policy
- Beyond permission prompts: making Claude Code more secure and autonomous - Anthropic on reducing approval friction without losing control through better sandboxing and policy design.
- Mitigating Prompt Injection Attacks in Software Agents - OpenHands' practical guide to confirmation mode, analyzers, sandboxing, and hard policies for reducing prompt-injection risk in autonomous coding agents.
- APort Agent Guardrails - Deterministic pre-action authorization hooks for AI-agent tool calls, with adapters for Claude Code, Cursor, OpenClaw, LangChain, CrewAI, and related runtimes.
- Lurkr - Static scanner that runs in CI before deploy to surface AI-agent capability risks, including shadow capabilities, credentials into LLM context, eval/subprocess in
@tool, direct prompt interpolation, and unverified MCP endpoints.
- HEAAL - Grammar-enforced safety constraints for AI agents via AIL (AI-Intent Language).
Operational Autonomy & Reliability
Specs, Agent Files & Workflow Design
- AGENTS.md - A lightweight open format for repo-local instructions that tell agents how to work inside a codebase.
- agent.md - A related standardization effort for machine-readable agent instructions across projects and tools.
Spec-Driven Development
Operating Principles & Human Oversight
Evals & Observability
Evaluation Design
- Testing Agent Skills Systematically with Evals - OpenAI's concrete guide to turning agent traces into repeatable evals with JSONL logs and deterministic checks.
- How to Evaluate Agent Skills (And Why You Should) - OpenHands' hands-on playbook for measuring whether a skill actually helps using bounded tasks, deterministic verifiers, no-skill baselines, and trace review.
- Agent evals - OpenAI's product guide for measuring agent quality with reproducible task-level and workflow-level evaluations.
- Evaluation best practices - OpenAI's general guide to building eval suites that match real-world distributions and catch regressions early.
- Trace grading - OpenAI documentation on grading agent traces directly, which is especially helpful for long multi-step tasks.
- Inspect AI - UK AISI's open-source evaluation framework with solver, scorer, sandboxing, tool-use, MCP, and log-viewer primitives for building reproducible agent eval harnesses.
- Demystifying Evals for AI Agents - Anthropic's guidance on what to measure when agents have many possible trajectories to success or failure.
- Evaluating Deep Agents: Our Learnings - LangChain's practical breakdown of single-step, full-run, and multi-turn eval design for stateful agents.
Verification & Quality Gates
- OpenTelemetry Semantic Conventions for Generative AI Systems - Standard span, metric, event, and attribute conventions for instrumenting LLM and agent workflows so harness traces stay portable across observability backends.
- AgentOps - Open-source Python SDK for agent monitoring, session replay, cost tracking, benchmarking, and tracing across common LLM and agent frameworks.
- agenttrace - Local-first TUI/CLI for auditing AI coding-agent session traces, health gates, cost spikes, tool failures, latency gaps, and attempt-to-attempt diffs.
- flameox - Profiling and optimization toolkit for agents: bounded CLI and MCP workflows capture traces, preserve native evidence, and compare experiments behind runtime conclusions.
- ax - Local-first telemetry and memory graph for auditing coding-agent sessions, costs, skills, tool usage, and OTLP events across multiple agent runtimes.
- Quantifying infrastructure noise in agentic coding evals - Anthropic on how runtime configuration can move coding benchmark scores by more than many leaderboard gaps.
Benchmarks
These benchmarks are especially useful when you want to compare harness quality, not just model quality. They stress context handling, tool calling, environment control, verification logic, and the runtime scaffolding around the model.
Coding & Terminal Agents
A curated list of articles, playbooks, benchmarks, specifications, and open-source projects about harness engineering — shaping the environment around AI agents so they can work reliably.