The project's README from github.com/decodingai-magazine/building-a-coding-agent-from-scratch-course. As one markdown file: README.md.
Open-source harness engineering course
by Decoding AI in collaboration with Modal, Opik (by Comet), and Kitaru (by ZenML).
https://github.com/user-attachments/assets/f8711e5b-05e2-4c31-aa7f-355374c332d8
📖 Read Lesson 1 (17 min) · 🎬 Watch Lesson 1 · 📚 See all 8 lessons
Try the finished agent first — 5 minutes, $0:
git clone https://github.com/decodingai-magazine/building-a-coding-agent-from-scratch-course.git cd building-a-coding-agent-from-scratch-course make install cp .env.example .env # set LLM API key uv run decodeThen type
/demo-and pick a demo. See what they do below. Full setup guide.
Type /demo- and the six demos are one keystroke away.
In LangChain's Terminal-Bench experiment, changing only the harness (with the same model) moved a coding agent from ~30th place into the top 5: the harness, not the model, is what makes a coding agent good.
agent = Agent(
build_model(settings.llm_provider), # gemini | openrouter | modal
deps_type=AgentDeps, # cwd, event sink, permission gate
output_type=[str, DeferredToolRequests], # final answer, or tools paused for approval
)
register_tools(agent) # read, edit, bash, grep, ...
async with agent.iter(prompt, message_history=history) as run:
async for node in run: # model request → tool calls → repeat
stream_events(node)
That's the entire tool-calling agent. Everything else in this repo (the tools, skills, permission layer, sandbox, steering queue, memory, compaction, session recording & replay, remote execution, subagent fan-out, and evals) is the harness. That's what you're here to build.
A fresh session powered by Qwen3.6-35B hosted on Modal
We spent months under the hood of Claude Code (via its leaked source), OpenCode, Pi, and Aider, then distilled what we learned into 8 articles and 6 videos where you'll build decode, your own coding agent, from scratch. One headless core hooked to two modes: an interactive TUI and Modal serverless functions running N copies in parallel, fired by CLI, webhook, or cron.
[!WARNING] Building a coding harness from scratch can make you dangerously good at building any other AI product, whether in finance, medicine, or e-commerce.
The harness architecture of the coding agent you will build during this course.
The finished agent ships with demo skills under .decode/skills/. Open the TUI, type /demo-, pick one, and watch the harness you're about to build do real work:
Implement the Skills Standard
Type /demo- and the six demos are one keystroke away.
Capable of Creating Games |
Fetching Data & Creating Dashboards |
Extracting Ontologies & Rendering Graphs |
And the infra that powers the agents.
Record & Replay for AI Agents |
Remote Sandboxing |
Powered by Open-Source Models |
Adding AI Evals & Observability |
Plan mode, live: the agent breaks the Snake demo into a task list with the todo tool — [x] done, [~] in progress.
The code is written in Python, with the following frameworks and libraries:
Otherwise, we build all the functionality from scratch to teach you the foundations that last, not frameworks that abstract away the hard parts.
Join 45k+ engineers subscribed to the Decoding AI Magazine to learn to build coding agents from scratch.
For the full experience, go through the articles and videos that cover what the code can't. The why behind every decision.
| Lesson | Written Lesson | Video Lesson | Running the Code |
|---|---|---|---|
| 1 Building a Coding Agent From Scratch Sketch the full harness: a headless core, six modules, TUI and remote modes, and an evals layer. ⏱ 17-min read |
Article 1 |
 Video 1 |
01_install_and_usage.md · 02_modal_endpoints.md |
| 2 The Bare-Bones Coding Agent Loop Your agent loops over 9 tools and 3 swappable LLM providers in a steerable TUI. ⏱ 25-min read |
 Article 2 |
01_install_and_usage.md · 02_modal_endpoints.md | |
| 3 From a Raw Shell to a Sandboxed Coding Agent Your agent's tools run in a Docker or Modal sandbox, never on your host. ⏱ 13-min read |
 Article 3 |
 Video 2 |
01_install_and_usage.md · 02_modal_endpoints.md · 03_sandboxing.md |
| 4 Context Engineering for Coding Agents Your agent loads memory, invokes skills, reads LSP diagnostics, and auto-compacts at 80%. ⏱ 15-min read |
 Article 4 |
 Video 3 |
01_install_and_usage.md · 02_modal_endpoints.md |
| 5 Subagents Are Context Engineering Your agent fans out parallel Explore subagents and switches between build, plan, and code-reviewer personas. ⏱ 14-min read |
 Article 5 |
- | 01_install_and_usage.md · 02_modal_endpoints.md |
| 6 Swarm of Remote Agents Your agent runs headless on Modal, fired by the CLI, a webhook, or a cron job, and ships branches. ⏱ 15-min read |
 Article 6 |
 Video 4 |
01_install_and_usage.md · 02_modal_endpoints.md · 03_sandboxing.md · 04_deploy.md |
| 7 Agent Evals 101 Score your agent on a 19-task benchmark and a regression suite in Opik. ⏱ 19-min read |
 Article 7 |
 Video 5 |
01_install_and_usage.md · 02_modal_endpoints.md · 05_evals.md |
| 8 Turn Your Agent’s Failures Into a Regression Suite Record runs in Kitaru, cluster failures into cohorts, and replay them to verify fixes. ⏱ 16-min read |
 Article 8 |
 Video 6 |
01_install_and_usage.md · 02_modal_endpoints.md · 06_evals_replays.md |
📖 Read all the articles on Substack · 🎬 Watch all the videos on YouTube
Enjoyed building Claude Code from scratch? Keep going with CodeCrafters, where you can rebuild real developer tools like Redis, Git, a Shell, or an HTTP Server through step-by-step challenges with automated tests, in your language of choice. Signing up is free, and my affiliate link gets you 40% off if you upgrade. Or start with their free build-your-own-x guides.
Engineers who learn by building. You finish with a working coding agent that teaches you harness engineering patterns to steal for your own agentic applications.
Best for ML/AI engineers who want to level up their craft and for software engineers and data scientists who want to transition into building agentic systems from scratch.
| Category | Requirements |
|---|---|
| Skills | - Python (Intermediate) - LLMs & agents (Beginner) |
| Hardware | Any modern machine will do. No GPU required, as we run all the LLMs in the cloud. |
| Level | Intermediate (but with a little sweat and patience, anyone can do it) |
| Time | ~4–8 hours for the whole course — 4 if you read and watch, 6–8 if you run everything. |
Running the code costs $0 if you stick to free tiers:
| Service | Cost |
|---|---|
| Gemini API (default provider — easy setup, but limited API requests) | free tier (Google AI Studio) |
| Modal (recommended provider, remote sandbox, remote agents) | $30 free credits — enough to run the course |
| OpenRouter (alternative provider) | $0 on :free models (optional $10 credit raises the daily cap) |
| Opik (tracing + evals) | free tier |
| Kitaru (recording + replays) | free — the OSS server runs on your laptop (make kitaru-local) |
Only reading? Everything's free!
As an open-source course, it is entirely self-paced and based on this repository, plus the attached lessons that walk you through the code. No paywall. No platform.
Read the lessons on the Decoding AI Magazine, watch the videos on the Decoding AI Channel, run the code on your own machine, break it, fix it, and learn from the process.
One Python package; each module maps to one part of the architecture:
.
├── docs/
│ ├── adr/ # Architecture Decision Records — the "why" of every choice
│ └── glossary.md # one canonical name per concept
├── evals/ # benchmark tasks + regression cases + the Opik harness
├── scripts/ # operator scripts: Kitaru bootstrap + eval table generation
├── tests/{unit,integration}/ # mirrors src/ 1:1; milestone capstones prove each milestone
└── src/decode/
├── cli.py # Click entrypoint → launches the TUI
├── tui/ # input: prompt_toolkit · output: Rich
├── harness/ # message queue + priority gate around the loop
├── agent/ # the Pydantic-AI ReAct loop (LLM ⇄ tools)
├── agents/ # agents catalog: Build / Plan / Code-Reviewer + Explore subagent
├── tools/ # file I/O, bash, web, todo, skills dispatch, LSP, ask_user
├── permissions/ # allow/ask/deny · modes · settings.json
├── sandbox/ # bash + file tools seam: none (host) / docker / modal
├── services/lsp/ # hand-rolled stdio LSP client (ty)
├── runtime/ # plain headless decode run + the Recording Seam
├── remote/ # decode remote deploy|run|attempts|logs — the Modal Headless App (CLI, webhook, cron)
├── context/ # compaction + conversation log (JSONL)
├── memory/ # AGENTS.md / MEMORY.md loading + write-back
├── observability/ # Opik tracing
└── config/, entities/ # settings singleton · shared models
The guides live under running_the_code/. Follow them in order; each ends with a link to the next:
| Guide | What's inside |
|---|---|
| 00_troubleshooting.md | Every known failure, and its fix |
| 01_install_and_usage.md | Start here: install, one key, first session |
| 02_modal_endpoints.md | Serving open models on Modal |
| 03_sandboxing.md | Docker (local) / Modal (remote) sandboxing + the sandbox git token |
| 04_deploy.md | The headless harness on Modal — CLI, webhook, cron |
| 05_evals.md | Benchmarks and regression cases on Opik |
| 06_evals_replays.md | Kitaru on your laptop: record, replay & the full evals loop |
Special thanks to Modal, Opik (by Comet), and Kitaru (by ZenML) for sponsoring this open-source course and keeping it free!
Opik and Kitaru are open source. Consider starring their repositories: Opik on GitHub · Kitaru on GitHub.
Open a GitHub issue for course questions, setup trouble, or concept clarifications. Known gotchas are documented in the running_the_code/ guides.
Do I need a paid API key?
No. The default Gemini provider has a free tier, OpenRouter routes across :free models, and Modal gives $30 in credits — see Cost Structure.
Why Python and not TypeScript or Go? Accessibility: our audience knows Python. The course focuses on the design decisions, which transfer to any language.
Why build from scratch instead of extending Pi, DeepAgents, or an existing harness? Because adding custom logic to an existing harness is the easy part. Knowing what to add requires understanding the internals. Those fundamentals are what still make AI engineers valuable. Build a coding agent once and you're equipped to build a custom agent for any use case.
Found a bug and know the fix? Fork, fix, run make ci (no API key needed), and open a pull request. Future readers will thank you 🤗
Paul Iusztin |
Senior AI Engineer, Educator & Founder of Decoding AI. Author of the best-selling LLM Engineer's Handbook. |
Join 45k+ engineers subscribed to the Decoding AI Magazine to learn to build coding agents from scratch.
If you found this course useful, consider starring the repository so others can find it too.
Released under Apache-2.0. Clone, fork, and build on it; keep the LICENSE and credit this repo.
Learn harness engineering by building Claude Code from scratch. Free, open-source course: 8 articles, 6 videos, 1 codebase.