wigolo README

The project's README from github.com/KnockOutEZ/wigolo. As one markdown file: README.md.

wigolo — the go-to web for your agent

Local-first web intelligence for AI agents — no keys, no cloud, no metered bill.

works with  Claude Code · Cursor · Codex · Gemini CLI · OpenCode · VS Code · Windsurf · Zed · Antigravity
and beyond  LangChain · CrewAI · LlamaIndex · Vercel AI SDK · n8n & self-hosted agents · any MCP client · plain REST

npm npm downloads GitHub stars CI node MCP license status follow on X

wigolo on Trendshift KnockOutEZ%2Fwigolo | Trendshift

Quickstart · Tools · Why wigolo · Sponsors · Benchmark · Docs · Examples · Feedback · FAQ

New features and updates ship steadily. Follow @yourtowhid on X for all of it and new ways to use wigolo, and reach out there for collaborations or feedback · also on LinkedIn


wigolo gives an AI agent one surface for everything web-related: search, fetch, crawl, extract, cache, find-similar, research, and autonomous gather loops. It runs wherever your agent runs — as an MCP server next to your coding agent, as a REST/MCP endpoint on the box where your self-hosted agents live, or embedded through an SDK inside your own app. The core tools need no API keys, nothing it touches leaves ~/.wigolo/, and no bill grows with how much your agent thinks.

wigolo demo — Claude Code answering a live web question through wigolo, no API keys

Quickstart

npx wigolo init                              # set up the local engine — any system
npx wigolo init --agents=claude-code,cursor  # …or set up + wire your day-to-day agents in one command

Requires Node ≥ 20 and ~1.5 GB of free disk on macOS, Linux, or Windows. Bare init sets up the local engine: it downloads the browser engine and on-device models, runs a health check, and reports each component. Adding --agents wires the named agents in the same run, so a coding agent you use daily is ready in one command.

init is unattended by default, so it's safe in scripts and CI, and any setup problem surfaces right here in the per-component report, before your agent's first call. Search, fetch, crawl, extract, cache, and find-similar work with no API key. Check it's healthy anytime:

npx wigolo doctor

To remove everything cleanly, run npx wigolo config --uninstall --yes. You can also paste the installation guide into any AI assistant and let it do the setup; it's written to be self-contained.

Search, fetch, crawl, extract, cache, and find-similar are fully keyless. research, agent, and search format=answer use an LLM to write the synthesized, cited answer. Without one they hand back a raw brief and evidence for your agent to assemble. A free Gemini key turns that into a finished answer:

export WIGOLO_LLM_PROVIDER=gemini
export GEMINI_API_KEY=<free-key>      # grab one at aistudio.google.com/apikey — the free tier is plenty

Any provider works (anthropic · openai · groq), or stay fully local and keyless with WIGOLO_LLM_PROVIDER=ollama (or any OpenAI-compatible URL). Set it in your shell or your agent's MCP env block. Providers, models, and the keyless local-model ladder are in the configuration guide.

What your agent gets back

Every search result is evidence the agent can act on. It carries a verbatim excerpt pinned to its exact position in the source, a citation ID the agent can quote, and a score it can inspect (abridged real shape):

{
  "results": [{
    "title": "Logical replication - PostgreSQL docs",
    "url": "https://www.postgresql.org/docs/current/logical-replication.html",
    "excerpt": "Logical replication is a method of replicating data objects…",
    "citation_id": "src-1",
    "source_span": { "start": 1042, "end": 1305 },          // byte-exact provenance
    "evidence_score": { "final": 0.86, "semantic": 0.91, "lexical": 0.78, "engine_consensus": 3 }
  }],
  "citations": [{ "id": "src-1", "url": "…" }],
  "freshness_signal": { "published": "2026-05-12", "confidence": "high" }
}

Weak results get flagged as junk by wigolo's own scorer. Failed engines are reported and stale cache is labeled, so the agent always knows what it's standing on. Full response contracts per tool are in the tools reference.

Tools

Tool What it does
🔎 search Multi-engine web search (18 direct adapters) with rank fusion, ML reranking, and an explainable per-result score. Pass a query array for parallel breadth. Scope by domain and time range, match an exact phrase, or return image results.
📄 fetch Load one URL through a tiered router that auto-escalates from plain HTTP to a headless browser engine on anti-bot challenges or SPA shells. Clean markdown + metadata + links. Handles PDFs, a single-heading section, authenticated sessions, and page actions (click / type / scroll / screenshot).
🕸️ crawl Multi-page crawl — BFS, DFS, sitemap, or map-only. Per-domain rate limits, robots.txt respect, boilerplate dedup.
🧩 extract Structured data from a page: tables, metadata, JSON-LD, brand identity, named schemas (Article / Recipe / Product / …), or any custom JSON Schema.
💾 cache Query everything already seen — keyword or hybrid semantic. Plus stats, clear, and change detection.
🧲 find_similar Pages similar to a URL or a concept, via 3-way fusion of keyword + semantic + live web.
🧠 research Decompose a question → fan out sub-queries → fetch sources → synthesize a cited report (or a structured brief the host LLM writes from).
🤖 agent Autonomous gather loop: plan → search → fetch → extract → synthesize, with a step log, time budget, and optional output schema.
🔁 diff + ⏱️ watch See exactly what changed on a page since last visit; re-check on demand and deliver changes to a webhook.

Every tool also runs from the terminal (wigolo search "…" --json), from an interactive shell with NDJSON piping (wigolo shell), over REST, and through the SDKs — CLI reference. Per-tool guides with the full parameter set are in docs/tools.md; runnable examples are in examples/.

Why it's different

wigolo isn't a free stand-in for the paid tools — it's built to match them. It's a focused web layer for your agents: an MCP and REST surface they call directly, with the search and extraction quality the paid services charge for. What separates it:

Here's what one real result looks like, dissected. It includes the failed engine and the weak result, because those are part of the answer too:

Anatomy of a wigolo result: explainable score decomposition, live engine telemetry, surfaced degradation, self-flagged junk — one real query, captured live

Sponsors

Thank you to the sponsors below, who help keep wigolo maintained and free for everyone to use. Their support goes straight into the work.

TestMu AI

TestMu AI (formerly LambdaTest) is the world's first full-stack agentic AI quality engineering platform, trusted by 18,000+ enterprises.

wigolo is free for all and is meant to stay that way. If you or your company would like to help keep it maintained, there's room for more sponsors — reach out at [email protected], or see SPONSORS.md for the terms. A one-off via Buy Me a Coffee is welcome too.

Benchmark

All four tools converged on the same core answer, and only one of them handed back verbatim, byte-pinned evidence while doing it.

One cold query ran live inside a single Claude Fable 5 session, fanned out to four web tools on equal footing (built-in WebSearch, wigolo, Tavily, Exa), and was judged by the agent on the evidence alone. All four converged on the same answer and the same top source, so the parity is demonstrated on-screen. wigolo alone returned verbatim excerpts pinned to byte-offset source spans, an explainable score decomposition, and live per-engine telemetry, and its own scorer flagged two weak results as junk. The cloud tools earn their place too: Exa rendered the official docs' comparison matrix in full. Run your own query and you'll see the same shape.

wigolo vs built-in WebSearch, Tavily, and Exa on one real query, driven by Claude Fable 5

How it compares

wigolo Firecrawl Exa Tavily
Multi-engine web search ✅ ✅ ✅ ✅
Fetch & structured extraction ✅ ✅ ✅ ✅
Whole-site crawl & map ✅ ✅ — ✅
Verbatim excerpts pinned to byte-offset source spans ✅ — — —
Explainable per-result score decomposition ✅ — — —
Persistent local memory — re-query instantly, offline ✅ — — —
Query data stays on your machine ✅ — — —
API key / account none required required required
Cost per query $0 metered metered metered

Feature standing as of July 2026 — check each vendor's docs for current state.

That last row compounds, because agents ask in bursts:

The meter: a metered cloud API's cost climbs with every query while wigolo stays flat at zero dollars — illustrative pricing

Beyond your editor

The same ten tools serve every kind of agent, over whichever surface fits: MCP for coding agents, REST for everything else, SDKs to embed, and framework wrappers to drop in.

REST API — wigolo serve

One process exposes a plain-JSON REST API next to the MCP transport. No MCP client needed, just curl:

wigolo serve                          # 127.0.0.1:3333 — loopback is open; off-loopback requires a token

curl -sX POST http://127.0.0.1:3333/v1/search \
  -H 'Content-Type: application/json' \
  -d '{"query":"local-first software","max_results":5}'

POST /v1/{tool} covers all ten tools, GET /openapi.json is the OpenAPI 3.1 contract, and /mcp + /sse serve remote MCP clients from the same port. Bind past loopback and a bearer token is required, so the server fails closed by default. Point n8n, a Hermes-style assistant, or any self-hosted agent at it. → REST API

SDKs — TypeScript & Python

Thin, typed clients with an embedded local mode that finds or starts the daemon for you. No separate serve step.

TypeScript — npm install wigolo-sdk (zero-dep; Node / Bun / Deno / edge):

import { createLocalClient } from 'wigolo-sdk/local';

const { client, close } = await createLocalClient();   // reuse a running daemon, or spawn one
const res = await client.search({ query: 'local-first web search', max_results: 5 });
console.log(res.results.map((r) => r.title));
await close();                                          // stops the daemon only if this call spawned it

Python — pip install wigolo (standard library only; sync + async):

from wigolo import local_client

with local_client() as client:                          # reuse a healthy daemon, or spawn one
    res = client.search(query="local-first web search", max_results=5)
    for r in res["results"]:
        print(r["title"], r["url"])

→ SDKs & embedded mode

Framework integrations

Drop wigolo's tools into the framework you already use. You get the full ten-tool surface, including the cache / find_similar / research / agent that most framework web-tools don't ship:

Framework Package What you get
LangChain wigolo-langchain each tool as a BaseTool, plus a BaseRetriever over search / find_similar for RAG
CrewAI wigolo-crewai wigolo_tools() → hand the set to any crew
LlamaIndex wigolo-llamaindex a BaseReader that loads fetched / crawled / searched pages as documents
Vercel AI SDK wigolo-vercel-ai-sdk tool factories for generateText / streamText, edge-friendly

→ Framework integrations

Docker

# stdio MCP — wire it into any MCP client as command: docker
docker run -i --rm -v wigolo-data:/data ghcr.io/knockoutez/wigolo

# HTTP server for remote / multi-client use
docker run -p 3333:3333 -v wigolo-data:/data \
  -e WIGOLO_API_TOKEN=a-long-random-secret \
  ghcr.io/knockoutez/wigolo serve --host 0.0.0.0

The slim image lazy-loads models into the volume; :full preinstalls the browser engine. Also on Docker Hub as towhid69420/wigolo. → installation & all channels

Agent skills

An 11-pack skill catalog teaches your coding agent to drive each tool well. It's installed by init and managed with wigolo skills add|list|remove. → skills

One note for self-hosters: some challenge-protected sites score IP reputation, so a datacenter IP won't clear walls a home connection would. wigolo labels those failures, and the self-hosting guide covers the opt-in proxy answer.

Star history

wigolo GitHub stars over time

Refreshed daily from the GitHub API. Add a ⭐ if wigolo is useful to you.

Architecture

A single Node process speaks MCP (JSON-RPC over stdio). Everything heavy is local and lazy-loaded, so a zero-key install pays nothing for the parts it isn't using.

flowchart TD
    A["🤖 AI agent<br/>any MCP client · REST · SDK"]
    A -->|MCP over stdio| B["<b>wigolo</b><br/>10 tools · dynamic instructions<br/>in-process browser pool + cache + models"]

    B --> C{"Tool layer"}
    C --> T1["search · fetch · crawl · extract"]
    C --> T2["cache · find_similar · research · agent"]

    T1 --> F["⚙️ Fetch router<br/>tiered escalation, learned per domain"]
    T1 --> S["⚙️ Search<br/>18 engines → rank fusion → ML rerank<br/><i>explainable evidence score</i>"]
    T2 --> DB[("🗄️ Local cache<br/>keyword + vector index")]
    T2 --> ML["🧠 On-device ML<br/>embeddings + reranker"]

    F -.->|optional| LLM["☁️ LLM<br/>synthesis only · opt-in"]
    S -.->|optional| SX["🔀 Aggregator backend<br/>opt-in legacy / hybrid"]

    F --> WEB["🌍 Public web"]
    S --> WEB

    style B fill:#7c3aed,stroke:#5b21b6,color:#fff
    style WEB fill:#0ea5e9,stroke:#0369a1,color:#fff
    style DB fill:#1e293b,stroke:#334155,color:#fff
    style LLM stroke-dasharray: 5 5
    style SX stroke-dasharray: 5 5

wigolo (KnockOutEZ/wigolo)

Searching Database…

5.5K ·GitHub
这是什么?wigolo 是一套本地优先、面向 AI 编程智能体的网页工具包:把搜索、抓取、爬取、内容提取、缓存、相似查找和研究功能整合到一个界面里,通过 MCP、REST、SDK 和命令行工具提供。不需要 API 密钥,不用云端,每次查询 0 成本。
它是运行在我自己的电脑上,还是需要注册的云服务?它在本地以单个 Node 进程运行:可以作为编程智能体旁边的 MCP 服务器,作为自托管智能体所在机器上的 REST/MCP 端点,或者作为嵌入你自己应用里的 SDK。缓存、模型和配置全部保存在 ~/.wigolo/ 目录下。
我怎么把它接到我的编程智能体上?运行 npx -y wigolo init。单独运行 init 会下载浏览器引擎和本地模型,并对每个组件做健康检查;加上 --agents claude-code,cursor(或任意支持的工具)则会在同一次运行中为这些智能体写好 MCP 配置。
运行之前我需要先装什么吗?需要 Node 20 或更新版本,以及大约 1.5 GB 的可用磁盘空间,macOS、Linux 或 Windows 都可以。安装过程全自动,某个组件下载失败也不会让 init 整体失败——它会报告缺了什么,并给出确切的解决方法。
不用 API 密钥真的什么都能用吗?核心工具(搜索、抓取、爬取、提取、缓存、相似查找)完全不需要密钥。research、agent 和 answer-format 搜索需要用到 LLM:没有密钥时,它们会返回一份原始简报加证据材料;你也可以添加一个免费的 Gemini 密钥、任意服务商的密钥,或者用 Ollama 保持全程本地运行。
我的智能体搜索的内容会不会离开我的电脑?不会。缓存、嵌入向量、模型和配置都保存在 ~/.wigolo/ 目录下,除非你明确选择让某个 LLM 来做综合处理,否则不会有任何数据发往第三方。
我的智能体能用到哪些工具?共十个:search(18 个直接搜索引擎,带排序融合和机器学习重排)、fetch、crawl、extract、cache、findsimilar、research、agent,外加用于跟踪网页变化并通过 webhook 推送的 diff 和 watch。每个工具也都能在终端、REST 或 SDK 里使用。
和 Tavily、Exa 这类付费工具比起来怎么样?按照它自己的对比表,它在搜索、抓取、提取和爬取功能上与这些工具相当,但不需要注册账号,不需要 API 密钥,每次查询 0 成本。它还能返回与字节偏移量源位置精确对应的原文摘录、每个结果可解释的评分,以及让重复查询即时且免费的本地缓存。
如果网站拦截机器人会怎样?抓取路由会从普通 HTTP 逐级升级,遇到反爬验证或 SPA 外壳时改用无头浏览器。如果仍然被拦截——数据中心 IP 常常过不了 IP 信誉检查——你会收到一个明确标记为 blockedbychallenge 的失败结果,绝不会把验证页面伪装成内容返回给你。
不用 MCP 客户端也能用吗?可以。wigolo serve 提供纯 JSON 的 REST API(全部十个工具都用 POST /v1/{tool},附 OpenAPI 3.1 规范),还有支持嵌入本地模式的 TypeScript 和 Python SDK、面向 LangChain、CrewAI、LlamaIndex 和 Vercel AI SDK 的框架集成包,以及带交互式 shell 的命令行工具。
Creator
KnockOutEZ
Size
42.3 MB
Stars
5.5K
Age
6mo
Commits · 90 Days