RustXNG README

The project's README from github.com/sahenjp/RustXNG. As one markdown file: README.md.

RustXNG

A self-hosted, SSRF-hardened web search and grounding engine for local AI agents.

No API keys. No fees. One request returns ranked results and the page text, so an agent can answer from sources instead of guessing.

日本語

CI Nightly benchmark License: AGPL-3.0-or-later

Why this exists

An agent searches, reads, and decides — then does it again. What it needs underneath is not another search UI but a retrieval layer:

What an agent needs What RustXNG does
No keys, no per-seat billing Free, keyless, self-hosted
Stay inside your network Runs on your machine; search terms never leave it
Fast enough to call 50 times a task 0.01 s on a repeated query (cached)
Text to ground on ~32,000 characters from the top 5 pages in one request
Survive one dead source 180 engines in parallel, per-engine suspension and rest
Safe with untrusted URLs Public addresses only (SSRF guard)

Install

1. Prebuilt binary (fastest)

Download the archive for your platform from Releases and put rustxng on your PATH.

curl -fsSL -o rustxng \
  https://github.com/sahenjp/RustXNG/releases/latest/download/rustxng-v0.25.0-x86_64-unknown-linux-musl
chmod +x rustxng && ./rustxng --version

Check the .sha256 published next to each asset before you run it.

2. Cargo

cargo install --git https://github.com/sahenjp/RustXNG.git --tag v0.25.0

This installs two binaries: rustxng (the server) and rustxng-index (the optional local index).

3. Docker

docker run -p 8888:8888 ghcr.io/sahenjp/rustxng:latest

Published from v0.26.0 onwards. To build it yourself instead:

git clone https://github.com/sahenjp/RustXNG.git && cd RustXNG
docker build -t rustxng . && docker run -p 8888:8888 rustxng

Bind your own settings with -v "$PWD/settings.toml:/app/settings.toml:ro".

4. From source

git clone https://github.com/sahenjp/RustXNG.git
cd RustXNG
cargo build --release
./target/release/rustxng

Requires Rust 1.85 or newer. No system libraries beyond a C toolchain.

Quickstart

# start on 127.0.0.1:8888
rustxng &

# search
curl "http://127.0.0.1:8888/search?q=tokio+runtime&format=json"

# search + page text for grounding
curl "http://127.0.0.1:8888/search?q=tokio+runtime&format=json&include_raw_content=1"

Response shape (SearXNG compatible):

{"results": [{"title": "...", "url": "...", "content": "...", "raw_content": "..."}]}

The browser UI is at http://127.0.0.1:8888/.

How it compares

100 questions with known answer pages, measured on the same machine:

RustXNG Exa Tavily
hit@1 (right page first) 79–83 77 34
hit@5 92–96 91 79
MRR 0.853–0.889 0.840 0.514
Cost $0 paid $8/1k
Page text per request ~32,000 chars ~9,000 ~1,200
Repeat query 0.01 s 0.6 s ~1 s
Cold query 0.8–1.6 s 0.6 s ~1 s

Two caveats, stated plainly: the numbers move by a few points between runs, and Google is unavailable in this build (its WML endpoint now returns 403, the HTML endpoint requires JavaScript, and headless browsers are detected). See docs/COMPARISON-EXA.md for a plain-language version.

What you get

Documentation
docs/AI-INFRA.md Why this is agent infrastructure, and why tests and audits matter
docs/COMPARISON-EXA.md The comparison above, explained without jargon
docs/BENCHMARK.md How the numbers were produced, including the variance
docs/COVERAGE.md Every engine, mapped to its SearXNG counterpart
docs/COMPATIBILITY.md Where this matches SearXNG and where it differs
docs/LOCAL-INDEX.md The experimental local index (crawl + BM25)
docs/PORTING.md Internals and configuration
SECURITY.md Vulnerability scan results and known advisories

Contributing

Issues and pull requests are welcome — see CONTRIBUTING.md. The rules are short: run cargo fmt, cargo clippy, cargo test, and keep the nightly benchmark green.

Good first issues live in the tracker; the parsers under src/engines/ are deliberately small and each has a fixture test.

License

AGPL-3.0-or-later. This is a port of SearXNG; attribution and the exact scope of the port are in NOTICE. If you run a modified version as a network service, AGPL section 13 asks you to offer its source to your users.

RustXNG (sahenjp/RustXNG)

metasearch
Searching Database…

707 ·GitHub
RustXNG
What is this?RustXNG is a self-hosted, SSRF-hardened web search and grounding engine built in Rust for local AI agents. One request returns ranked search results plus the extracted page text, so an agent can answer from real sources instead of guessing.
Is this a search UI or something agents call over an API?It is primarily an API backend (a Rust port of SearXNG) meant to be called by AI agents; it returns SearXNG-compatible JSON and also ships a small browser UI for manual searches.
How do I get it running?Grab a prebuilt binary for your platform from the Releases page, install it with cargo, run the Docker image (published from v0.26.0 onward), or build from source.
Do I need any API keys or accounts to use it?No. There are no API keys and no fees since it is fully self-hosted and keyless; building from source only needs Rust 1.85+ and a C toolchain.
Any search engines that don't work right now?Google is unavailable in this build: its WML endpoint returns 403, the HTML endpoint now requires JavaScript, and headless browsers get detected. The optional local index (BM25 crawl) is also still experimental and off by default.
Where do my search terms go?They stay on your machine; RustXNG runs locally so search terms never leave your network, and it only hits public addresses so your server cannot be tricked into reaching internal URLs (SSRF guard).
How does it compare to paid tools like Exa or Tavily?On a 100-question benchmark run on the same machine, RustXNG scored hit@1 79-83 versus Exa's 77 and Tavily's 34, returns about 32,000 characters of page text per request versus Exa's ~9,000 and Tavily's ~1,200, and costs $0 compared to Tavily's $8 per 1,000 queries.
What's inside out of the box?226 engine types ported from SearXNG with 352 settings entries, 180 engines enabled by default running in parallel, plus per-engine timeouts, suspension with backoff, and automatic resting of engines that stop returning useful results.
What is the optional local index for?rustxng-index lets you crawl your own corpus, rank it with BM25, and merge those results into search output; it's experimental and disabled by default.
Why build this instead of just using SearXNG directly?SearXNG is a search UI for humans; RustXNG is purpose-built agent infrastructure — a Rust rewrite focused on speed (0.01s on cached repeat queries), SSRF safety for untrusted URLs, and returning ready-to-ground page text rather than just a results list.
What are similar projects?wigolo, searxng, xerj, 3d-asset-server, agentgateway, Vane, searx, bifrost
Which topics is it in?metasearch, rate-limiting, web-search, search-engine, caching, search, privacy-first, observability, agpl, rag, monitoring, benchmark
Creator
sahenjp
Size
3.1 MB
Stars
707
Updated
CommitNov 02Releasev0.26.0Nov 01
Age
16d
Commits · 90 Days