The self-hosted execution boundary for AI systems.
Control where AI can go, what it may use, what authority it receives,
what it may cost, and what evidence is retained—before it acts.
Busbar sits between AI applications and the destinations they can reach:
Applications and agents → Busbar → Models, MCP tools, and A2A agents
For traffic routed through Busbar, operators can verify, govern, route,
and record model calls, tool use, and agent delegation before they reach
approved destinations.
Busbar is a self-hosted, customer-operated static Rust binary. It keeps
provider credentials at the boundary, preserves supported native SDK
workflows, and puts policy, budgets, routing, resilience, and execution
evidence in one enforcement point.
What this README demonstrates today: Busbar's model execution
plane—native provider protocols, credential boundary, routing,
budgets, resilience, and audit evidence. MCP tool governance and A2A
agent trust use the same execution-boundary model.
Six model wire protocols, first class on both sides: OpenAI, OpenAI
Responses, Anthropic, Gemini, Cohere, and Bedrock Converse.
Same-protocol routes are byte-for-byte identical to calling the provider
directly, because Busbar forwards the original request bytes rather than
re-serializing them. Cross-protocol routes translate modelled fields into
the target provider's native shape.
Self-hosted, always. No hosted service, no signup, and nothing phones
home. Provider credentials stay in your infrastructure, at the
enforcement boundary.
Built for the governed hot path
An enforcement boundary only works if teams can deploy it on every
governed request path. Busbar is designed to make the decision in the
path without becoming the bottleneck.
Busbar 1.5.1, AWS m7g.4xlarge (Graviton3), 4-core pin, measured 2026-08-03. Same-protocol OpenAI cell. Across all 36 cells idle stayed between 7.32 and 7.42 MiB.
A boundary, not just a proxy
Busbar
LiteLLM Py
LiteLLM Rust
Kong
Portkey
Wire protocol pairs served, of 36
36
8
1
4
8
Added latency p99, µs
82
8,221
106
389
3,720
CPU per request at c=8, µs
65
6,775
89
210
1,486
Requests/sec, zero failures
67,837
170
48,354
22,418
855
Time to first token p50, µs
129
9,088
181
105,907
27,908
Idle resident memory
7.3 MiB
1,080 MiB
253 MiB
403 MiB
124 MiB
Container image
Install
Busbar
5.74 MiB, 3 layers
one 12.39 MiB binary, no runtime
LiteLLM
360.77 MiB, 21 layers
558 MiB across 107 packages
Measured 2026-08-03 on an m7g.4xlarge pinned to 4 cores. Image sizes are compressed registry layers for getbusbar/busbar:latest and ghcr.io/berriai/litellm:main-latest, linux/amd64.
Busbar overlaps with AI gateways on protocol support, routing, and
resilience. Its direction is broader: it is the customer-operated
enforcement point for AI execution, beginning with model traffic and
extending to governed tool and agent access.
Three things we will say against ourselves:
LiteLLM's provider catalogue is far larger than ours, and that is a real reason to pick it. The coverage row counts ingress-to-upstream wire protocol pairs, not providers.
LiteLLM Rust is early beta with a deliberately narrow surface, and its overhead is the same class as ours. The difference there is scope, not speed.
Kong is a general API gateway with an LLM plugin bolted on. That is why its streaming row reads 106 ms and ours reads 129 µs.
Every cell is published with its own verdict and reason at onthebench.ai. The harness is open source, so you can disagree with it in public. Busbar vs LiteLLM
Put the boundary in the path
Your application already speaks one of these protocols. Change the base
URL and key; Busbar becomes the point where model destinations,
credentials, routing, budgets, and execution evidence are controlled.
- client = OpenAI(api_key=OPENAI_KEY)
+ client = OpenAI(api_key=BUSBAR_TOKEN, base_url="http://localhost:8080/v1")
# "fast" is a pool you define in config: 80% Claude, 20% GPT, Bedrock on failover
client.chat.completions.create(model="fast", messages=[{"role": "user", "content": "Hi"}])
That request left as OpenAI, may have been served by Anthropic, and came back as OpenAI. If Anthropic had failed before the first byte, Busbar would have moved to the next lane without your client noticing.
The same swap in the other five SDKs
# Anthropic: the SDK appends /v1/messages, so the pool name goes in the base URL
import anthropic
client = anthropic.Anthropic(api_key="busbar-token", base_url="http://localhost:8080/fast")
client.messages.create(model="ignored", max_tokens=1024, messages=[{"role": "user", "content": "Hi"}])
# OpenAI Responses: same client as above, same base URL
client.responses.create(model="fast", input="Hi")
# Gemini
from google import genai
from google.genai import types
client = genai.Client(api_key="busbar-token",
http_options=types.HttpOptions(base_url="http://localhost:8080"))
client.models.generate_content(model="fast", contents="Hi")
# Cohere v2
import cohere
client = cohere.ClientV2(api_key="busbar-token", base_url="http://localhost:8080")
client.chat(model="fast", messages=[{"role": "user", "content": "Hi"}])
# Bedrock Converse: a SigV4 boto3 client, load balanced across non-Bedrock backends
import boto3
client = boto3.client("bedrock-runtime", region_name="us-east-1",
endpoint_url="http://localhost:8080")
client.converse(modelId="fast", messages=[{"role": "user", "content": [{"text": "Hi"}]}])
Every one of these was run against Busbar 1.5.3 while writing this file. Full route and auth reference: Protocols.
Model routing and resilience
Model routing is one execution-plane capability. A pool is a weighted
group of lanes that share a circuit breaker; a lane that fails before
the first byte is replaced mid-request, and the breaker attributes the
fault so a bad key benches one lane instead of tripping a healthy one.
Your client never sees the hop, because the hop happens before the first byte reaches it, even on a streaming request. Once that first byte is out Busbar does not switch providers under your client: it records the fault against that lane and ends the stream with an error event, and the client retries. The state machine, the fault classes and the recovery probe are in Reliability.
Run it
One binary and one YAML file. No interpreter, no database, no sidecar.
curl -fsSL https://getbusbar.com/install.sh | sh # busbar + providers.yaml into ./
cat > config.yaml <<'EOF'
providers:
anthropic: { api_key: { env: ANTHROPIC_KEY } } # the NAME of the env var, never the key
models:
claude: { provider: anthropic, upstream_model: claude-sonnet-4-5 }
pools:
fast:
members:
- { model: claude, weight: 1 }
EOF
export ANTHROPIC_KEY=sk-ant-...
BUSBAR_CONFIG=./config.yaml ./busbar &
curl -s localhost:8080/v1/chat/completions -H 'content-type: application/json' \
-d '{"model":"fast","messages":[{"role":"user","content":"Hello!"}]}'
Or the container, which ships a bootable default config, so one line is the whole thing:
docker run --rm -p 8080:8080 -e ANTHROPIC_KEY -e BUSBAR_ADMIN_TOKEN getbusbar/busbar
busbar --validate parses your config and every provider reference and exits non-zero on anything wrong, with no server, no network and no state, so it belongs in CI. Full walkthrough: Getting started.
Kubernetes
One container, no sidecar, nothing to run beside it. The image is 5.74 MiB compressed and the process idles at 7.3 MiB, both stamped in the comparison below, so it fits a 32Mi request and a 128Mi limit with room to spare.
config: { locked: true } is what lets the root filesystem be read-only: a mutable config needs a writable overlay path and Busbar refuses to boot without one. This Service is cluster-internal and the data plane has no auth chain, so turn on virtual keys before you expose it (Governance).
What Busbar governs
Busbar's model execution plane includes native protocol support,
provider credentials at the boundary, virtual keys, group budgets and
spend tracking, weighted routing, per-lane concurrency caps,
fault-attributed circuit breaking, in-flight failover, native TLS and
mTLS, Prometheus and OTLP telemetry, and per-request audit webhooks.
The broader execution-boundary model spans three planes:
Model control: Native model traffic, routing, budgets, provider
credentials, resilience, and execution evidence.
Tool governance: Caller grants, approved schemas, budget
enforcement, and drift quarantine for MCP tools.
Agent trust: Verification, pinning, re-verification, egress
control, and target-bound credentials for A2A agents.
The SemVer-protected contract is the runtime: the data-plane HTTP
surface and supported wire-protocol contracts do not break inside a
major version. config.yaml is an operator artifact outside that
freeze; changes ship with busbar --migrate-config and a loud
fail-closed boot rather than silent behavior changes.
Single Rust binary, MSRV 1.97, Apache-2.0. Docs at
getbusbar.com, contributor documentation in
docs/.
busbar (GetBusbar/busbar)
Choose a tag and it will appear here. With no tags selected, all catalog projects are shown.
Searching Database…
Choose a tag and it will appear here. With no tags selected, all catalog projects are shown.
What is this?Busbar is a self-hosted execution control plane that sits between AI applications and the AI providers they call, letting operators verify, govern, route, and record every model request, MCP tool call, and A2A agent delegation before it reaches its destination.
Is this a proxy or a framework I build on top of?It is a reverse proxy / gateway, not a framework: a single self-hosted static Rust binary that forwards and translates six native LLM wire protocols without requiring code changes beyond swapping the base URL and key.
How do I get it running?Run the single static binary with one YAML config file, or start the provided Docker container, which ships a bootable default config; `busbar --validate` checks your config and exits non-zero on errors without starting a server.
Do I need a database or any other services alongside it?No, it is a self-contained static binary with no interpreter, database, or sidecar required; the install is just one ~12.39 MiB binary.
Is MCP tool and agent governance already fully built, or just model routing?The core model execution plane (native protocols, routing, budgets, resilience, audit evidence) is what the README demonstrates in detail; MCP tool governance (caller grants, approved schemas, drift quarantine) and A2A agent trust use the same enforcement model but are described more briefly.
Do my provider credentials or traffic ever leave my infrastructure?No, Busbar is always self-hosted with no hosted service, no signup, and nothing phoning home; provider credentials stay inside your own infrastructure at the enforcement boundary.
Which LLM protocols can it translate between?Busbar natively speaks six wire protocols on both sides—OpenAI, OpenAI Responses, Anthropic, Gemini, Cohere, and Bedrock Converse—forwarding same-protocol requests byte-for-byte and translating cross-protocol ones into the target provider's native shape.
How does it perform compared to LiteLLM or Kong?Published benchmarks show Busbar adding only 82μs p99 latency and idling at 7.3 MiB, versus LiteLLM Python's 8,221μs/1,080 MiB and Kong's 105,907μs time-to-first-token on streaming versus Busbar's 129μs.
What happens if a provider fails mid-request?Routes are grouped into weighted 'lanes' sharing a circuit breaker; a lane that fails before the first byte is swapped transparently and mid-stream, but once a stream has started a failure ends it with an error event instead of silently switching providers.
Can I deploy it on Kubernetes without extra infrastructure?Yes, it runs as a single container with no sidecar; the 5.74 MiB image and 7.3 MiB idle footprint fit comfortably inside a 32Mi request / 128Mi limit pod.
Busbar is a self-hosted execution control plane that sits between AI applications and the AI providers they call, letting operators verify, govern, route, and record every model request, MCP tool call, and A2A agent delegation before it reaches its destination.