bifrost README

The project's README from github.com/maximhq/bifrost. As one markdown file: README.md.

Bifrost AI Gateway

maximhq%2Fbifrost | Trendshift

Discord badge codecov Docker Pulls Run In Postman Artifact Hub CodSpeed License

The fastest way to build AI applications that never go down

Bifrost is a high-performance AI gateway that unifies access to 23+ providers (OpenAI, Anthropic, AWS Bedrock, Google Vertex, and more) through a single OpenAI-compatible API. Deploy in seconds with zero configuration and get automatic failover, load balancing, semantic caching, and enterprise-grade features.

Quick Start

Get started

Go from zero to production-ready AI gateway in under a minute.

Step 1: Start Bifrost Gateway

# Install and run locally
npx -y @maximhq/bifrost

# Or use Docker
docker run -p 8080:8080 maximhq/bifrost

Step 2: Configure via Web UI

# Open the built-in web interface
open http://localhost:8080

Step 3: Make your first API call

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello, Bifrost!"}]
  }'

That's it! Your AI gateway is running with a web interface for visual configuration, real-time monitoring, and analytics.

Complete Setup Guides:

Enterprise Deployments

Bifrost supports enterprise-grade, private deployments for teams running production AI systems at scale. In addition to private networking, custom security controls, and governance, enterprise deployments unlock advanced capabilities including adaptive load balancing, clustering, guardrails, MCP gateway, and other features designed for enterprise-grade scale and reliability.

Book a Demo
Book a Demo
Explore enterprise capabilities

Key Features

Core Infrastructure

Advanced Features

Enterprise & Security

Developer Experience

Repository Structure

Bifrost uses a modular architecture for maximum flexibility:

bifrost/
├── npx/                 # NPX script for easy installation
├── core/                # Core functionality and shared components
│   ├── providers/       # Provider-specific implementations (OpenAI, Anthropic, etc.)
│   ├── schemas/         # Interfaces and structs used throughout Bifrost
│   └── bifrost.go       # Main Bifrost implementation
├── framework/           # Framework components for data persistence
│   ├── configstore/     # Configuration storage backends
│   ├── logstore/        # Request logging storage backends
│   └── vectorstore/     # Vector storages
├── transports/          # HTTP gateway and other interface layers
│   └── bifrost-http/    # HTTP transport implementation
├── ui/                  # Web interface for HTTP gateway
├── plugins/             # Extensible plugin system
│   ├── governance/      # Budget management and access control
│   ├── jsonparser/      # JSON parsing and manipulation utilities
│   ├── logging/         # Request logging and analytics
│   ├── maxim/           # Maxim's observability integration
│   ├── mocker/          # Mock responses for testing and development
│   ├── semanticcache/   # Intelligent response caching
│   └── telemetry/       # Monitoring and observability
├── docs/                # Documentation and guides
└── tests/               # Comprehensive test suites

Getting Started Options

Choose the deployment method that fits your needs:

1. Gateway (HTTP API)

Best for: Language-agnostic integration, microservices, and production deployments

# NPX - Get started in 30 seconds
npx -y @maximhq/bifrost

# Docker - Production ready
docker run -p 8080:8080 -v $(pwd)/data:/app/data maximhq/bifrost

Features: Web UI, real-time monitoring, multi-provider management, zero-config startup

Learn More: Gateway Setup Guide

2. Go SDK

Best for: Direct Go integration with maximum performance and control

go get github.com/maximhq/bifrost/core

Features: Native Go APIs, embedded deployment, custom middleware integration

Learn More: Go SDK Guide

3. Drop-in Replacement

Best for: Migrating existing applications with zero code changes

# OpenAI SDK
- base_url = "https://api.openai.com"
+ base_url = "http://localhost:8080/openai"

# Anthropic SDK
- base_url = "https://api.anthropic.com"
+ base_url = "http://localhost:8080/anthropic"

# Google GenAI SDK
- api_endpoint = "https://generativelanguage.googleapis.com"
+ api_endpoint = "http://localhost:8080/genai"

Learn More: Integration Guides

Performance

Bifrost adds virtually zero overhead to your AI requests. In sustained 5,000 RPS benchmarks, the gateway added only 11 µs of overhead per request.

Metric t3.medium t3.xlarge Improvement
Added latency (Bifrost overhead) 59 µs 11 µs -81%
Success rate @ 5k RPS 100% 100% No failed requests
Avg. queue wait time 47 µs 1.67 µs -96%
Avg. request latency (incl. provider) 2.12 s 1.61 s -24%

Key Performance Highlights:

Complete Benchmarks: Performance Analysis

Documentation

Complete Documentation: https://docs.getbifrost.ai

Quick Start

Features

Integrations

Enterprise

Need Help?

Join our Discord for community support and discussions.

Get help with:

Contributing

We welcome contributions of all kinds! See our Contributing Guide for:

For development requirements and build instructions, see our Development Setup Guide.

License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.

Built with ❤️ by Maxim

bifrost (maximhq/bifrost)

Searching Database…

8.9K ·GitHub
Bifrost
What is this?An open-source, high-performance AI gateway that unifies 23+ LLM providers behind a single OpenAI-compatible API, with automatic failover, load balancing, semantic caching, and budget controls built in.
Is this a hosted service I sign up for, or something I run myself?It is software you run yourself: deploy it as a standalone HTTP gateway with a Web UI, embed it in Go apps via the SDK, or point it at existing calls as a drop-in replacement.
How do I get it running?Three steps: start the Bifrost gateway, configure providers in the Web UI, and make your first API call. The README claims zero to production-ready in under a minute with zero configuration.
Will it work with my existing OpenAI code, or do I have to rewrite it?It is a drop-in replacement — you swap OpenAI/Anthropic/GenAI APIs with one line of code. Native integrations exist for the OpenAI, Anthropic, AWS Bedrock, Google GenAI, LiteLLM, and LangChain SDKs with zero code changes.
Is everything open source, or are some features reserved for the enterprise edition?The core gateway is free under Apache 2.0. Enterprise deployments add adaptive load balancing, clustering, guardrails, and the MCP gateway for teams running production AI at scale.
What can the MCP integration do?Model Context Protocol support lets your models call external tools such as the filesystem, web search, and databases. Bifrost can also act as an MCP gateway itself, which the README lists among enterprise capabilities.
Does it cache repeated requests to cut costs?Yes — semantic caching stores responses based on semantic similarity, so similar requests can be served from cache, reducing both cost and latency.
Does it handle images and audio, not just text?Yes. Text, images, and audio are supported, including streaming, all behind a single common interface.
How do I keep an eye on it in production?It ships native Prometheus metrics, distributed tracing, and comprehensive logging, and the Web UI includes real-time monitoring and analytics.
What are similar projects?busbar, gateway, ThinkWatch-Lite, casbin-gateway, litellm, tyk, mlflow, router
Which topics is it in?failover, llm-router, secret-management, llm-observability, mcp-gateway, ai-governance, llm-proxy, model-router, guardrails, rate-limiting, cost-tracking, tracing
Creator
maximhq
Size
1.1 GB
Stars
8.9K
Updated
CommitOct 16Releasecli/v1.0.0Oct 13
Age
1y
Commits · 90 Days