Global

8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems.

5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦

> Breakthrough inference speed for MiniMax H3 Reference-to-Video (Ref2VA / R2VA): Generate a full 5-second, 24 fps video conditioned on reference character images in 72s (warm) / 95s (first run) on a single consumer NVIDIA GeForce RTX 4090 (24GB).

AirLLM dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. 8-Flash-Next (125B) on 6GB, and DeepSeek-V3 (671B) on 12GB. 8-Flash-Next (125B) under 6GB.

Use it for structured workloads with repeated policies, account context, catalog attributes, or correlated fields. 5-9B, through vLLM. It returns binary, categorical or scored decisions with probabilities.

All of those free LLM APIs are available at manifest.build - make reliable agents.

ggml / ops / maintainer PRs%20sort%3Aupdated-desc) / dev stats / lib llama API / llama-server REST API

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. cpp and MLX.

py starts the SGLang server automatically and sends concurrent requests for an image directory or PDF:

A high-throughput and memory-efficient inference and serving engine for LLMs

Most people learn ML frameworks by importing them. We built ours by hand, then made it hit harder.

Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, or just a CPU.

3-Flash (321B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

This repo is a structured, machine-readable directory of every free LLM API — rate limits, context windows, one-click config snippets, and direct API key links. Updated daily.

This browser demo generates talking-head video in real time. Animate an image using uploaded audio or a live microphone, dub a source video while preserving its pose and timing, or connect a chat agent for interactive speech-driven animation.

To download the model weights manually, use the following command:

Qwen3-8B on a real BANKING77 item. Every number is a model output.

Make videos on the computer you already own. FreeVideo runs MiniMax H3 in as little as 8 GB of VRAM and 16 GB of RAM, and adapts its acceleration path to your hardware.

llamafile lets you distribute and run LLMs with a single file.

AI System Design - Learn how to design AI systems built on LLMs, RAG, and AI Agents step by step.

预览地址为 ，简体中文位于根目录，英文与繁體中文分别位于 en/ 和 zh-tw/。生成文件位于 build/，详细说明见网站构建与发布。

Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode.

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

SGLang is a high-performance serving framework for large language models and multimodal models.

md and local files. yaml only adds optional UI metadata (display name, description, default prompt) for the ChatGPT/Codex skills UI, per OpenAI's skill spec — it does not limit the skill to OpenAI agents.

faster-whisper is a reimplementation of OpenAI's Whisper model using CTranslate2, which is a fast inference engine for Transformer models.

Run:ai is the AI workload orchestration platform for Kubernetes. This monorepo contains the control plane services, in-cluster Kubernetes operator/controllers, shared libraries and clients, the installer, and the local dev environment.