Topics / #llmops

Llmops projects on GitHub

Operationalizing LLM apps: gatewaying, tracing, and cost management.

As of , the most-starred Llmops projects on GitHub are litellm (61.4K stars), serviette (58.8K stars) and Made-With-ML (49.8K stars). GitHub.Locker tracks 32 Llmops repositories with 563.4K stars combined. The fastest-growing on October 16 was Soup, with +79 stars in a day.

32Projects
563.4KStars in total
+437Stars in a day
Browse in the catalog →github.com/topics/llmops ↗

Hot right now

Most stars gained in a day · Oct 16

+79

Soup

MakazhanAlpamys/Soup · ★ 9.2K

Soup is a command-line tool for fine-tuning and post-training LLMs from a single YAML config file, automating GPU detection, batching and quantization so you just write a config and run one command.

+64

litellm

BerriAI/litellm · ★ 61.4K

LiteLLM is an open source AI Gateway that gives you one unified interface to call 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, and more) using the OpenAI API format.

+54

langfuse

langfuse/langfuse · ★ 35.9K

Langfuse is an open source LLM engineering platform for tracing, evaluating, and debugging LLM and agent applications, with prompt management and a playground built in.

+45

promptfoo

promptfoo/promptfoo · ★ 26.1K

promptfoo is a CLI and library for evaluating and red-teaming LLM apps — you test prompts, agents, and RAGs and run security scans on them, instead of relying on trial-and-error.

The all-time top

32 projects · by stars

  1. #1 · BerriAI/litellm

    litellm

    LiteLLM is an open source AI Gateway that gives you one unified interface to call 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, and more) using the OpenAI API format.

    61.4K
  2. #2 · pathwaycom/serviette

    serviette

    This repo provides ready-to-run cloud templates for RAG pipelines, AI pipelines, and enterprise search, built on the Pathway Live Data Framework and kept in sync with live data sources.

    58.8K
  3. #3 · GokuMohandas/Made-With-ML

    Made-With-ML

    Made-With-ML is a free hands-on course that teaches how to design, develop, deploy, and iterate on production-grade machine learning applications, combining ML with software engineering best practices.

    49.8K
  1. 4

    langfuse

    35.9Klangfuse/langfuse · TypeScript

    Langfuse is an open source LLM engineering platform for tracing, evaluating, and debugging LLM and agent applications, with prompt management and a playground built in.

  2. 5

    composio

    30.6KComposioHQ/composio · TypeScript

    Composio is a platform for AI agents: it gives your agents 1000+ pre-authenticated toolkits, per-user sessions, auth, triggers, and a sandbox, so they can take action across apps instead of just chatting.

  3. 6

    mlflow

    28.4Kmlflow/mlflow · Python

    MLflow is an open source AI engineering platform for agents, LLM apps, and classic ML models: one place to trace, evaluate, monitor, and optimize production AI while controlling costs. It is widely adopted, with over 60 million monthly downloads.

  4. 7

    promptfoo

    26.1Kpromptfoo/promptfoo · TypeScript

    promptfoo is a CLI and library for evaluating and red-teaming LLM apps — you test prompts, agents, and RAGs and run security scans on them, instead of relying on trial-and-error.

  5. 8

    llm-action

    25.2Kliguodongiot/llm-action · HTML

    It is a Chinese-language hub of in-depth tutorials on large language model engineering: training, fine-tuning, distributed training, inference, compression, evaluation, data engineering, LLMOps and AI infrastructure, organized as a big index linking the author's article series, many with companion code.

  6. 9

    opik

    22.7Kcomet-ml/opik · Python

    Opik is an open-source platform for debugging, evaluating, and monitoring LLM applications, RAG systems, and agentic workflows, with tracing, automated evaluation, and production dashboards.

  7. 10

    awesome-ai-engineering

    22.4Kdipakkr/awesome-ai-engineering

    A large curated list of AI engineering resources covering LLMs, RAG, AI Agents, MCP, fine-tuning, evaluations, and production AI systems, plus broader learning resources for students (programming languages, hackathons, cybersecurity, blockchain, and more).

  8. 11

    serve

    21.9Kjina-ai/serve · Python

    Jina is an open-source Python framework for building multimodal AI applications on a cloud-native stack — you wrap models into gRPC microservices, chain them into flows, and scale from local to cloud.

  9. 12

    deepeval

    18.9Kconfident-ai/deepeval · Python

    DeepEval is an open-source LLM evaluation framework: a simple-to-use way to evaluate large-language model systems with research-backed metrics like G-Eval, answer relevancy, and hallucination checks.

  10. 13

    SuperAGI

    17.7KTransformerOptimus/SuperAGI · Python

    SuperAGI is a dev-first open source framework for building, managing and running autonomous AI agents — agents that perform tasks and improve with each run.

  11. 14

    RagaAI-Catalyst

    16.2Kraga-ai-hub/RagaAI-Catalyst · Python

    RagaAI Catalyst is a platform for managing and optimizing LLM projects — tracing, evaluating and safeguarding LLM and agent applications, shipped as a Python SDK.

  12. 15

    ragas

    16Kvibrantlabsai/ragas · Python

    Ragas is an open-source Python toolkit for evaluating and optimizing LLM applications: objective metrics, automatic test-set generation and data-driven insights replace slow, subjective manual assessment.

  13. 16

    gateway

    13.2KPortkey-AI/gateway · TypeScript

    An open-source AI gateway that routes your requests to 1,600+ language, vision, audio and image models across 45+ providers through one fast, friendly API, with guardrails, caching, retries and fallbacks built in.

  14. 17

    OpenLLM

    12.6Kbentoml/OpenLLM · Python

    OpenLLM lets you run open-source LLMs like Llama, DeepSeek, Mistral and Qwen as OpenAI-compatible API endpoints with a single command, including a built-in chat UI and cloud deployment support.

  15. 18

    bisheng

    12.1Kdataelement/bisheng · Python

    BISHENG is an open-source LLM application devops platform built for enterprise AI, covering GenAI workflow building, RAG, agents, model management, evaluation, SFT, dataset management and observability.

  16. 19

    phoenix

    11.9KArize-ai/phoenix · Python

    Arize Phoenix is an open-source AI observability platform for tracing, evaluating, and troubleshooting LLM applications, agents, and RAG pipelines.

  17. 20

    tensorzero

    11.7Ktensorzero/tensorzero · Rust

    TensorZero is an open-source LLMOps platform that combines an LLM gateway, observability, evaluation, optimization, and experimentation tooling into one self-hosted stack.

  18. 21

    promptflow

    11.3Kmicrosoft/promptflow · Python

    Prompt flow is a suite of development tools from Microsoft for building LLM-based AI applications end-to-end: prototyping, prompt tuning, testing, evaluation, and production deployment and monitoring.

  19. 22

    metaflow

    10.3KNetflix/metaflow · Python

    Metaflow is an open-source Python framework for building, managing and deploying real-life AI and ML systems — from quick prototyping in notebooks to reliable production deployments.

  20. 23

    Soup

    9.2KMakazhanAlpamys/Soup · Python

    Soup is a command-line tool for fine-tuning and post-training LLMs from a single YAML config file, automating GPU detection, batching and quantization so you just write a config and run one command.

  21. 24

    bifrost

    8.9Kmaximhq/bifrost · Go

    An open-source, high-performance AI gateway that unifies 23+ LLM providers behind a single OpenAI-compatible API, with automatic failover, load balancing, semantic caching, and budget controls built in.

  22. 25

    maths-cs-ai-compendium

    7.7KHenryNdubuaku/maths-cs-ai-compendium · TypeScript

    An open, unconventional online textbook covering maths, computing and AI from the ground up — intuition first, real-world context, for practitioners who want deep understanding rather than exam survival.

  23. 26

    os

    821iblai/os · TypeScript

    ibl.ai/os is an open-source AI agent platform for building, deploying, and managing intelligent conversational agents, from prototype to production, with support for any LLM provider.

  24. 27

    ai-engineer-notebooks

    700calmrocks/ai-engineer-notebooks · Jupyter Notebook

    A collection of framework-free, hands-on Jupyter/Colab notebooks teaching the applied-LLM engineer skill set — model APIs, RAG, evals, agents, fine-tuning, security, and LLMOps — built for engineers moving into AI Engineer / Forward Deployed Engineer (FDE) roles.

  25. 28

    overmind

    662overmind-core/overmind · Python

    Overmind points at your AI agent's codebase and production traces, then turns them into a fine-tuned model that is benchmarked against metrics you define and served through one unified API — no separate ML infrastructure to build.

  26. 29

    busbar

    194GetBusbar/busbar · Rust

    Busbar is a self-hosted execution control plane that sits between AI applications and the AI providers they call, letting operators verify, govern, route, and record every model request, MCP tool call, and A2A agent delegation before it reaches its destination.

  27. 30

    ZizkaDB

    145ZIZKA-AI-SL/ZizkaDB · Python

    ZizkaDB is an audit trail database built for AI agents. It logs every agent decision with a SHA-256 checksum and a causal link back to what caused it, so you can replay sessions and time-travel to see exactly what the agent knew at any past moment, supporting EU AI Act Article 12 record-keeping.

  28. 31

    agento11y

    129grafana/agento11y · Go

    Grafana Agent Observability monitors AI agents — the ones you build and the coding agents you use. This repo holds the open-source SDKs and plugins that send telemetry to it.

  29. 32

    apowerb

    121apowerb/apowerb · Python

    apowerb is an open-source platform for building, running and governing AI agents in production, covering multi-agent orchestration, RAG, Text-to-SQL, MCP servers and built-in business tools.

Often tagged together