Topics / #llm-ops

LLM ops projects on GitHub

Tools for deploying, serving, and monitoring large language model workloads.

As of , the most-starred LLM ops projects on GitHub are kong (44.3K stars), OpenLLM (12.6K stars) and bisheng (12.1K stars). GitHub.Locker tracks 5 LLM ops repositories with 88.7K stars combined. The fastest-growing on October 16 was bisheng, with +6 stars in a day.

5Projects
88.7KStars in total
+12Stars in a day
Browse in the catalog →github.com/topics/llm-ops ↗

Hot right now

Most stars gained in a day · Oct 16

+6

bisheng

dataelement/bisheng · ★ 12.1K

BISHENG is an open-source LLM application devops platform built for enterprise AI, covering GenAI workflow building, RAG, agents, model management, evaluation, SFT, dataset management and observability.

+3

astron-agent

iflytek/astron-agent · ★ 8.9K

An enterprise-grade agentic workflow platform from iFLYTEK: it combines workflow orchestration, model management, AI and MCP tool integration, RPA automation, and team collaboration for building production-ready agent applications.

+2

kong

Kong/kong · ★ 44.3K

Kong is a cloud-native API gateway that also acts as an AI and MCP gateway, handling routing, load balancing, authentication, and traffic management for microservices and LLM/agentic traffic.

+1

tyk

TykTechnologies/tyk · ★ 10.9K

Tyk is an open-source, cloud-native API Gateway that lets you manage REST, GraphQL, TCP, gRPC and MCP (Model Context Protocol) traffic in one place, adding authentication, rate limiting, quotas and analytics to your APIs.

The all-time top

5 projects · by stars

  1. #1 · Kong/kong

    kong

    Kong is a cloud-native API gateway that also acts as an AI and MCP gateway, handling routing, load balancing, authentication, and traffic management for microservices and LLM/agentic traffic.

    44.3K
  2. #2 · bentoml/OpenLLM

    OpenLLM

    OpenLLM lets you run open-source LLMs like Llama, DeepSeek, Mistral and Qwen as OpenAI-compatible API endpoints with a single command, including a built-in chat UI and cloud deployment support.

    12.6K
  3. #3 · dataelement/bisheng

    bisheng

    BISHENG is an open-source LLM application devops platform built for enterprise AI, covering GenAI workflow building, RAG, agents, model management, evaluation, SFT, dataset management and observability.

    12.1K
  1. 4

    tyk

    10.9KTykTechnologies/tyk · Go

    Tyk is an open-source, cloud-native API Gateway that lets you manage REST, GraphQL, TCP, gRPC and MCP (Model Context Protocol) traffic in one place, adding authentication, rate limiting, quotas and analytics to your APIs.

  2. 5

    astron-agent

    8.9Kiflytek/astron-agent · Java

    An enterprise-grade agentic workflow platform from iFLYTEK: it combines workflow orchestration, model management, AI and MCP tool integration, RPA automation, and team collaboration for building production-ready agent applications.

Often tagged together