mlflow README

The project's README from github.com/mlflow/mlflow. As one markdown file: README.md.

MLflow logo

The Open Source AI Engineering Platform for Agents, LLMs & Models

MLflow is the largest open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data. With over 60 million monthly downloads, thousands of organizations rely on MLflow each day to ship AI to production with confidence.

MLflow's comprehensive feature set for agents and LLM applications includes production-grade observability, evaluation, prompt management, prompt optimization and an AI Gateway for managing costs and model access. Learn more at MLflow for LLMs and Agents.

Python SDK PyPI Downloads License follow on X(Twitter) follow on LinkedIn Ask DeepWiki

Website · Try Demo · Docs · News · Events

Get Started in 3 Simple Steps

From zero to full-stack LLMOps in minutes. No complex setup or major code changes required. Get Started →

Fastest start — set up tracing with our CLI

uvx mlflow@latest agent setup

One command installs the MLflow skills and launches your coding agent of choice to add tracing to your app. Prefer to wire it up yourself? Follow the three steps below.

1. Start MLflow Server

uvx mlflow server

2. Enable Logging

import mlflow

mlflow.set_tracking_uri("http://localhost:5000")
mlflow.openai.autolog()

3. Run Your Code

from openai import OpenAI

client = OpenAI()
client.responses.create(
    model="gpt-5.4-mini",
    input="Hello!",
)

Explore traces and metrics in the MLflow UI at http://localhost:5000.

LLMs & Agents

MLflow provides everything you need to build, debug, evaluate, and deploy production-quality LLM applications and AI agents. Supports Python, TypeScript/JavaScript, Java and any other programming language. MLflow also natively integrates with OpenTelemetry and MCP.

Observability

Observability

Capture complete traces of your LLM applications and agents for deep behavioral insights. Built on OpenTelemetry, supporting any LLM provider and agent framework. Monitor production quality, costs, and safety.

Getting Started →
Try Demo →

Evaluation

Evaluation

Run systematic evaluations, track quality metrics over time, and catch regressions before they reach production. Choose from 50+ built-in metrics and LLM judges, or define your own.

Getting Started →
Try Demo →

Prompts & Optimization

Prompts & Optimization

Version, test, and deploy prompts with full lineage tracking. Automatically optimize prompts with state-of-the-art algorithms to improve performance.

Getting Started →
Try Demo →

AI Gateway

AI Gateway

Unified API gateway for all LLM providers. Route requests, manage rate limits, handle fallbacks, and control costs through an OpenAI-compatible interface with built-in credential management, guardrails and traffic splitting for A/B testing.

Getting Started →

Model Training

For machine learning and deep learning model development, MLflow provides a full suite of tools to manage the ML lifecycle:

Learn more at MLflow for Model Training.

Integrations

MLflow supports all agent frameworks, LLM providers, tools, and programming languages. We offer one-line automatic tracing for more than 60 frameworks. See the full integrations list.

OpenTelemetry


OpenTelemetry

Agent Frameworks (Python)


LangChain

LangGraph

OpenAI Agent

DSPy

PydanticAI

Google ADK

Microsoft Agent

CrewAI

LlamaIndex

AutoGen

Strands

LiveKit Agents

Agno

Bedrock AgentCore

Smolagents

Semantic Kernel

DeepAgent

AG2

Haystack

Koog

txtai

Pipecat

Watsonx

Agent Frameworks (TypeScript)


LangChain

LangGraph

Vercel AI SDK

Mastra

VoltAgent

Agent Frameworks (Java)


Spring AI

Quarkus LangChain4j

Model Providers


OpenAI

Anthropic

Databricks

Gemini

Amazon Bedrock

LiteLLM

Mistral

xAI / Grok

Ollama

Groq

DeepSeek

Qwen

Moonshot AI

Cohere

BytePlus

Novita AI

FireworksAI

Together AI

Gateways


Databricks

LiteLLM Proxy

Vercel AI Gateway

OpenRouter

Portkey

Helicone

Kong AI Gateway

PydanticAI Gateway

TrueFoundry

Tools & No-Code


Instructor

Claude Code

Opencode

Langfuse

Arize / Phoenix

Goose

Langflow

Hosting MLflow

MLflow can be used in a variety of environments, including your local environment, on-premises clusters, cloud platforms, and managed services. Being an open-source platform, MLflow is vendor-neutral — whether you're building AI agents, LLM applications, or ML models, you have access to MLflow's core capabilities.


Databricks

Amazon SageMaker

Azure ML

Nebius

Red Hat OpenShift AI

Self-Hosted

💭 Support

🤝 Contributing

We happily welcome contributions to MLflow!

Please see our contribution guide to learn more about contributing to MLflow.

⭐️ Star History

Star History Chart

✏️ Citation

If you use MLflow in your research, please cite it using the "Cite this repository" button at the top of the GitHub repository page, which will provide you with citation formats including APA and BibTeX.

👥 Core Members

MLflow is currently maintained by the following core members with significant contributions from hundreds of exceptionally talented community members.

mlflow (mlflow/mlflow)

Searching Database…

28.4K ·GitHub
这是什么?MLflow 是一个开源的 AI 工程平台,面向智能体、LLM 应用和传统机器学习模型:它把生产环境 AI 的追踪、评估、监控和优化集中在一处,同时帮你控制成本。它应用广泛,每月下载量超过 6000 万次。
它是加到我代码里的库,还是需要我自己运行的服务?两者都是。你在应用里安装 mlflow 包,并把它指向一个 MLflow 服务器,服务器的网页界面会展示调用轨迹、指标和提示词。服务器可以跑在自己的笔记本上、本地机房、云端,或者使用托管服务。
怎么把它接入我的应用?三步:启动 MLflow 服务器,在代码里开启日志记录,然后运行应用——调用轨迹和指标就会出现在 MLflow 界面里。如果你用的是编程智能体,MLflow CLI 只需一条命令就能装好相关技能并自动加上追踪。
它支持哪些语言和智能体框架?支持 Python、TypeScript/JavaScript、Java 以及其他任何编程语言。一行代码就能自动追踪 60 多个框架——包括 LangChain、LangGraph、DSPy、CrewAI、LlamaIndex、AutoGen 等——还覆盖从 OpenAI、Anthropic 到 Ollama、DeepSeek 的各家模型服务商。
用上它需要重构我的代码吗?不需要——快速上手指南承诺无需复杂配置、也无需大幅修改代码,而且 60 多个框架只需一行代码就能自动追踪。主要的投入是在某处运行服务器,并在应用里记录日志。
它是给 LLM 应用用的,还是给传统机器学习模型用的,还是两者都可以?两者都可以。LLM 和智能体开发者可以用调用追踪、评估、提示词管理和 AI 网关;机器学习团队可以用实验跟踪、模型注册表,以及部署到 Docker、Kubernetes、Azure ML 或 SageMaker。
它是完全开源的,还是用了就意味着要购买付费产品?采用 Apache-2.0 协议且厂商中立:无论你把它部署在哪里,核心功能都保持可用。像 Databricks 这样的托管产品是在其之上提供的,但开源核心是免费的。
AI Gateway 到底是做什么的?它是一个统一的、兼容 OpenAI 接口的网关,面向所有 LLM 服务商:负责请求路由、管理速率限制和故障回退、控制成本,还提供凭证管理、内容防护栏以及用于 A/B 测试的流量分流。
它是怎么评估 LLM 质量的?你可以用 50 多个内置指标和 LLM 评审运行系统化的评估,也可以自定义指标,然后长期跟踪质量指标,在问题进入生产环境之前发现性能倒退。
调用轨迹是开放格式吗,还是被锁定在 MLflow 里?追踪基于 OpenTelemetry 构建,支持任何 LLM 服务商和智能体框架,MLflow 还原生集成了 MCP。
Creator
mlflow
Size
1.4 GB
Stars
28.4K
Age
8y
Commits · 90 Days