#398 · +94 · 1d ·GitHub
One workspace: what ran, what broke, what is already pinned as a test.
Global
Fine-tune an 8B model on a 4 GB laptop GPU. Layer streaming keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time. 32 GB.
Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready.
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
is a CLI and library for evaluating and red-teaming LLM apps. Stop the trial-and-error approach - start shipping secure, reliable AI apps.
🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
The open-source agentic framework to build, orchestrate, and operate production AI agents.
Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode.
Ragas is your ultimate toolkit for evaluating and optimizing Large Language Model (LLM) applications. Say goodbye to time-consuming, subjective assessments and hello to data-driven, efficient evaluation workflows. Don't have a test dataset ready?
Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
DeepEval is a simple-to-use, open-source LLM evaluation framework, for evaluating large-language model systems. It is similar to Pytest but specialized for unit testing LLM apps. , which use LLM-as-a-judge and other NLP models that run locally on your machine.
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
Arize Phoenix is Arize's open-source AI observability platform designed for experimentation, evaluation, and troubleshooting. For managed production workflows, Arize also offers Arize AX. Phoenix provides:
#1898 · +13 · 1d ·GitHub
In this course, we'll go from experimentation (design + development) to production (deployment + iteration). We'll do this iteratively by motivating the components that will enable us to build a reliable production system.
#2401 · +9 · 1d ·GitHub
Complete Beginner? → Beginner-Friendly Resources Want to Learn Trending Tech? → Web3 & Blockchain | AI & ML | Cybersecurity Building Web Apps?
Point it at your agent's codebase and it turns production traces (or any dataset) into a fine-tuned model, benchmarked against the eval metrics you define and served via 1 unified API, with no ML infrastructure to build.
A no-code RAG server that keeps answers in sync with your documents. Works with 8 vector databases. Built on Pathway.
#3601 · +4 · 1d ·GitHub
本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
0 release. You can try the pre-release branch here. > Read more about what's next for Portkey in our Series A announcement.
The execution control plane for AI agents. Govern every model request, MCP tool call, A2A delegation, and downstream action—while keeping your native protocols intact: six native LLM protocols in, six native LLM protocols out.
#4313 · +3 · 1d ·GitHub
until forever. Exceptional understanding of French with the flawless accent implies correct exposure to exceptional French and flawless accent. Similarly, great AI Researchers & engineers with excellent problem-skills imply quality knowledge consumed and exposure rich experience.
SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.
#5697 · +1 · 1d ·GitHub
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view
Metaflow is a human-centric framework designed to help scientists and engineers build and manage real-life AI and ML systems.
#6358 · +1 · 1d ·GitHub
Hands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set — model APIs, structured output, tool calling, RAG, evals-as-the-spine, agents (loop from scratch, tool design, guardrails, MCP, Skills), fine-tuning vs LoRA…
Jina-serve is a framework for building and deploying AI services that communicate via gRPC, HTTP and WebSockets. Scale your services from local development to production while focusing on your core logic.
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
BISHENG is an open LLM devops platform for next generation Enterprise AI applications.
TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.
#7507 · +0 · 1d ·GitHub
Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.