run-ai README

The project's README from github.com/dsx-ai-factory/run-ai. As one markdown file: README.md.

NVIDIA Run:ai: AI Workload and GPU Orchestration

NVIDIA Run:ai

License CI Latest release Contributors

Docs | Roadmap | Installation | API Reference | Contributing | License

NVIDIA Run:ai is an open-source AI workload and GPU orchestration platform for managing shared AI infrastructure on Kubernetes.

NVIDIA Run:ai provides a consistent way to run development, training, and inference workloads across Kubernetes environments, whether deployed on-premises, in the cloud, or across hybrid infrastructure. It combines GPU scheduling, resource management, workload orchestration, governance, and observability to help organizations share GPU resources efficiently across teams and clusters.

This monorepo contains the NVIDIA Run:ai control plane and cluster services, installer, CLI, MCP server, documentation, shared client packages, and the local development environment.

Architecture

NVIDIA Run:ai system overview

Architecture Overview →

NVIDIA Run:ai is made up of two components, both installed on a Kubernetes cluster:

At a high level:

Core Capabilities

Capability What it does
AI-aware GPU scheduling Schedules GPU workloads using fair-share, preemption, bin-packing, priorities, and topology-aware placement.
Dynamic GPU allocation Shares GPU resources across workloads with fractional GPU allocation and dynamic use of available capacity.
Resource quotas and policies Allocates GPU resources across departments and projects and enforces organizational resource policies.
Multi-cluster management Centrally manages workloads, resources, and configuration across multiple Kubernetes clusters.
AI workload management Manages workspaces, training, distributed training, and inference, with extensible support for additional Kubernetes workloads.
Access control and governance Controls access to NVIDIA Run:ai resources with role-based permissions and enforces organizational policies across workloads and resources.
Monitoring and observability Provides visibility into clusters, nodes, GPUs, workloads, resource utilization, and platform activity.
Multiple interfaces Supports NVIDIA Run:ai operations through the web UI, CLI, and REST API.

How NVIDIA Run:ai Relates to Other Tools

NVIDIA Run:ai is a scheduling and orchestration layer on top of Kubernetes: it does not replace Kubernetes, a container runtime, or a CI/CD system.

Layer Tools How NVIDIA Run:ai relates
Cluster orchestration Kubernetes, OpenShift NVIDIA Run:ai runs on these. It extends the cluster with GPU-aware scheduling rather than replacing native scheduling.
Schedulers Default kube-scheduler, Volcano, Kueue NVIDIA Run:ai ships the KAI Scheduler and adds fair-share, preemption, fractional GPUs, and quota hierarchies across departments and projects.
GPU enablement NVIDIA GPU Operator, Network Operator Complementary: these expose and configure GPUs on nodes; NVIDIA Run:ai decides which workloads get them and for how long.
ML platforms and pipelines Kubeflow, Ray, Argo Workflows, Slurm Complementary for the Kubernetes-based ones: they submit workloads, NVIDIA Run:ai schedules and governs the GPU capacity those workloads consume. Slurm is an alternative for HPC-style batch on bare metal.
Workload types Training Operator, LeaderWorkerSet, Knative, KServe NVIDIA Run:ai integrates with these to manage workspaces, training, distributed training, and inference as first-class workloads.

If you need GPU sharing, quota enforcement, and multi-tenant governance across teams sharing one Kubernetes GPU estate, that is the gap NVIDIA Run:ai fills. If you only need to run a single team's jobs on dedicated GPUs, native Kubernetes scheduling may be sufficient.

Repository Map

Component Path Description
Control Plane control-plane/ APIs and services that manage workloads, policies, identity, organizational structure, and platform configuration
NVIDIA Run:ai Cluster runai-cluster/ Controllers, admission webhooks, and services deployed into each managed Kubernetes cluster
Installer runai-installer/ CLI tool and provisioner service for deploying NVIDIA Run:ai on Kubernetes
MCP Server runai-mcp-server/ Model Context Protocol server that exposes NVIDIA Run:ai operations as tools for AI assistants
Clients and shared packages runai-clients/, runai-common-packages/, runai-application-packages/ Generated API clients and shared Go libraries
Scale/Load Testing scale-tests/ Load-testing SDK and unified load-test suite used to validate platform behavior at scale
Documentation docs/, fern/ Product and developer documentation
Local environment local-dev.sh, local-dev.md One-command local Kubernetes (Kind) development environment

For a detailed breakdown of individual services and components, see the Service Reference.

Usage

Local Development Environment

The local-dev.sh script builds the NVIDIA Run:ai control plane and cluster services from source and deploys them to a self-contained Kind cluster with a local image registry.

./local-dev.sh                                  # full build + cluster + deploy
./local-dev.sh --skip-build --skip-cluster      # re-deploy only
./local-dev.sh --teardown                       # tear the environment down

See Local Development Guide for prerequisites (Docker/Podman, make, git, curl, python3, openssl, and minimum CPU/RAM/disk; kind, kubectl, and helm are installed automatically) and all available flags. Go, Node, and Java are not needed on the host: all builds run inside containers.

Building Individual Services

Each service directory has a Makefile with standard targets (make build, make test, make lint, make validate) and a README describing how to build, test, and work with that service. See the Service Reference for a breakdown of individual services and components.

Installation

NVIDIA Run:ai can be deployed on Kubernetes or OpenShift in connected or air-gapped environments. To understand the full installation lifecycle, see the Installation and Configuration Overview.

Before you start:

Choose an installation method based on how you want to manage the deployment:

To build and run NVIDIA Run:ai locally for development, see Usage.

Distribution Channels

Released NVIDIA Run:ai container images and Helm charts are published to the NVIDIA NGC container registry under nvcr.io/nvidia/runai: this is the registry the installer and Helm charts pull from by default. Source releases and their notes are on the GitHub Releases page; see Release Process for how releases are cut and versioned.

Contributing

Contributions are welcome. Please read Contributing for the development workflow, branch and PR policy, and CI expectations, and Code of Conduct for community guidelines.

Roadmap

NVIDIA Run:ai does not publish a long-range feature roadmap in this repository. What is planned and in progress is tracked in the open:

Release cadence, branch naming, and who cuts a release are documented in Release Process.

Known limitations: the root Makefile targets are placeholders; real build, test, and lint targets live in each service directory (see Building Individual Services). The full CI gate runs remotely and is reported back on pull requests as the Validate Successful Run check.

Community

Security & Support

License

Licensed under the Apache License, Version 2.0; see LICENSE. Third-party attributions are listed in NOTICE and in the per-component NOTICE files.

Trademark notice: Apache 2.0 applies to the software code in this repository. It does not grant permission to use NVIDIA or Run:ai trademarks, logos, product names, fonts, artwork, or other brand assets. Please see Trademarks for details.

Nested and Third-Party Licenses

Some subdirectories contain vendored third-party code or bundled assets under their own license. Each is listed below with its license and the path to that license file:

Subdirectory License License file
skills/ Apache-2.0 AND CC-BY-4.0 skills/LICENSE
runai-cluster/cluster/goofys/ Apache-2.0 runai-cluster/cluster/goofys/LICENSE.md (attributions in the adjacent NOTICE)
control-plane/assets-service/third_party/go_acc/ Apache-2.0 .../go_acc/LICENSE
control-plane/workloads-and-policies/third_party/kserve/ Apache-2.0 .../kserve/LICENSE
runai-application-packages/jsonutils/conjungo/ MIT .../conjungo/LICENSE
runai-application-packages/jsonutils/gojsonq/ MIT .../gojsonq/LICENSE.md
runai-application-packages/validation/ MIT .../validation/LICENSE
runai-cluster/cluster/workload-controller/third_party/validation/ MIT .../third_party/validation/LICENSE
control-plane/frontend/public/static-cli/fonts/ Apache-2.0 .../fonts/LICENSE.txt

The following subdirectories ship an aggregated third-party dependency license report (not a license for their own code):

Subdirectory Report
runai-cluster/cluster/init-ca/licenses/ THIRD-PARTY-LICENSES.md (generated by go-licenses)

run-ai (dsx-ai-factory/run-ai)

helm-charts
Searching Database…

462 ·GitHub
Run-ai
What is this?NVIDIA Run:ai is an open-source AI workload and GPU orchestration platform for managing shared AI infrastructure on Kubernetes, handling GPU scheduling, resource quotas, and governance across teams.
Is this a replacement for Kubernetes, or does it run on top of it?It runs on top of Kubernetes (or OpenShift) as a scheduling and orchestration layer; it does not replace Kubernetes, the container runtime, or CI/CD.
How do I install and run it?For production, use the guided installation with the runai-installer CLI and a config YAML (recommended), or deploy the Helm charts manually; for local development, run the local-dev.sh script to spin up a self-contained Kind cluster with the control plane and cluster services.
What do I need on my machine to run the local dev environment?You need Docker or Podman, make, git, curl, python3, openssl, and minimum CPU/RAM/disk; kind, kubectl, and helm are installed automatically, and you do not need Go, Node, or Java since all builds run in containers.
Is there a published roadmap or are the root Makefile build targets ready to use?There is no published long-range roadmap in the repo (tracked via open issues and releases instead), and the root Makefile targets are placeholders — real build/test/lint targets live in each service's own directory.
Who is this for and what environments does it support?It's aimed at organizations running shared GPU infrastructure for AI workloads, and can be deployed on Kubernetes or OpenShift in connected or air-gapped environments.
Does this require NVIDIA GPUs and GPU operators?Yes, it's built around GPU-aware scheduling and complements the NVIDIA GPU Operator and Network Operator, which expose and configure GPUs on nodes while Run:ai decides which workloads get them.
What's inside this monorepo?It bundles the control plane services (control-plane/), the in-cluster operator/controllers (runai-cluster/), the installer CLI (runai-installer/), an MCP server, shared client/common packages, scale/load-testing tools, docs, and the local-dev environment.
How does Run:ai differ from the default Kubernetes scheduler or tools like Volcano/Kueue?Run:ai ships its own KAI Scheduler adding fair-share scheduling, preemption, fractional GPU allocation, and quota hierarchies across departments and projects, which the default kube-scheduler and alternatives like Volcano or Kueue don't fully provide out of the box.
What interfaces can I use to interact with it?You can operate Run:ai through a web UI, a CLI, a REST API, and there's also an MCP server that exposes Run:ai operations as tools for AI assistants.
Where do the official container images and Helm charts come from?Released images and Helm charts are published to the NVIDIA NGC container registry under nvcr.io/nvidia/runai, which is what the installer and Helm charts pull from by default.
What are similar projects?evenfire, kubeshark, encore, kubesphere, rook, thanos, OpenLore, linkerd2
Which topics is it in?helm-charts, access-control, rbac, cloud-native, orchestration, observability, distributed-systems, kubernetes, inference, nvidia, sdk, monitoring
Creator
dsx-ai-factory
Size
168.6 MB
Stars
462
Updated
CommitOct 16Release3.0.20Oct 11
Age
1mo
Commits · 90 Days