Topics / #pipeline

Pipeline projects on GitHub

Data and deployment pipeline orchestration

As of , the most-starred Pipeline projects on GitHub are kestra (29.7K stars), argo-cd (24.4K stars) and prefect (24K stars). GitHub.Locker tracks 23 Pipeline repositories with 348.5K stars combined. The fastest-growing on October 16 was kestra, with +29 stars in a day.

23Projects
348.5KStars in total
+111Stars in a day
Browse in the catalog →github.com/topics/pipeline ↗

Hot right now

Most stars gained in a day · Oct 16

+29

kestra

kestra-io/kestra · ★ 29.7K

Kestra is an open-source, event-driven orchestration platform for building data, AI, and infrastructure workflows. You describe them in declarative YAML and manage everything from a built-in web UI.

+20

marimo

marimo-team/marimo · ★ 23.2K

A reactive Python notebook: run a cell or touch a UI element and dependent cells re-run automatically, keeping code and outputs consistent. Notebooks are pure Python files that can run as scripts or deploy as web apps.

+14

openobserve

openobserve/openobserve · ★ 22.4K

OpenObserve is an open-source, cloud-native observability platform for logs, metrics, traces, frontend monitoring (RUM), session replay and AI/LLM observability — a cost-effective alternative to Datadog, Splunk and Elasticsearch.

+8

argo-cd

argoproj/argo-cd · ★ 24.4K

Argo CD is a declarative GitOps continuous delivery tool for Kubernetes that keeps your cluster state synced with a Git repository.

The all-time top

23 projects · by stars

  1. #1 · kestra-io/kestra

    kestra

    Kestra is an open-source, event-driven orchestration platform for building data, AI, and infrastructure workflows. You describe them in declarative YAML and manage everything from a built-in web UI.

    29.7K
  2. #2 · argoproj/argo-cd

    argo-cd

    Argo CD is a declarative GitOps continuous delivery tool for Kubernetes that keeps your cluster state synced with a Git repository.

    24.4K
  3. #3 · PrefectHQ/prefect

    prefect

    Prefect is a Python workflow orchestration framework for turning ordinary scripts into resilient, production-grade data pipelines with scheduling, retries, caching, and event-based automations.

    24K
  1. 4

    marimo

    23.2Kmarimo-team/marimo · Python

    A reactive Python notebook: run a cell or touch a UI element and dependent cells re-run automatically, keeping code and outputs consistent. Notebooks are pure Python files that can run as scripts or deploy as web apps.

  2. 5

    openobserve

    22.4Kopenobserve/openobserve · TypeScript

    OpenObserve is an open-source, cloud-native observability platform for logs, metrics, traces, frontend monitoring (RUM), session replay and AI/LLM observability — a cost-effective alternative to Datadog, Splunk and Elasticsearch.

  3. 6

    airbyte

    22.2Kairbytehq/airbyte · Python

    Airbyte is an open-source data movement platform for ELT pipelines and AI agents: it moves data from APIs, databases, and files into warehouses, data lakes, and AI applications, backed by a catalog of 600+ connectors. You can run it self-hosted or as Airbyte Cloud.

  4. 7

    serve

    21.9Kjina-ai/serve · Python

    Jina is an open-source Python framework for building multimodal AI applications on a cloud-native stack — you wrap models into gRPC microservices, chain them into flows, and scale from local to cloud.

  5. 8

    FlameGraph

    19.8Kbrendangregg/FlameGraph · Perl

    Brendan Gregg's FlameGraph: a set of Perl scripts that turn captured stack traces into interactive SVG flame graphs for visualizing profiled code.

  6. 9

    taipy

    19.4KAvaiga/taipy · Python

    Taipy is a Python framework that turns data and AI code into production-ready web applications, taking you from a simple pilot script to a deployable app without extra web development work.

  7. 10

    kubeflow

    15.9Kkubeflow/kubeflow

    Kubeflow is the foundation of tools for building AI platforms on Kubernetes, covering every stage of the AI lifecycle through a set of composable, Kubernetes-native subprojects.

  8. 11

    logstash

    15Kelastic/logstash · Java

    Logstash is a server-side data processing pipeline, part of the Elastic Stack, that ingests data from many sources at once, transforms it, and sends it to a destination like Elasticsearch.

  9. 12

    Meshroom

    13Kalicevision/Meshroom · Python

    Meshroom is an open-source, node-based visual programming framework for building and running photogrammetry and 3D computer-vision pipelines, turning collections of 2D photos into 3D models, camera tracks, HDR panoramas, and more.

  10. 13

    great_expectations

    11.9Kfivetran/great_expectations · Python

    GX Core (Great Expectations) is a Python framework for testing, validating, and documenting the quality of your data. Its core concept is the 'Expectation' — an expressive unit test for your data, built from the collective experience of thousands of data quality deployments.

  11. 14

    spug

    11.1Kopenspug/spug · JavaScript

    Spug is a lightweight, agent-free ops platform built for small and medium-sized businesses that bundles host management, batch command execution, an online SSH terminal, file transfer, build/deploy pipelines, task scheduling, a configuration center, and monitoring/alerting into one web app.

  12. 15

    kedro

    11Kkedro-org/kedro · Python

    Kedro is an open-source Python framework for building production-ready data engineering and data science pipelines, using software engineering best practices to make them reproducible, maintainable, and modular.

  13. 16

    prql

    10.9KPRQL/prql · Rust

    PRQL (pronounced 'Prequel') is a modern language for transforming data — a simple, powerful, pipelined replacement for SQL.

  14. 17

    StreamDiffusion

    10.8Kcumulo-autumn/StreamDiffusion · Python

    StreamDiffusion is a pipeline-level library for real-time, interactive diffusion image generation, letting you turn text prompts or a live webcam/screen feed into generated images at high frame rates.

  15. 18

    httpx

    10.5Kprojectdiscovery/httpx · Go

    httpx is a fast, multi-purpose HTTP toolkit from ProjectDiscovery that runs many probes against lists of hosts and URLs, built on the retryablehttp library to stay reliable under high thread counts.

  16. 19

    metaflow

    10.3KNetflix/metaflow · Python

    Metaflow is an open-source Python framework for building, managing and deploying real-life AI and ML systems — from quick prototyping in notebooks to reliable production deployments.

  17. 20

    Jenkins-Zero-To-Hero

    10.1Kiam-veeramalla/Jenkins-Zero-To-Hero · Python

    A learning repo that walks through Jenkins from zero to hero: installing Jenkins, configuring Docker as a build agent, setting up CI/CD, and deploying apps to Kubernetes using Argo CD in a GitOps workflow.

  18. 21

    tpot

    10.1KEpistasisLab/tpot · Jupyter Notebook

    TPOT, the Tree-based Pipeline Optimization Tool, is a Python AutoML library that uses genetic programming to automatically optimize machine learning pipelines for your data; the README calls it your Data Science Assistant.

  19. 22

    mortiflix-oss

    639GTKottman/mortiflix-oss · JavaScript

    Mortiflix is a self-hosted motion design studio where Claude makes a video step by step - brief, script, style frames, transition board, animatic, final - and you review and approve each stage before the next one starts.

  20. 23

    ESP32s3-LLM-Cluster

    239Low-Zi-Hong/ESP32s3-LLM-Cluster · C++

    This project is a distributed inference engine that runs a 0.5B-parameter LLM, quantized to 1.58-bit (BitNet) ternary weights, split across a cluster of 7 ESP32-S3 microcontrollers connected via SPI.

Often tagged together