Topics / #pipelines

Pipelines projects on GitHub

Data and workflow pipeline tools for processing streams and jobs.

As of , the most-starred Pipelines projects on GitHub are kafka (34K stars), vector (22.7K stars) and taipy (19.4K stars). GitHub.Locker tracks 6 Pipelines repositories with 119.1K stars combined. The fastest-growing on October 16 was kafka, with +10 stars in a day.

6Projects
119.1KStars in total
+22Stars in a day
Browse in the catalog →github.com/topics/pipelines ↗

Hot right now

Most stars gained in a day · Oct 16

+10

kafka

apache/kafka · ★ 34K

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

+9

vector

vectordotdev/vector · ★ 22.7K

An open-source, high-performance observability data pipeline: it collects, transforms, and routes your logs and metrics to any vendors you choose, so you keep control of your observability data. It is maintained by Datadog's Community Open Source Engineering team.

+2

fluentd

fluent/fluentd · ★ 13.6K

Fluentd is an open-source log collector that sits at the center of your logging setup: it gathers events from many data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and more.

+1

taipy

Avaiga/taipy · ★ 19.4K

Taipy is a Python framework that turns data and AI code into production-ready web applications, taking you from a simple pilot script to a deployable app without extra web development work.

The all-time top

6 projects · by stars

  1. #1 · apache/kafka

    kafka

    Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

    34K
  2. #2 · vectordotdev/vector

    vector

    An open-source, high-performance observability data pipeline: it collects, transforms, and routes your logs and metrics to any vendors you choose, so you keep control of your observability data. It is maintained by Datadog's Community Open Source Engineering team.

    22.7K
  3. #3 · Avaiga/taipy

    taipy

    Taipy is a Python framework that turns data and AI code into production-ready web applications, taking you from a simple pilot script to a deployable app without extra web development work.

    19.4K
  1. 4

    argo-workflows

    17Kargoproj/argo-workflows · Go

    An open source, container-native workflow engine for Kubernetes: it orchestrates parallel jobs where every step of your workflow runs as a container. It is a CNCF graduated project.

  2. 5

    fluentd

    13.6Kfluent/fluentd · Ruby

    Fluentd is an open-source log collector that sits at the center of your logging setup: it gathers events from many data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and more.

  3. 6

    grunt

    12.2Kgruntjs/grunt · JavaScript

    Grunt is a JavaScript task runner: you describe your project's repetitive tasks once, and Grunt executes them for you so you don't run each step by hand.

Often tagged together