Topics / #data-integration

Data-integration projects on GitHub

Pipelines that integrate and sync data between systems and warehouses.

As of , the most-starred Data-integration projects on GitHub are airflow (47.2K stars), kafka (34K stars) and airbyte (22.2K stars). GitHub.Locker tracks 6 Data-integration repositories with 139.4K stars combined. The fastest-growing on October 16 was kafka, with +10 stars in a day.

6Projects
139.4KStars in total
+25Stars in a day
Browse in the catalog →github.com/topics/data-integration ↗

Hot right now

Most stars gained in a day · Oct 16

+10

kafka

apache/kafka · ★ 34K

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

+7

airflow

apache/airflow · ★ 47.2K

Apache Airflow is a platform to programmatically author, schedule, and monitor workflows, defined as code using Python.

+4

dagster

dagster-io/dagster · ★ 16.3K

Dagster is a cloud-native data pipeline orchestrator covering the whole development lifecycle: it builds, runs, and observes data assets like tables, datasets, ML models, and reports, with built-in lineage and observability.

+3

airbyte

airbytehq/airbyte · ★ 22.2K

Airbyte is an open-source data movement platform for ELT pipelines and AI agents: it moves data from APIs, databases, and files into warehouses, data lakes, and AI applications, backed by a catalog of 600+ connectors. You can run it self-hosted or as Airbyte Cloud.

The all-time top

6 projects · by stars

  1. #1 · apache/airflow

    airflow

    Apache Airflow is a platform to programmatically author, schedule, and monitor workflows, defined as code using Python.

    47.2K
  2. #2 · apache/kafka

    kafka

    Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

    34K
  3. #3 · airbytehq/airbyte

    airbyte

    Airbyte is an open-source data movement platform for ELT pipelines and AI agents: it moves data from APIs, databases, and files into warehouses, data lakes, and AI applications, backed by a catalog of 600+ connectors. You can run it self-hosted or as Airbyte Cloud.

    22.2K
  1. 4

    taipy

    19.4KAvaiga/taipy · Python

    Taipy is a Python framework that turns data and AI code into production-ready web applications, taking you from a simple pilot script to a deployable app without extra web development work.

  2. 5

    dagster

    16.3Kdagster-io/dagster · Python

    Dagster is a cloud-native data pipeline orchestrator covering the whole development lifecycle: it builds, runs, and observes data assets like tables, datasets, ML models, and reports, with built-in lineage and observability.

  3. 6

    walshadow

    242ClickHouse/walshadow · Rust

    walshadow replicates PostgreSQL rows into ClickHouse by reading the physical WAL — initial load, continuous changes, schema evolution, restarts, and planned source switchover are all part of the flow.

Often tagged together