Topics / #etl

Etl projects on GitHub

Extract-transform-load pipelines and workflow orchestration for data.

As of , the most-starred Etl projects on GitHub are pathway (62.1K stars), airflow (47.2K stars) and canal (29.7K stars). GitHub.Locker tracks 20 Etl repositories with 338.9K stars combined. The fastest-growing on October 16 was kestra, with +29 stars in a day.

20Projects
338.9KStars in total
+113Stars in a day
Browse in the catalog →github.com/topics/etl ↗

Hot right now

Most stars gained in a day · Oct 16

+29

kestra

kestra-io/kestra · ★ 29.7K

Kestra is an open-source, event-driven orchestration platform for building data, AI, and infrastructure workflows. You describe them in declarative YAML and manage everything from a built-in web UI.

+29

helix-foundry

HelixDB/helix-foundry · ★ 1.5K

Helix Foundry is a local-first data workspace that connects your company's tools (databases, Stripe, WorkOS, PostHog, files, APIs) into one suggested ontology stored in HelixDB, letting you explore and ask questions about your data.

+9

vector

vectordotdev/vector · ★ 22.7K

An open-source, high-performance observability data pipeline: it collects, transforms, and routes your logs and metrics to any vendors you choose, so you keep control of your observability data. It is maintained by Datadog's Community Open Source Engineering team.

+9

debezium

debezium/debezium · ★ 13.2K

Debezium is an open source change data capture (CDC) platform that monitors databases and streams row-level change events to applications in real time.

The all-time top

20 projects · by stars

  1. #1 · pathwaycom/pathway

    pathway

    Pathway is a Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG, with the same code usable for batch jobs, CI/CD tests, and live data streams.

    62.1K
  2. #2 · apache/airflow

    airflow

    Apache Airflow is a platform to programmatically author, schedule, and monitor workflows, defined as code using Python.

    47.2K
  3. #3 · alibaba/canal

    canal

    Canal is Alibaba's MySQL binlog incremental subscription and consumption component. It disguises itself as a MySQL slave, pulls binlog events from a MySQL master, and lets you consume the parsed changes for things like cache refresh, index building, or database synchronization.

    29.7K
  1. 4

    kestra

    29.7Kkestra-io/kestra · Java

    Kestra is an open-source, event-driven orchestration platform for building data, AI, and infrastructure workflows. You describe them in declarative YAML and manage everything from a built-in web UI.

  2. 5

    dataease

    24.6Kdataease/dataease · Java

    DataEase is an open-source BI (business intelligence) tool for quickly analyzing data and spotting business trends: you connect data sources, build charts by drag-and-drop and share the results. It is positioned as an open alternative to Tableau.

  3. 6

    vector

    22.7Kvectordotdev/vector · Rust

    An open-source, high-performance observability data pipeline: it collects, transforms, and routes your logs and metrics to any vendors you choose, so you keep control of your observability data. It is maintained by Datadog's Community Open Source Engineering team.

  4. 7

    airbyte

    22.2Kairbytehq/airbyte · Python

    Airbyte is an open-source data movement platform for ELT pipelines and AI agents: it moves data from APIs, databases, and files into warehouses, data lakes, and AI applications, backed by a catalog of 600+ connectors. You can run it self-hosted or as Airbyte Cloud.

  5. 8

    luigi

    18.8Kspotify/luigi · Python

    Luigi is a Python package for building complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, failure handling and command line integration.

  6. 9

    DataX

    17.4Kalibaba/DataX · Java

    DataX is an offline data synchronization tool/platform from Alibaba Cloud, open-sourced from the DataWorks Data Integration product, used widely inside Alibaba to move data between heterogeneous data sources.

  7. 10

    dagster

    16.3Kdagster-io/dagster · Python

    Dagster is a cloud-native data pipeline orchestrator covering the whole development lifecycle: it builds, runs, and observes data assets like tables, datasets, ML models, and reports, with built-in lineage and observability.

  8. 11

    unstructured

    15.6KUnstructured-IO/unstructured · HTML

    Unstructured is an open-source ETL toolkit that converts complex documents like PDFs, Word docs, HTML and images into clean, structured data (JSON) ready for LLMs, RAG pipelines and vector databases.

  9. 12

    debezium

    13.2Kdebezium/debezium · Java

    Debezium is an open source change data capture (CDC) platform that monitors databases and streams row-level change events to applications in real time.

  10. 13

    cocoindex

    11.7Kcocoindex-io/cocoindex · Rust

    CocoIndex is an incremental engine that turns codebases, meeting notes, inboxes, Slack, PDFs, and videos into live, continuously fresh context for AI agents and LLM apps, using minimal incremental processing instead of stale batch jobs.

  11. 14

    ducklake

    3.3Kduckdb/ducklake · C++

    DuckLake is an open lakehouse table format that stores data in Parquet files and stores metadata/catalog info in a SQL database, implemented as a DuckDB extension.

  12. 15

    hotdata-cli

    1.8Khotdata-dev/hotdata-cli · Rust

    Hotdata CLI is a command-line client that lets you query, search, and join data from one place — external databases, APIs, cloud storage, Iceberg catalogs, and uploaded files — using plain SQL and a handful of commands.

  13. 16

    helix-foundry

    1.5KHelixDB/helix-foundry · TypeScript

    Helix Foundry is a local-first data workspace that connects your company's tools (databases, Stripe, WorkOS, PostHog, files, APIs) into one suggested ontology stored in HelixDB, letting you explore and ask questions about your data.

  14. 17

    DoveVannoINostriSoldi

    344Italian-Builders-Org/DoveVannoINostriSoldi · TypeScript

    DoveVannoINostriSoldi is a civic open-source project that collects and analyzes Italian public spending data from official sources, using AI to highlight where public resources could be used more efficiently. It also includes a budget law (Legge di Bilancio) reallocation simulator based on OpenBDAP commitment-basis appropriations.

  15. 18

    projeto-dados-ia-databricks

    279lvgalvao/projeto-dados-ia-databricks · Python

    This is a 4-night hands-on course that builds the data and sales analytics stack of a fictional B2B perfume distributor from scratch, using Databricks Free Edition, SQL, Python and Claude Code.

  16. 19

    walshadow

    242ClickHouse/walshadow · Rust

    walshadow replicates PostgreSQL rows into ClickHouse by reading the physical WAL — initial load, continuous changes, schema evolution, restarts, and planned source switchover are all part of the flow.

  17. 20

    full-stack-data-engineering

    145schumannx/full-stack-data-engineering · Python

    A full reference implementation of a streaming data warehouse on AWS: playback events flow from Kafka into a five-zone S3 Iceberg lake, get modeled with Kimball dimensional design, and are served through Athena SQL and HTML reports, all orchestrated by self-hosted Apache Airflow 3.

Often tagged together