Global

data-pipeline
14 projects tagged “data-pipeline” that gained the most stars yesterday
#2985 · +7 · 1d ·GitHub

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

Creator
dbt-labs
Size
126.5 MB
Stars
14K
Age
10y
Commits · 90 Days
#3414 · +5 · 1d ·GitHub

An open-source engine that turns continuous content streams into search and monitoring. Durable ingestion, hybrid search, alerts, and plugins for formats, models and business rules.

Size
170 MB
Stars
39.6K
Age
3y
Commits · 90 Days
#3456 · +5 · 1d ·GitHub

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

Creator
airbytehq
Size
923.3 MB
Stars
22.2K
Age
6y
Commits · 90 Days
#3541 · +5 · 1d ·GitHub

Debezium is an open source project that provides a low latency data streaming platform for change data capture (CDC). You set up and configure Debezium to monitor your databases, and then your applications consume events for each row-level change made to the database.

Creator
debezium
Size
90.8 MB
Stars
13.2K
Updated
CommitOct 19
Age
10y
Commits · 90 Days
#5007 · +2 · 1d ·GitHub

Dask is a flexible parallel computing library for analytics. See documentation for more information.

Creator
dask
Size
131 MB
Stars
13.9K
Age
11y
Commits · 90 Days
#5410 · +2 · 1d ·GitHub

Open source SDK for scalable multimodal data pipelines in robotics and physical AI

Size
10.6 MB
Stars
302
Age
2mo
Commits · 90 Days
#5659 · +1 · 1d ·GitHub

Apache ShardingSphere is positioned as Database Plus, a standard and ecosystem built on top of heterogeneous databases.

Creator
apache
Size
692.2 MB
Stars
20.8K
Age
10y
Commits · 90 Days
#5932 · +1 · 1d ·GitHub

Fluentd: Open-Source Log Collector ===================================

Creator
fluent
Size
21.2 MB
Stars
13.6K
Age
15y
Commits · 90 Days
#6479 · +1 · 1d ·GitHub

O projeto inteiro — do CSV cru ao app que escreve de volta na gold — sai de doze prompts colados em sequência no Claude Code. Cada um termina com um deploy.

Creator
lvgalvao
Size
42.4 MB
Stars
280
Updated
CommitAug 27
Age
1mo
Commits · 90 Days
#6948 · +0 · 1d ·GitHub

High-performance AI pipeline engine with a C++ core and 50+ Python-extensible nodes. Build, debug, and scale LLM workflows with 13+ model providers, 8+ vector databases, and agent orchestration, all from your IDE.

Size
127.1 MB
Stars
17.7K
Age
8mo
Commits · 90 Days
#6974 · +0 · 1d ·GitHub

DataX是阿里云DataWorks数据集成的开源版本。

Creator
alibaba
Size
22.4 MB
Stars
17.4K
Age
8y
Commits · 90 Days
#8119 · +0 · 1d ·GitHub

db). js só-leitura em /web — busca candidato e mostra a ficha completa com a fonte de cada campo.

Creator
YuriRDev
Size
3.7 MB
Stars
458
Updated
CommitOct 03
Age
1mo
Commits · 90 Days
#8129 · +0 · 1d ·GitHub

The in-context human video generation pipeline starts from task-sampled robot trajectories that retain executable actions.

Size
124 MB
Stars
435
Updated
CommitSep 14
Age
1mo
Commits · 90 Days
#8313 · +0 · 1d ·GitHub

WARNING] > Experimental > > Project status: Development Preview > > walshadow is under active development and being hardened through real-world > testing and feedback. We recommend validating it thoroughly with your > workloads. Interfaces and behavior may evolve

Creator
ClickHouse
Size
6.8 MB
Stars
242
Age
4mo
Commits · 90 Days