Topics / #bigdata

Bigdata projects on GitHub

Large-scale data processing platforms and data engineering practices.

As of , the most-starred Bigdata projects on GitHub are data-engineer-handbook (44.5K stars), rustfs (35K stars) and TDengine (25.2K stars). GitHub.Locker tracks 8 Bigdata repositories with 182.3K stars combined. The fastest-growing on October 16 was rustfs, with +49 stars in a day.

8Projects
182.3KStars in total
+72Stars in a day
Browse in the catalog →github.com/topics/bigdata ↗

Hot right now

Most stars gained in a day · Oct 16

+49

rustfs

rustfs/rustfs · ★ 35K

An open-source, high-performance distributed object storage system written in Rust. It speaks the S3 API, is optimized for data lakes, AI and big data workloads, and supports migration and coexistence with MinIO and Ceph.

+9

data-engineer-handbook

DataExpert-io/data-engineer-handbook · ★ 44.5K

A curated handbook of links and resources for learning data engineering: roadmaps, bootcamp materials, books, communities, company tooling lists, blogs, whitepapers, and a directory of data engineering content creators.

+6

juicefs

juicedata/juicefs · ★ 14.5K

JuiceFS is a distributed POSIX file system that stores data in object storage (like S3) and metadata in a database engine (like Redis), letting you use massive cloud storage as if it were a local disk.

+3

TDengine

taosdata/TDengine · ★ 25.2K

TDengine is an open-source, high-performance, cloud-native, AI-powered time-series database built for IoT, Connected Cars, and Industrial IoT workloads, handling TB-to-PB scale data from billions of sensors.

The all-time top

8 projects · by stars

  1. #1 · DataExpert-io/data-engineer-handbook

    data-engineer-handbook

    A curated handbook of links and resources for learning data engineering: roadmaps, bootcamp materials, books, communities, company tooling lists, blogs, whitepapers, and a directory of data engineering content creators.

    44.5K
  2. #2 · rustfs/rustfs

    rustfs

    An open-source, high-performance distributed object storage system written in Rust. It speaks the S3 API, is optimized for data lakes, AI and big data workloads, and supports migration and coexistence with MinIO and Ceph.

    35K
  3. #3 · taosdata/TDengine

    TDengine

    TDengine is an open-source, high-performance, cloud-native, AI-powered time-series database built for IoT, Connected Cars, and Industrial IoT workloads, handling TB-to-PB scale data from billions of sensors.

    25.2K
  1. 4

    shardingsphere

    20.8Kapache/shardingsphere · Java

    An Apache Top-Level Project positioned as 'Database Plus': a layer built over your existing heterogeneous databases that adds distributed SQL - sharding, read/write splitting, encryption - without making you replace them.

  2. 5

    BigData-Notes

    17Kheibaiying/BigData-Notes · Java

    This is a comprehensive Chinese-language study guide covering the big data ecosystem: Hadoop, Hive, Spark, Storm, Flink, HBase, Kafka, Zookeeper, Flume, Sqoop, Azkaban, and Scala.

  3. 6

    awesome-bigdata

    14.7Koxnr/awesome-bigdata

    A curated list of frameworks, databases, tools, papers, and resources related to big data, covering everything from distributed processing engines to key-value stores and visualization tools.

  4. 7

    juicefs

    14.5Kjuicedata/juicefs · Go

    JuiceFS is a distributed POSIX file system that stores data in object storage (like S3) and metadata in a database engine (like Redis), letting you use massive cloud storage as if it were a local disk.

  5. 8

    God-Of-BigData

    10.6Kwangzhiwubigdata/God-Of-BigData

    A curated Chinese-language knowledge base for learning big data engineering and passing big data interviews, covering Java fundamentals, distributed systems theory, and deep dives into Hadoop, Hive, Spark, Flink, HBase, Kafka, and Zookeeper.

Often tagged together