Topics / #scalability

Scalability projects on GitHub

Architecture patterns for scaling distributed systems

As of , the most-starred Scalability projects on GitHub are kubernetes (128.3K stars), awesome-scalability (74.8K stars) and socket.io (63.2K stars). GitHub.Locker tracks 57 Scalability repositories with 1.3M stars combined. The fastest-growing on October 16 was awesome-system-design-resources, with +161 stars in a day.

57Projects
1.3MStars in total
+493Stars in a day
Browse in the catalog →github.com/topics/scalability ↗

Hot right now

Most stars gained in a day · Oct 16

+161

awesome-system-design-resources

ashishps1/awesome-system-design-resources · ★ 42.4K

A free, curated collection of System Design resources: core concepts, networking and API fundamentals, database and caching basics, distributed systems patterns, and a huge list of interview problems, books, courses, and papers — all aimed at prepping for System Design interviews.

+126

ax

google/ax · ★ 13.9K

AX is Google's open agentic orchestration runtime — a declarative, high-throughput orchestrator that runs billions of autonomous agent workloads in a Kubernetes cluster, sandboxing each task and fencing its network.

+25

awesome-scalability

binhnguyennus/awesome-scalability · ★ 74.8K

A curated, continuously updated reading list of articles, case studies, and talks from engineers at companies like Netflix, Uber, Facebook, and Amazon, showing concrete patterns for building scalable, reliable, and performant large-scale systems.

+24

seaweedfs

seaweedfs/seaweedfs · ★ 35.5K

SeaweedFS is a distributed storage system that serves billions of small and large files with O(1) disk access, exposing the same data as an S3 object store, a POSIX file system, and an Iceberg/Lance data lakehouse through one binary.

The all-time top

57 projects · by stars

  1. #1 · kubernetes/kubernetes

    kubernetes

    Kubernetes (K8s) is an open source system for managing containerized applications across multiple hosts, handling the deployment, maintenance, and scaling of those applications.

    128.3K
  2. #2 · binhnguyennus/awesome-scalability

    awesome-scalability

    A curated, continuously updated reading list of articles, case studies, and talks from engineers at companies like Netflix, Uber, Facebook, and Amazon, showing concrete patterns for building scalable, reliable, and performant large-scale systems.

    74.8K
  3. #3 · socketio/socket.io

    socket.io

    Socket.IO is a library that enables bidirectional, low-latency, real-time communication between a Node.js server and clients such as browsers or mobile apps.

    63.2K
  1. 4

    professional-programming

    51.6Kcharlax/professional-programming · Python

    It's a curated collection of learning resources (articles, books, talks, papers) meant to make software engineers more well-rounded professionals, covering everything from algorithms to career growth to system architecture.

  2. 5

    system-design

    46.6Kkaranpratapsingh/system-design

    It is a free course and ebook that teaches how to design large-scale systems and prepares readers for system design interviews.

  3. 6

    milvus

    46.4Kmilvus-io/milvus · Go

    Milvus is a high-performance, cloud-native vector database built to store and search embedding vectors at scale for AI applications like semantic search, RAG, and recommendation systems.

  4. 7

    awesome-system-design-resources

    42.4Kashishps1/awesome-system-design-resources · Java

    A free, curated collection of System Design resources: core concepts, networking and API fundamentals, database and caching basics, distributed systems patterns, and a huge list of interview problems, books, courses, and papers — all aimed at prepping for System Design interviews.

  5. 8

    seaweedfs

    35.5Kseaweedfs/seaweedfs · Go

    SeaweedFS is a distributed storage system that serves billions of small and large files with O(1) disk access, exposing the same data as an S3 object store, a POSIX file system, and an Iceberg/Lance data lakehouse through one binary.

  6. 9

    tokio

    33.4Ktokio-rs/tokio · Rust

    Tokio is a runtime for writing reliable, asynchronous applications with Rust: it supplies the scheduling, non-blocking I/O, networking and timers that async Rust programs need.

  7. 10

    echo

    32.8Klabstack/echo · Go

    Echo is a high performance, minimalist web framework for Go, used to build web apps, RESTful APIs and microservices.

  8. 11

    rocksdb

    32.2Kfacebook/rocksdb · C++

    RocksDB is an embeddable, persistent key-value store for fast storage, built as a C++ library and developed by Facebook's Database Engineering team.

  9. 12

    dragonfly

    31.8Kdragonflydb/dragonfly · C++

    An in-memory data store built as a modern replacement for Redis and Memcached — fully compatible with both APIs, so no code changes needed; it claims 25X more throughput on up to 80% fewer resources.

  10. 13

    shardeum

    31.2Kshardeum/shardeum · TypeScript

    Shardeum is an EVM-compliant blockchain platform that uses dynamic state sharding to scale horizontally, aiming for faster processing and lower transaction costs without sacrificing security or decentralization.

  11. 14

    system-design-academy

    29.4Ksystemdesign42/system-design-academy

    A curated collection of system design case studies, fundamentals, interview prep, and AI engineering resources, organized as a big reference list tied to a newsletter.

  12. 15

    celery

    29Kcelery/celery · Python

    Celery is a distributed task queue for Python: clients put units of work (tasks) on a queue and dedicated worker processes monitor the queue and run them — across threads, machines, even datacenters.

  13. 16

    rethinkdb

    27Krethinkdb/rethinkdb · C++

    RethinkDB is an open-source, distributed NoSQL database built for realtime web applications; instead of making your app poll for changes, it pushes updated query results straight to you.

  14. 17

    nsq

    25.8Knsqio/nsq · Go

    NSQ is a realtime distributed messaging platform — a message queue built to operate at scale, handling billions of messages per day.

  15. 18

    TDengine

    25.2Ktaosdata/TDengine · C

    TDengine is an open-source, high-performance, cloud-native, AI-powered time-series database built for IoT, Connected Cars, and Industrial IoT workloads, handling TB-to-PB scale data from billions of sensors.

  16. 19

    Paddle

    24.1KPaddlePaddle/Paddle · C++

    PaddlePaddle is an open-source deep learning platform — China's first independently developed one, open-sourced since 2016 — spanning a core framework, model libraries, end-to-end dev kits and service platforms.

  17. 20

    tornado

    22.2Ktornadoweb/tornado · Python

    Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed. Thanks to non-blocking network I/O it can scale to tens of thousands of open connections.

  18. 21

    serve

    21.9Kjina-ai/serve · Python

    Jina is an open-source Python framework for building multimodal AI applications on a cloud-native stack — you wrap models into gRPC microservices, chain them into flows, and scale from local to cloud.

  19. 22

    shardingsphere

    20.8Kapache/shardingsphere · Java

    An Apache Top-Level Project positioned as 'Database Plus': a layer built over your existing heterogeneous databases that adds distributed SQL - sharding, read/write splitting, encryption - without making you replace them.

  20. 23

    architecture.of.internet-product

    20.8Kdavideuler/architecture.of.internet-product · HTML

    This is a curated Chinese-language collection of real-world technical architecture documents from major internet companies — WeChat/Tencent, Taobao/Alibaba, Meituan-Dianping, Baidu, Qihoo 360, and global players like OpenAI, Google, Facebook, Amazon and eBay.

  21. 24

    ml-engineering

    19.5Kstas00/ml-engineering · Python

    An open book on machine learning engineering: methodologies, tools and step-by-step instructions for training, fine-tuning and running inference on large language models and multi-modal (VLM) models, written by the engineer who led the BLOOM-176B training.

  22. 25

    system-design-resources

    18.5KInterviewReady/system-design-resources

    A curated collection of the best System Design resources on the Internet — one README that organizes engineering writeups, papers and talks into a topic-by-topic reading list.

  23. 26

    VictoriaMetrics

    17.9KVictoriaMetrics/VictoriaMetrics · Go

    VictoriaMetrics is a fast, cost-effective monitoring solution and time series database. It stores metrics long-term and can serve as storage for Prometheus or a drop-in replacement for Prometheus and Graphite in Grafana.

  24. 27

    ceph

    17.1Kceph/ceph · C++

    Ceph is an open-source distributed storage platform: it pools a cluster of servers into one scalable storage system that serves object, block, and file storage at the same time.

  25. 28

    tikv

    16.9Ktikv/tikv · Rust

    An open-source, distributed, transactional key-value database. Unlike traditional NoSQL systems, it offers not only classical key-value APIs but also transactional APIs with ACID compliance.

  26. 29

    open-im-server

    16.7Kopenimsdk/open-im-server · Go

    OpenIM is an open-source instant messaging solution made up of OpenIMServer (backend) and OpenIMSDK (client library), giving developers everything needed to add chat, user management, and group management to their own apps.

  27. 30

    scylladb

    15.8Kscylladb/scylladb · C++

    Scylla is a real-time big data database — a distributed NoSQL data store that is API-compatible with Apache Cassandra and Amazon DynamoDB and built on the Seastar framework.

  28. 31

    hadoop

    15.7Kapache/hadoop · Java

    Apache Hadoop is the open-source big data platform for reliable, distributed storage and processing of very large data sets across clusters of machines.

  29. 32

    ants

    14.5Kpanjf2000/ants · Go

    ants is a goroutine pool library for Go: it manages and recycles a massive number of goroutines with a fixed capacity, letting you limit how many run concurrently in your program.

  30. 33

    ax

    13.9Kgoogle/ax · Go

    AX is Google's open agentic orchestration runtime — a declarative, high-throughput orchestrator that runs billions of autonomous agent workloads in a Kubernetes cluster, sandboxing each task and fencing its network.

  31. 34

    asynq

    13.8Khibiken/asynq · Go

    Asynq is a Go library for queueing tasks and processing them asynchronously with workers, backed by Redis. It distributes work across multiple machines and is designed to be scalable yet easy to get started.

  32. 35

    awesome-sre

    13.7Kdastergon/awesome-sre

    A curated list of Site Reliability Engineering (SRE) and Production Engineering resources — a hand-picked collection of articles, books, talks, podcasts and tools, maintained as a single GitHub README.

  33. 36

    rook

    13.7Krook/rook · Go

    Rook is an open-source storage orchestrator for Kubernetes: it deploys and manages Ceph inside your cluster, turning Kubernetes itself into your storage platform.

  34. 37

    akka-core

    13.3Kakka/akka-core · Scala

    Akka is a platform for building highly responsive, resilient and scalable distributed services; this repo holds its core libraries. It is used for transactional, analytical, agentic-AI, digital-twin, IoT and edge-to-cloud systems.

  35. 38

    codis

    13.2KCodisLabs/codis · Go

    Codis is a proxy-based, high-performance Redis cluster solution written in Go — it sits between your application and a pool of Redis servers. It is production-ready and widely used at wandoujia.com and many other companies.

  36. 39

    citus

    12.8Kcitusdata/citus · C

    Citus is a PostgreSQL extension that turns Postgres into a distributed database, letting you shard tables across a cluster to scale CPU, memory, storage and I/O for large or high-traffic workloads.

  37. 40

    nebula

    12.4Kvesoft-inc/nebula · C++

    NebulaGraph is an open-source distributed graph database built to handle large volumes of highly connected data with millisecond latency and quick horizontal scaling.

  38. 41

    reaction

    12.4Kreactioncommerce/reaction · JavaScript

    Mailchimp Open Commerce (formerly Reaction Commerce) is an open-source, API-first, headless e-commerce platform built on Node.js, MongoDB, and GraphQL for running online shops.

  39. 42

    otp

    12.4Kerlang/otp · Erlang

    Erlang is a programming language and runtime system for building massively scalable soft real-time systems that must stay highly available. This repo ships both the language and OTP.

  40. 43

    garnet

    12.1Kmicrosoft/garnet · C#

    Garnet is a remote cache-store from Microsoft Research: a key-value server that existing Redis clients can talk to, built for high throughput, low latency, persistence, and cluster sharding.

  41. 44

    pomelo

    11.9KNetEase/pomelo · JavaScript

    A fast, scalable game server framework for Node.js, originally built by NetEase. It ships the core framework plus related components — libraries and tools — and also suits realtime web apps.

  42. 45

    jemalloc

    11.2Kjemalloc/jemalloc · C

    jemalloc is a general purpose malloc(3) implementation — a memory allocator for C programs — emphasizing fragmentation avoidance and scalable concurrency support.

  43. 46

    automq

    10.9KAutoMQ/automq · Java

    AutoMQ is a diskless, stateless fork of Apache Kafka that stores all log data on S3 or S3-compatible object storage instead of local disks, cutting cloud costs by up to 10x and removing cross-AZ traffic fees.

  44. 47

    orleans

    10.9Kdotnet/orleans · C#

    Orleans is a cross-platform .NET framework for building robust, scalable distributed applications — from a single on-premises server up to globally distributed, highly available cloud services. It's often referred to as 'Distributed .NET'.

  45. 48

    centrifugo

    10.9Kcentrifugal/centrifugo · Go

    Centrifugo is an open-source, language-agnostic real-time messaging server that lets any backend push messages to connected clients over WebSocket, HTTP-streaming, SSE, GRPC or WebTransport. It acts as a self-hosted alternative to Pubnub, Pusher, Ably, Socket.IO, Phoenix.PubSub and SignalR.

  46. 49

    distribution

    10.7Kdistribution/distribution · Go

    The open source registry implementation for storing and distributing container images and other content — the toolkit to pack, ship, store, and deliver container content, built on the OCI Distribution Specification.

  47. 50

    yugabyte-db

    10.6Kyugabyte/yugabyte-db · C

    YugabyteDB is an open-source, PostgreSQL-compatible distributed SQL database. It combines the transactional correctness of a traditional RDBMS with the horizontal scalability of NoSQL systems, aimed at business-critical cloud-native OLTP apps.

  48. 51

    electric

    10.4Kelectric-sql/electric · TypeScript

    Electric is a real-time sync engine for Postgres: it replicates data out of your database into anything — web apps, mobile clients, AI agents — handling the hard parts like partial replication, fan-out and data delivery.

  49. 52

    modin

    10.4Kmodin-project/modin · Python

    Modin is a drop-in replacement for pandas: you keep the same API and syntax, but your workflows run on all of your CPU cores instead of a single thread. Change one import line and your existing pandas code just runs faster.

  50. 53

    metaflow

    10.3KNetflix/metaflow · Python

    Metaflow is an open-source Python framework for building, managing and deploying real-life AI and ML systems — from quick prototyping in notebooks to reliable production deployments.

  51. 54

    cassandra

    10.1Kapache/cassandra · Java

    Apache Cassandra is an open source, transactional distributed database — a highly-scalable partitioned row store built for linear scalability and proven fault-tolerance on commodity hardware or cloud infrastructure.

  52. 55

    Smartstore

    1.7Ksmartstore/Smartstore · C#

    Smartstore is a modular, open-source all-in-one eCommerce platform built on ASP.NET Core 10, offering a full storefront plus backend for managing a multi-language, multi-store, multi-currency online shop.

  53. 56

    MetaBridgeDiamond

    114minseochoj/MetaBridgeDiamond · PHP

    MetaBridgeDiamond is a lightweight tool described as enabling high-performance real-time data processing and auto-scaling, deploying scalable agents as a 'robust manager'.

  54. 57

    BloomBeacon

    83vinamega/BloomBeacon · Python

    BloomBeacon is a Python tool for real-time data processing and auto-scaling of cloud-based systems, built to be cloud-agnostic rather than tied to one provider.

Often tagged together