Topics / #compression

Compression projects on GitHub

Compression techniques for data, media or model sizes.

As of , the most-starred Compression projects on GitHub are rtk (83.8K stars), OmniRoute (78.1K stars) and headroom (75.9K stars). GitHub.Locker tracks 48 Compression repositories with 924.2K stars combined. The fastest-growing on October 18 was OmniRoute, with +428 stars in a day.

48Projects
924.2KStars in total
+1.2KStars in a day
Browse in the catalog →github.com/topics/compression ↗

Hot right now

Most stars gained in a day · Oct 18

+428

OmniRoute

diegosouzapw/OmniRoute · ★ 78.1K

OmniRoute is a free, open-source AI gateway that puts 359 providers and 1200+ models (Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax, etc.) behind one API endpoint, with quota-aware automatic fallback when a provider runs out.

+338

headroom

headroomlabs-ai/headroom · ★ 75.9K

Headroom compresses everything an AI agent reads - tool outputs, logs, RAG chunks, files, and conversation history - before it reaches the LLM, cutting tokens while keeping the same answers. It ships as a library, a local proxy, and an MCP server.

+134

billion-context

ranxianglei/billion-context · ★ 1.9K

billion-context is a context-compression proxy that sits between any coding agent and its model API, rewriting Anthropic/OpenAI streams so a 100K context window is enough and a single session can run for months across billions of tokens.

+114

rtk

rtk-ai/rtk · ★ 83.8K

rtk is a CLI proxy that filters and compresses command outputs before they reach your LLM context, cutting 60-90% of the bash output your coding agent reads and lowering token costs.

The all-time top

48 projects · by stars

  1. #1 · rtk-ai/rtk

    rtk

    rtk is a CLI proxy that filters and compresses command outputs before they reach your LLM context, cutting 60-90% of the bash output your coding agent reads and lowering token costs.

    83.8K
  2. #2 · diegosouzapw/OmniRoute

    OmniRoute

    OmniRoute is a free, open-source AI gateway that puts 359 providers and 1200+ models (Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax, etc.) behind one API endpoint, with quota-aware automatic fallback when a provider runs out.

    78.1K
  3. #3 · headroomlabs-ai/headroom

    headroom

    Headroom compresses everything an AI agent reads - tool outputs, logs, RAG chunks, files, and conversation history - before it reaches the LLM, cutting tokens while keeping the same answers. It ships as a library, a local proxy, and an MCP server.

    75.9K
  1. 4

    ripgrep

    69.2KBurntSushi/ripgrep · Rust

    ripgrep (rg) is a line-oriented command-line tool that recursively searches directories for a regex pattern, automatically respecting .gitignore rules and skipping hidden and binary files by default.

  2. 5

    DeepSpeed

    43.2Kdeepspeedai/DeepSpeed · Python

    DeepSpeed is a deep learning optimization library that makes distributed training and inference of very large models easy, efficient and effective — it powered models like MT-530B and BLOOM.

  3. 6

    leveldb

    39.5Kgoogle/leveldb · C++

    LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values, supporting Put, Get, Delete and range iteration.

  4. 7

    restic

    36.6Krestic/restic · Go

    restic is a command-line backup program that is fast, secure, and efficient, supporting Linux, macOS, Windows, FreeBSD, and OpenBSD.

  5. 8

    CyberChef

    36.2Kgchq/CyberChef · JavaScript

    CyberChef is a web app that bundles encoding, encryption, compression, hashing and other data-analysis operations ('cyber' operations) into a single drag-and-drop recipe builder that runs entirely in your browser.

  6. 9

    stb

    34.9Knothings/stb · C

    A collection of 21 single-file libraries for C/C++ covering image loading, writing and resizing, font rasterizing, audio decoding, containers and more — all released into the public domain, with an MIT option if you need a regular license.

  7. 10

    zstd

    28.1Kfacebook/zstd · C

    Zstandard (zstd) is a fast lossless compression algorithm for real-time scenarios, at zlib-level ratios or better. This repo is the reference C implementation plus a CLI; the format is stable and documented in RFC8878.

  8. 11

    squoosh

    26KGoogleChromeLabs/squoosh · TypeScript

    Squoosh is a web app from Google Chrome Labs that compresses images into various formats directly in the browser, letting you shrink file sizes without uploading anything to a server.

  9. 12

    actix-web

    24.9Kactix/actix-web · Rust

    Actix Web is a powerful, pragmatic and extremely fast web framework for Rust, used to build web servers, APIs and realtime services.

  10. 13

    timescaledb

    23.7Ktimescale/timescaledb · C

    TimescaleDB is a PostgreSQL extension for high-performance real-time analytics on time-series and event data — sensor readings, financial ticks, application events. You work with it in regular SQL.

  11. 14

    ws

    22.8Kwebsockets/ws · JavaScript

    ws is a simple to use, blazing fast, thoroughly tested WebSocket client and server implementation for Node.js, with support for the permessage-deflate compression extension.

  12. 15

    upx

    18Kupx/upx · C++

    UPX (Ultimate Packer for eXecutables) is a command-line tool that compresses executable files and DLLs, typically shrinking their size by 50-70% while keeping them runnable without any runtime penalty for most formats.

  13. 16

    VictoriaMetrics

    17.9KVictoriaMetrics/VictoriaMetrics · Go

    VictoriaMetrics is a fast, cost-effective monitoring solution and time series database. It stores metrics long-term and can serve as storage for Prometheus or a drop-in replacement for Prometheus and Graphite in Grafana.

  14. 17

    digital_video_introduction

    16.4Kleandromoreira/digital_video_introduction · Jupyter Notebook

    A free, hands-on guide that teaches digital video concepts in plain language: how images and color work, how video compression and codecs (H.264, H.265, VP9, AV1) function, and how online streaming works.

  15. 18

    NanaZip

    15.9KM2Team/NanaZip · C++

    An open source file archiver forked from the source code of 7-Zip, rebuilt for the modern Windows experience with a Fluent-style UI, MSIX packaging and extra codecs and hash algorithms.

  16. 19

    httpx

    15.6Kencode/httpx · Python

    HTTPX is a fully featured HTTP client library for Python 3, with a broadly requests-compatible API, HTTP/1.1 and HTTP/2 support, and both sync and async interfaces. It also ships with an integrated command-line client.

  17. 20

    FileConverter

    15.4KTichau/FileConverter · C#

    File Converter is a Windows tool that lets you convert and compress files directly from the right-click context menu in Windows Explorer, without opening a separate app.

  18. 21

    duplicati

    15.1Kduplicati/duplicati · C#

    Duplicati is a free, open-source backup client that stores encrypted, incremental, and compressed backups on cloud storage services and remote file servers.

  19. 22

    brotli

    14.9Kgoogle/brotli · TypeScript

    Brotli is Google's general-purpose lossless compression algorithm, combining a modern LZ77 variant, Huffman coding and 2nd order context modeling. It runs at about deflate's speed but compresses denser.

  20. 23

    libsodium

    14Kjedisct1/libsodium · C

    Libsodium is an easy-to-use software library providing cryptographic operations like encryption, decryption, digital signatures, and secure password hashing. It is a portable, cross-compilable fork of NaCl that keeps API compatibility while improving usability.

  21. 24

    borg

    13.9Kborgbackup/borg · Python

    BorgBackup is a deduplicating backup program that supports optional compression and authenticated encryption. It's designed to let you run efficient, secure daily backups since it only stores changed data.

  22. 25

    PaddleNLP

    13KPaddlePaddle/PaddleNLP · Python

    PaddleNLP is a large language model (LLM) and small language model (SLM) development toolkit built on the PaddlePaddle deep learning framework, offering efficient training, lossless compression, and high-performance inference for large models.

  23. 26

    redux-persist

    12.9Kendograph/redux-persist · TypeScript

    Redux Persist is a library that saves (persists) your Redux store to storage and automatically rehydrates it when your app reloads, so state survives page refreshes or app restarts.

  24. 27

    guetzli

    12.9Kgoogle/guetzli · C++

    Guetzli is a perceptual JPEG encoder from Google that produces JPEG files 20-30% smaller than libjpeg at the same visual quality by optimizing for human perception instead of simple pixel metrics.

  25. 28

    RxTool

    12.3KTamsiree/RxTool · Kotlin

    A toolbox of utility classes for everyday Android development: payments, QR/barcode scanning, dialogs, GPS location, compression and a stack of custom views, packaged as a Kotlin library.

  26. 29

    lz4

    12.2Klz4/lz4 · C

    LZ4 is an open-source lossless compression algorithm and its reference C implementation, built for extreme speed: 500+ MB/s compression per core, a multi-GB/s decoder, scaling across CPU cores.

  27. 30

    wcdb

    11.6KTencent/wcdb · C

    WCDB is a cross-platform database framework built by WeChat on top of SQLite and SQLCipher, providing ORM, encryption, corruption recovery and other database features for mobile and desktop apps.

  28. 31

    imgproxy

    11.1Kimgproxy/imgproxy · Go

    imgproxy is a fast, standalone server that resizes, crops, converts, and optimizes images on the fly by fetching them from a source URL you provide.

  29. 32

    jszip

    10.4KStuk/jszip · JavaScript

    JSZip is a JavaScript library for creating, reading, and editing .zip files, usable in both the browser and Node.js.

  30. 33

    warp

    10.4Kseanmonstar/warp · Rust

    warp is a super-easy, composable web server framework for Rust, promising 'warp speeds'.

  31. 34

    phpredis

    10.2Kphpredis/phpredis · C

    A PHP extension that gives PHP a fast API for talking to the Redis key-value store. It also works with Valkey, Dragonfly, and KeyDB, and is released under the PHP License 3.01.

  32. 35

    node-mongodb-native

    10.2Kmongodb/node-mongodb-native · TypeScript

    The official MongoDB driver for Node.js — a client library that Node.js apps use to connect to a MongoDB server and run queries, CRUD operations and index management against it.

  33. 36

    parameter-golf

    5.2Kopenai/parameter-golf · Python

    This is OpenAI's 'Model Craft Challenge: Parameter Golf' - a competition to train the best language model that fits in a 16MB artifact and trains in under 10 minutes on 8xH100 GPUs, scored by bits-per-byte compression on the FineWeb validation set.

  34. 37

    SimpleMem

    3.8Kaiming-lab/SimpleMem · Python

    SimpleMem is a lifelong memory stack for LLM agents that stores, compresses, and retrieves long-term memories across text, image, audio, and video, so agents keep context without reprocessing redundant history.

  35. 38

    billion-context

    1.9Kranxianglei/billion-context · TypeScript

    billion-context is a context-compression proxy that sits between any coding agent and its model API, rewriting Anthropic/OpenAI streams so a 100K context window is enough and a single session can run for months across billions of tokens.

  36. 39

    MacPacker

    1.3Ksarensw/MacPacker · Swift

    MacPacker is a free, open source archive manager for macOS that lets you browse archives like folders, preview files without extracting, and drag out only the files you need. It's meant as a native 7-Zip replacement for the Mac.

  37. 40

    libdragon

    1.3KDragonMinded/libdragon · C

    Libdragon is an open-source SDK for homebrew Nintendo 64 development, giving you a modern C11 toolchain plus graphics, audio, filesystem and debugging libraries to build real N64 ROMs.

  38. 41

    marisa-trie

    1.1Kpytries/marisa-trie · Cython

    Static, memory-efficient trie-like data structures for Python, built on the marisa-trie C++ library. Strings stored inside can take up to 50x-100x less memory than in a standard Python dict, with comparable lookup speed and fast prefix search.

  39. 42

    pyrowave

    767Themaister/pyrowave · C++

    An intra-only video codec — effectively a still-image codec — that runs entirely on the GPU as Vulkan compute shaders: under ~0.1 ms per frame at 1080p and ~0.2 ms at 4K. It is built for local-network game streaming.

  40. 43

    Arachnidium

    398p2r3/Arachnidium · Python

    Arachnidium is a self-hosted HTTP(S) optimization and compression proxy powered by mitmproxy, meant to run on a spare computer at home and be reached by your devices through a WireGuard VPN tunnel.

  41. 44

    TurboBench

    374powturbo/TurboBench · C++

    TurboBench is a data compression benchmark: a single executable that tests 100+ compressors in memory and shows which ones win on speed and compression ratio.

  42. 45

    steam2_downloader

    229extremebleem/steam2_downloader · C#

    A desktop browser and downloader for the 'terarelease' Steam2 content leak - a dump of 10,876 depots and 116,339 files totaling 13.3 TB. It shows what the archive holds, works out which files a given depot version actually needs, downloads them, verifies them with sha256, and unpacks them.

  43. 46

    squeez

    217claudioemmanuel/squeez · Rust

    Squeez is a hook-based token compressor for seven AI coding CLI hosts (Claude Code, Copilot CLI, OpenCode, Gemini CLI, Codex CLI, Pi, Hermes) that intercepts tool output and shrinks bash results by up to 95% without losing exact identifiers.

  44. 47

    dsh-undo-savepoint

    181lire1131/dsh-undo-savepoint · JavaScript

    dsh-undo-savepoint is a crash-rescue plugin for DeepSeek Harness (DSH) that snapshots your config and plugin code so you can undo, redo, or roll back to any version, plus an offline CLI/GUI/WebUI that keeps working even when DSH itself won't boot.

  45. 48

    applesauce

    170Dr-Emann/applesauce · Rust

    Applesauce is a Rust command-line tool that compresses, decompresses, and inspects files using macOS's transparent HFS+/APFS file compression — a modernized replacement for the older afsctool.

Often tagged together