Topics / #inference-optimization

Inference-optimization projects on GitHub

Engineering LLM inference for lower latency and cost.

As of , the most-starred Inference-optimization projects on GitHub are HyperQwen (2.1K stars), llm-inference-engineering (372 stars) and ComfyUI-H3VAE_TRT (253 stars). GitHub.Locker tracks 6 Inference-optimization repositories with 3.3K stars combined. The fastest-growing on October 18 was HyperQwen, with +41 stars in a day.

6Projects
3.3KStars in total
+67Stars in a day
Browse in the catalog →github.com/topics/inference-optimization ↗

Hot right now

Most stars gained in a day · Oct 18

+41

HyperQwen

syv-ai/HyperQwen · ★ 2.1K

HyperQwen is a self-hosted serving setup that runs the Qwen3.8-27B model on a single 24 GB consumer GPU (like an RTX 3090) using vLLM, exposing it through an OpenAI-compatible API.

+21

llm-inference-engineering

amitshekhariitbhu/llm-inference-engineering · ★ 372

A free, continuously growing blog series that teaches LLM inference engineering step by step, covering KV cache, PagedAttention, continuous batching, speculative decoding, and serving engines like vLLM, SGLang, TensorRT-LLM, and GGUF.

+3

Hyperloom

AMD-AGI/Hyperloom · ★ 242

A multi-agent harness that autonomously optimizes LLM inference on AMD Instinct GPUs: it profiles your workload, searches framework and GPU-kernel optimizations, validates every candidate end to end, and saves proven results into a recipe knowledge base.

+1

ComfyUI-H3VAE_TRT

lihaoyun6/ComfyUI-H3VAE_TRT · ★ 253

A ComfyUI custom node that runs the MiniMax-H3 VAE as a compiled TensorRT engine instead of the regular one, making it up to 1.7x faster.

The all-time top

6 projects · by stars

  1. #1 · syv-ai/HyperQwen

    HyperQwen

    HyperQwen is a self-hosted serving setup that runs the Qwen3.8-27B model on a single 24 GB consumer GPU (like an RTX 3090) using vLLM, exposing it through an OpenAI-compatible API.

    2.1K
  2. #2 · amitshekhariitbhu/llm-inference-engineering

    llm-inference-engineering

    A free, continuously growing blog series that teaches LLM inference engineering step by step, covering KV cache, PagedAttention, continuous batching, speculative decoding, and serving engines like vLLM, SGLang, TensorRT-LLM, and GGUF.

    372
  3. #3 · lihaoyun6/ComfyUI-H3VAE_TRT

    ComfyUI-H3VAE_TRT

    A ComfyUI custom node that runs the MiniMax-H3 VAE as a compiled TensorRT engine instead of the regular one, making it up to 1.7x faster.

    253
  1. 4

    Hyperloom

    242AMD-AGI/Hyperloom · Python

    A multi-agent harness that autonomously optimizes LLM inference on AMD Instinct GPUs: it profiles your workload, searches framework and GPU-kernel optimizations, validates every candidate end to end, and saves proven results into a recipe knowledge base.

  2. 5

    dgx-spark-qwen38

    230hasso5703/dgx-spark-qwen38 · Python

    A one-command installer that sets up a hardened, boot-persistent SGLang serving stack for the Qwen3.8 model family on a single NVIDIA DGX Spark (GB10), reaching 50 tok/s greedy median and up to 1M context on the 27B NVFP4 config, with a 176B Flash-Next MoE model also fitting on one box.

  3. 6

    nari-qwen3-tts

    187nari-labs/nari-qwen3-tts · Python

    Nari Qwen3-TTS is a high-performance serving implementation of the Qwen3-TTS 1.7B CustomVoice model, built to run on a single NVIDIA H100 and exposed as an API with streaming and non-streaming speech generation.

Often tagged together