dflash README

The project's README from github.com/z-lab/dflash. As one markdown file: README.md.

DFlash: Block Diffusion for Flash Speculative Decoding

DFlash is a lightweight block diffusion model designed for speculative decoding. It enables efficient and high-quality parallel drafting.

DFlash 2

Blog | Models

DFlash 2 architecture

https://github.com/user-attachments/assets/f786e7c5-c2bc-47d4-8a32-1f730a689e1b

DFlash

Paper | Blog | Models

DFlash architecture

https://github.com/user-attachments/assets/5b29cabb-eb95-44c9-8ffe-367c0758de8c

Supported Models

DFlash 2

Available checkpoints: Muse-Glimmer-30B and Qwen3.8-27B. See the DFlash 2 collection for updates.

DFlash

Public checkpoints are available in the DFlash collection:

Use the Transformers or MLX backends below for their explicitly listed model families. Other checkpoints can be benchmarked through an OpenAI-compatible SGLang or vLLM server.

📦 Installation

Install the base package for an OpenAI-compatible server, or include local inference dependencies. The local install uses MLX on Apple Silicon and Transformers on Linux.

pip install dflash
pip install "dflash[local]"  # local inference

For serving benchmarks, install a supported version of SGLang, vLLM, oMLX, or llama.cpp separately, launch its OpenAI-compatible server with DFlash, and pass its --base-url below.

🚀 Quick Start

Transformers

The Transformers backend supports DFlash 2 for Muse-Glimmer-30B, and DFlash for Qwen3 and LLaMA-3.1-8B. Muse uses reasoning_strength: low, medium, high (default), or xhigh.

dflash generate transformers \
    --model meta-models/Muse-Glimmer-30B \
    --draft z-lab/Muse-Glimmer-30B-DFlash2 \
    --reasoning high --temperature 1 --top-p 0.95 --top-k 64 \
    "How many positive whole-number divisors does 196 have?"

MLX (Apple Silicon)

The MLX backend supports DFlash 2 for Qwen3.8-27B, and DFlash for Qwen3, Qwen3.5, Qwen3.6, and Gemma 4. Qwen3.8 uses reasoning_effort: low, medium, or xhigh (default). For quantized targets or drafts, use block_size <= 5: MLX's current quantized matmul kernel becomes less efficient at larger verify widths. The example below runs both the target and draft with 4-bit weights.

dflash generate mlx \
    --model mlx-community/Qwen3.8-27B-4bit \
    --draft z-lab/Qwen3.8-27B-DFlash2 \
    --draft-bits 4 --block-size 5 --reasoning xhigh \
    "How many positive whole-number divisors does 196 have?"

OpenAI-compatible server

Launch the latest SGLang or vLLM server separately, then run:

dflash generate openai \
    --base-url http://127.0.0.1:8000 --model Qwen/Qwen3.8-27B \
    "How many positive whole-number divisors does 196 have?"

📊 Evaluation

All benchmarks share the same datasets (gsm8k, math500, humaneval, mbpp, mt-bench), downloaded and cached by Hugging Face Datasets.

OpenAI-compatible server (SGLang or vLLM):

dflash benchmark openai \
    --base-url http://127.0.0.1:8000 --model Qwen/Qwen3.8-27B \
    --dataset gsm8k --num-prompts 128 --concurrency 1 --reasoning xhigh \
    --temperature 1 --top-p 0.95 --top-k 20

Transformers (Muse-Glimmer-30B DFlash 2):

dflash benchmark transformers \
    --model meta-models/Muse-Glimmer-30B --draft z-lab/Muse-Glimmer-30B-DFlash2 \
    --dataset gsm8k --max-samples 128 --reasoning high

MLX (Qwen3.8-27B 4-bit DFlash 2):

dflash benchmark mlx \
    --model mlx-community/Qwen3.8-27B-4bit --draft z-lab/Qwen3.8-27B-DFlash2 \
    --dataset gsm8k --max-samples 128 --reasoning xhigh --block-size 5 --draft-bits 4

Acknowledgement

Huge thanks to @dcw02, @gongy, and the team at @modal-labs for their fast, high-quality support in bringing DFlash to SGLang. And huge thanks as well to @benchislett at NVIDIA for his work in bringing DFlash to vLLM and helping make it available to the broader serving community.

Citation

If you find DFlash useful, please cite our work. To share feedback on DFlash or request new model support, please fill out this form: DFlash Feedback.

@article{chen2026dflash,
  title   = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
  author  = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  journal = {arXiv preprint arXiv:2602.06036},
  year    = {2026}
}

@misc{inco2026dflash2,
  title  = {DFlash 2: Keep Drafting Parallel},
  author = {{Inco AI}},
  year   = {2026},
  month  = {August},
  url    = {https://inco.ai/blog/dflash2/}
}

dflash (z-lab/dflash)

Searching Database…

6.2K ·GitHub
Dflash
这是什么？DFlash 是一个轻量级的块扩散模型，专为推测解码设计。它以并行的块为单位起草 token，让较大的目标大模型能比逐个生成 token 更快地完成验证。
DFlash 是我可以直接聊天的独立模型，还是要接入现有的大模型服务？它是一个草稿模型包，可以接入现有的推理服务（Transformers、MLX，或兼容 OpenAI 的 SGLang/vLLM 服务），以加快目标大模型的生成速度。它本身并不是聊天机器人。
我该怎么把它跑起来？可以只安装基础包，用于兼容 OpenAI 的服务器；也可以安装本地推理依赖，其中 Apple Silicon 上使用 MLX，Linux 上使用 Transformers。如果要做服务基准测试，需要另外安装受支持的 SGLang、vLLM、oMLX 或 llama.cpp 版本，并将 DFlash 的 --base-url 指向它。
DFlash 到底支持哪些模型和后端？公开的检查点涵盖 Qwen、Gemma、MiniMax、Kimi、GPT-OSS、Llama-3.1-8B、GLM 和 Alpamayo。对于这些模型系列，DFlash 通过 Transformers 或 MLX 后端运行；对于其他检查点，则通过任何兼容 OpenAI 的 SGLang/vLLM 服务运行。
用量化模型时有什么需要注意的吗？在 MLX 上，量化的目标模型或草稿模型应将块大小设为 5 或更小，因为当前的量化矩阵乘法内核在验证宽度较大时效率会降低。
在 Linux 和 Mac 上用法一样吗？本地推理按平台区分：Linux 使用 Transformers，Apple Silicon 版 Mac 使用 MLX，两者都通过同一条本地安装路径支持。
DFlash 是怎么评测的？基准测试通过兼容 OpenAI 的 SGLang/vLLM 服务、Transformers 或 MLX 运行，它们共用同样的数据集：gsm8k、math500、humaneval、mbpp 和 mt-bench，并通过 Hugging Face Datasets 缓存。
DFlash 2 有什么新东西？DFlash 2 新增了 Muse-Glimmer-30B 和 Qwen3.8-27B 的检查点，并支持可配置的推理强度（Muse 使用 reasoningstrength，Qwen3.8 使用 reasoningeffort），可在 Transformers 和 MLX 后端运行。
是谁帮 DFlash 接入主流推理引擎的？DFlash 团队感谢 Modal Labs 的工程师将 DFlash 集成到 SGLang 中，也感谢一位 NVIDIA 的贡献者将其引入 vLLM。
What are similar projects?ms-swift, NeoHorse, vllm, airllm, jeff, sglang, jeeves, unsloth
Which topics is it in?gpt-oss, gemma, minimax, sglang, glm, kimi, mlx, speculative-decoding, model-serving, llama, transformers, vllm
Creator
z-lab
Size
1.1 MB
Stars
6.2K
Updated
CommitAug 18Releasev0.1.0Aug 18
Age
9mo
Commits · 90 Days