What is this?
DFlash is a lightweight block diffusion model built for speculative decoding, drafting tokens in parallel blocks so a larger target LLM can verify them faster than generating one token at a time.
DFlash is a lightweight block diffusion model built for speculative decoding, drafting tokens in parallel blocks so a larger target LLM can verify them faster than generating one token at a time.Is DFlash a standalone model I chat with, or does it plug into an existing LLM server?It is a draft-model package that plugs into an existing serving setup (Transformers, MLX, or an OpenAI-compatible SGLang/vLLM server) to speed up generation from a target LLM, not a chatbot on its own.
It is a draft-model package that plugs into an existing serving setup (Transformers, MLX, or an OpenAI-compatible SGLang/vLLM server) to speed up generation from a target LLM, not a chatbot on its own.How do I get it running?Install the base package for use with an OpenAI-compatible server, or install with local inference dependencies, which use MLX on Apple Silicon and Transformers on Linux; for serving benchmarks you separately install a supported SGLang, vLLM, oMLX, or llama.cpp build and point DFlash at its --base-url.
Install the base package for use with an OpenAI-compatible server, or install with local inference dependencies, which use MLX on Apple Silicon and Transformers on Linux; for serving benchmarks you separately install a supported SGLang, vLLM, oMLX, or llama.cpp build and point DFlash at its --base-url.Which models and backends does DFlash actually support?Public checkpoints cover Qwen, Gemma, MiniMax, Kimi, GPT-OSS, Llama-3.1-8B, GLM, and Alpamayo, and DFlash runs through the Transformers or MLX backends for those families, or through any SGLang/vLLM OpenAI-compatible server for other checkpoints.
Public checkpoints cover Qwen, Gemma, MiniMax, Kimi, GPT-OSS, Llama-3.1-8B, GLM, and Alpamayo, and DFlash runs through the Transformers or MLX backends for those families, or through any SGLang/vLLM OpenAI-compatible server for other checkpoints.Any gotchas when using quantized models?On MLX, quantized target or draft models should use a block size of 5 or less, because the current quantized matmul kernel gets less efficient at larger verify widths.
On MLX, quantized target or draft models should use a block size of 5 or less, because the current quantized matmul kernel gets less efficient at larger verify widths.Does it work the same way on Linux and Mac?Local inference is split by platform: Transformers is used on Linux and MLX is used on Apple Silicon Macs, both supported through the same local install path.
Local inference is split by platform: Transformers is used on Linux and MLX is used on Apple Silicon Macs, both supported through the same local install path.How is DFlash evaluated?Benchmarks run through an OpenAI-compatible SGLang/vLLM server, Transformers, or MLX, all sharing the same datasets: gsm8k, math500, humaneval, mbpp, and mt-bench, cached via Hugging Face Datasets.
Benchmarks run through an OpenAI-compatible SGLang/vLLM server, Transformers, or MLX, all sharing the same datasets: gsm8k, math500, humaneval, mbpp, and mt-bench, cached via Hugging Face Datasets.What's new in DFlash 2?DFlash 2 adds checkpoints for Muse-Glimmer-30B and Qwen3.8-27B with configurable reasoning intensity (reasoningstrength for Muse, reasoningeffort for Qwen3.8), running on the Transformers and MLX backends.
DFlash 2 adds checkpoints for Muse-Glimmer-30B and Qwen3.8-27B with configurable reasoning intensity (reasoningstrength for Muse, reasoningeffort for Qwen3.8), running on the Transformers and MLX backends.Who helped bring DFlash to popular serving engines?The DFlash team credits engineers from Modal Labs for integrating DFlash into SGLang and an NVIDIA contributor for bringing it to vLLM.
The DFlash team credits engineers from Modal Labs for integrating DFlash into SGLang and an NVIDIA contributor for bringing it to vLLM.Which topics is it in?gpt-oss, gemma, minimax, sglang, glm, kimi, mlx, speculative-decoding, model-serving, llama, transformers, vllm
gpt-oss, gemma, minimax, sglang, glm, kimi, mlx, speculative-decoding, model-serving, llama, transformers, vllm