DeepSpec
DeepSpec is a full-stack codebase for training and evaluating draft models for speculative decoding. It contains data preparation utilities, draft model implementations, training code, and evaluation scripts.
Environment
Install the Python dependencies:
python -m pip install -r requirements.txt
Data preparation additionally requires an inference engine to serve the target model when regenerating answers; see scripts/data/README.md for details.
Workflow
Run the stages in order — each stage's output feeds the next:
- Data Preparation — download prompts, regenerate target answers, and build the target cache.
- Training — train a draft model against the cached target outputs.
- Evaluation — measure speculative-decoding acceptance on benchmark tasks.
Data Preparation
See scripts/data/README.md for the step-by-step data pipeline:
- download and split training data,
- regenerate answers,
- prepare the target cache (storage warning: this can be very large — roughly 38 TB for the default
Qwen/Qwen3-4B setting).
Training
bash scripts/train/train.sh
train.sh launches train.py, which spawns one worker per visible GPU. Select the algorithm and target model by pointing config_path at one of the configs under config/ (e.g. config/dspark/dspark_qwen3_4b.py); see the script header for the full list of configs, how to override config_path / target_cache_dir, and how to use --opts to override individual config fields. Checkpoints are written to ~/checkpoints/<project_name>/<exp_name>/step_*.
Hardware: the default configs and scripts assume a single node with 8 GPUs. For fewer GPUs, reduce CUDA_VISIBLE_DEVICES.
Evaluation
bash scripts/eval/eval.sh
eval.sh runs eval.py against a trained draft checkpoint over the speculative-decoding benchmarks in eval_datasets/ (gsm8k, math500, aime25, humaneval, mbpp, livecodebench, mt-bench, alpaca, arena-hard-v2). Set:
target_name_or_path — the target model the draft was trained against (e.g. Qwen/Qwen3-4B),draft_name_or_path — the draft checkpoint, e.g. ~/checkpoints/deepspec/dspark_block7_qwen3_4b/step_latest, or one of the Hugging Face repo IDs listed in Released Checkpoints.
Released Checkpoints
The checkpoints below are the ones used for Table 1 in the paper. Each checkpoint was trained on open-perfectblend data generated by its corresponding target model in non-thinking mode, and is the direct output of the corresponding training configuration under config/.
[!IMPORTANT]
If you cite these results in a new paper, align your setup with the training settings in this repository; otherwise, the comparison is not meaningful. For domain-specific use, fine-tune the draft model again for better results, especially if the target model is expected to run in thinking mode.
Supported Algorithms
Currently, DeepSpec includes three draft models: DSpark, DFlash and Eagle3.
License
DeepSpec is released under the MIT License. It includes code adapted
from third-party projects under their own licenses; see NOTICE for the
full attribution.
Acknowledgements
DeepSpec builds on the ideas and code of several excellent open-source projects:
- SpecForge (Apache-2.0) — the overall training framework and Eagle3 implementation; portions of the Eagle3 modeling, loss, optimizer, attention, and evaluation code are adapted from it. Adapted files carry an in-file attribution comment, and the full notice is recorded in NOTICE.
- DFlash (MIT) — the DFlash draft-model design and training recipe.
- Qwen3 and Gemma — the target model families supported in this repo.
We thank the authors and maintainers of these projects. Contributions of new algorithms are welcome.
Citation
@misc{cheng2026dsparkconfidencescheduledspeculativedecoding,
title={DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation},
author={Xin Cheng and Xingkai Yu and Chenze Shao and Jiashi Li and Yunfan Xiong and Yi Qian and Jiaqi Zhu and Shirong Ma and Xiaokang Zhang and Jiasheng Ye and Qinyu Chen and Chengqi Deng and Jiping Yu and Damai Dai and Zhengyan Zhang and Yixuan Wei and Yixuan Tan and Wenkai Yang and Runxin Xu and Yu Wu and Zhean Xu and Xuanyu Wang and Muyang Chen and Rui Tian and Xiao Bi and Zhewen Hao and Shaoyuan Chen and Huanqi Cao and Wentao Zhang and Anyi Xu and Huishuai Zhang and Dongyan Zhao and Wenfeng Liang},
year={2026},
eprint={2607.05147},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2607.05147},
}
DeepSpec 是 DeepSeek 的一套全栈 Python 代码库，用于训练和评估投机解码（speculative decoding）的草稿模型——这是一种由小草稿模型加速更大目标大语言模型的技术。