Qwen3-1.7B-DSpark: A DSpark draft model for Qwen3-1.7B (upto 3x speedup; 220 --> 646 tokens/sec)
Qwen3-1.7B-DSpark
Model Link: https://huggingface.co/rasyosef/Qwen3-1.7B-DSpark
A DSpark draft model for speculative decoding with Qwen/Qwen3-1.7B as the verifier, trained with speculators. The drafter proposes 8 tokens at a time and the verifier checks them in one forward pass, so output is identical to running the verifier alone — a lossless speedup. Mean acceptance length is 3.40 tokens committed per verification step, up to 4.52 on math_reasoning. That gives a throughput speedup over the verifier alone of 2.93× on math_reasoning and 2.43× on HumanEval.
Training code: rasyosef/train-dspark-draft-models.
Trained on 80,000 samples.
Usage
vLLM loads the verifier automatically from the config — don't pass it separately.
vllm serve rasyosef/Qwen3-1.7B-DSpark --port 8000 --gpu-memory-utilization 0.8
Then query the OpenAI-compatible endpoint at http://localhost:8000/v1.
Details
4 Qwen3 layers (hidden size 2048, intermediate size 6144, 16 attention heads over 8 KV heads, head dim 128). The first three layers use sliding-window attention with a 2048-token window and the last layer uses full attention. Block size 8, aux hidden-state layers 2/10/18/26.
Trained for 3 epochs at lr 3e-4 (cosine schedule with 4% warmup) on 80,000 Open PerfectBlend prompts regenerated by the verifier itself, split 96/4 into train and validation, with a {"ce": 0.1, "tv": 0.9} loss. Prompts prepared at 2048 tokens; training sequence length 8192, up to 1024 anchors per sample. Verifier hidden states were pulled on demand from a running vLLM server during training and deleted after use rather than staged to disk up front.
Evaluation
evaluate.py throughput across the nine RedHatAI/speculator_benchmarks subsets. acceptance_length is mean tokens committed per verification step, including the bonus token — floor 1.0, ceiling 9.0 at block size 8.
| subset | acceptance_length | pos_0 | pos_1 | pos_2 | pos_3 | pos_4 | pos_5 | pos_6 | pos_7 |
|---|---|---|---|---|---|---|---|---|---|
| math_reasoning | 4.523 | 83.7% | 68.8% | 55.9% | 44.8% | 35.2% | 27.5% | 20.9% | 15.4% |
| HumanEval | 3.807 | 80.0% | 61.4% | 45.8% | 33.5% | 24.0% | 16.8% | 11.5% | 7.7% |
| writing | 3.211 | 71.7% | 49.8% | 34.5% | 24.1% | 16.8% | 11.3% | 7.7% | 5.1% |
| question | 3.159 | 71.5% | 49.1% | 33.9% | 23.3% | 15.8% | 10.6% | 7.1% | 4.5% |
| rag | 3.040 | 71.5% | 49.2% | 33.2% | 21.6% | 13.4% | 7.9% | 4.6% | 2.6% |
| translation | 2.977 | 71.6% | 49.4% | 32.2% | 19.5% | 11.8% | 7.0% | 3.9% | 2.3% |
| tool_call | 2.959 | 70.3% | 47.1% | 30.7% | 19.7% | 12.7% | 7.8% | 4.7% | 2.7% |
| qa | 2.872 | 68.5% | 45.0% | 29.0% | 18.8% | 11.7% | 7.3% | 4.4% | 2.4% |
| summarization | 2.405 | 63.1% | 36.6% | 20.1% | 10.4% | 5.4% | 2.6% | 1.4% | 0.8% |
Weighted across all subsets: 3.397 over 392,118 verification steps.
Acceptance is highest where the verifier's next token is most predictable — math and code. math_reasoning leads HumanEval by about 0.7 tokens and holds a margin at every position in the block; math_reasoning's pos_7 (15.4%) is double HumanEval's (7.7%), and its pos_3 (44.8%) is above summarization's pos_1 (36.6%). Unlike many drafters, tool_call does not stand apart here: at 2.96 it sits inside the prose-like group rather than above it.
Throughput (tokens/s)
Output throughput (tokens/s) across the same nine subsets. Baseline is Qwen3-1.7B with no speculative decoding. Without a drafter, throughput doesn't depend on the task (220.5–223.4 tokens/s across HumanEval, math_reasoning, and qa), so for the other six subsets the baseline is estimated as the mean of those three runs, 221.6 tokens/s (marked *). Speedup is the mean throughput relative to baseline.
| subset | baseline (no drafter) | DSpark (this model) | speedup |
|---|---|---|---|
| math_reasoning | 220.9 | 646.7 | 2.93× |
| HumanEval | 220.5 | 534.8 | 2.43× |
| writing | 221.6* | 440.4 | 1.99× |
| question | 221.6* | 441.8 | 1.99× |
| rag | 221.6* | 436.2 | 1.97× |
| translation | 221.6* | 467.8 | 2.11× |
| tool_call | 221.6* | 432.4 | 1.95× |
| qa | 223.4 | 468.9 | 2.10× |
| summarization | 221.6* | 352.9 | 1.59× |
| average speedup | 1.00× | — | 2.12× |
* Estimated baseline (mean of the three measured baseline runs).
DSpark averages a 2.12× speedup (unweighted mean of per-subset speedups). The biggest gains are on math_reasoning at 2.93× (220.9 → 646.7 tokens/s) and HumanEval at 2.43×. math_reasoning's mean is well above its median (619.1 tokens/s), so a handful of highly predictable responses lift its average; on a typical math request the speedup is closer to 2.8×. translation and qa are next at around 2.1×, and writing, question, rag, and tool_call sit in a tight band between 1.95× and 1.99×. summarization is lowest at 1.59×.
Limitations
Works only with Qwen3-1.7B and is not usable as a standalone model. Acceptance falls off steeply past the first few positions on prose-like traffic, and on summarization in particular a block size of 8 is mostly wasted — the gains concentrate in math and code. Tool-call traffic sees only prose-level acceptance with this drafter. Real-world speedup depends on your traffic mix, and because verification is lossless, the verifier's own behavior and biases carry through unchanged.
License
Apache-2.0, matching the verifier. The speculators training code is also Apache-2.0.