Speculative Decoding on CPUs: Nearly 4x Faster Token Generation with DFlash

Speculative decoding accelerates language model inference by using a lightweight draft model to propose several future tokens at once, which are then verified in a single pass by the larger target model. This technique bypasses the slow, sequential nature of standard autoregressive generation, where each token is produced one at a time. A method called DFlash, integrated into vLLM, uses a block-diffusion drafter to achieve significant speedups on CPUs, demonstrating a nearly 4x increase in token throughput for the Qwen3.5-9B model. This performance gain is achieved by trading spare CPU compute for saved memory bandwidth, effectively turning memory-bound operations into more efficient compute-bound ones.