Running Qwen3.8-27B-FP8 on two H100s, MTP speculative decoding was the biggest decode win (49 → 95 tok/s). But there’s a catch: in vLLM, MTP and prefix caching are mutually exclusive. So the question became: which config should actually go into production?

The answer came from running the two configs head-to-head: GPU0 no-MTP + prefix caching, GPU1 MTP (3 draft tokens). Both are TP1, 262K max context, FP8 KV cache, with a LiteLLM proxy load-balancing between them. All numbers are TTFT p50 unless noted.