Running Qwen3.8-27B-FP8 on two H100s, MTP speculative decoding was the biggest decode win (49 → 95 tok/s). But there’s a catch: in vLLM, MTP and prefix caching are mutually exclusive. So the question became: which config should actually go into production?

The answer came from running the two configs head-to-head: GPU0 no-MTP + prefix caching, GPU1 MTP (3 draft tokens). Both are TP1, 262K max context, FP8 KV cache, with a LiteLLM proxy load-balancing between them. All numbers are TTFT p50 unless noted.

I finally summed up all the Qwen 3.6 model test results I gathered over the past few days. I compared two models in detail: the Qwen3.6-35B-A3B (MoE, hybrid attention/delta) and the Qwen3.6-27B (dense, hybrid attention/delta). I ran both with turbo3 KV cache compression on an RTX 4090 as a llama.cpp server.

If I had to summarize briefly: the 35B-A3B is 3-4x faster in everything, but the 27B delivers better quality. This is the classic MoE vs. dense tradeoff, just backed by numbers.