Qwen on a GB10: 197 to 779 tok/s With Receipts, Not Headlines

We ran a controlled inference-tuning matrix for Qwen3.6-35B-A3B-NVFP4 on one NVIDIA GB10. Baseline: 197.5517 tok/s. After tuning, three consecutive c64x512 runs returned 774.5402, 779.0010, and 778.4611 tok/s. Mean: 777.3341 tok/s, 3.93x baseline.

That number applies to 99 to 114 prompt tokens. At 1,335 to 1,505 prompt tokens, the same configuration returns 521.4124 tok/s.

The benchmark contract

• Workload: c64x512 (64 concurrent, 512 completion tokens, 99 to 114 prompt tokens).
• Numerator: server-reported completion tokens from raw SSE usage.
• Denominator: full client wall time. No trimming.
• Distinctness: 64 distinct request hashes and content hashes per run.
• Repeatability: three consecutive runs within a 0.24s band.

Winning configuration

FP8 KV cache, FlashInfer attention, Marlin MoE backend
GPU util 0.6, seq64, batch32768
Prefix caching: disabled
MTP-1 with Triton drafter MoE

Three qualifying runs

Run	Completion tokens	Wall time	Throughput
1	32,768	42.306s	774.5402 tok/s
2	32,768	42.064s	779.0010 tok/s
3	32,768	42.093s	778.4611 tok/s

Upstream recipe and setup: https://github.com/Weschera/spark-bench at 45491e80a22f4701d8f3a82ad0b5ce7fa9597089.