boltbeam.fyi

Qwen3.8-27B Quark W4A16 on 2× Radeon R9700

Quark W4A16 export of Qwen3.8-27B (exact file not recorded)

Engines

vLLM

0.31.0 ↗measured 2026-10-27

56.1 tok/s

No roofline yet: written record

More
memory, measured
–
4-bit output head (patched)
61.2 tok/s
--attention-backend TRITON_ATTN, GPU_MAX_HW_QUEUES=1 (vLLM's default ROCM_ATTN slows to 9.4 tok/s by 5,000 tokens)

vLLM + MTP k=2

0.31.0 + split-KV verify patch ↗measured 2026-10-27

92.3 tok/s

No roofline yet: written record

More
memory, measured
–
over 3,000 tokens
holds about 90 tok/s (stock vLLM: 84 falling to 51)
stock vLLM, no patch
88.9 tok/s
with a 4-bit output head too (patched)
115.5 tok/s, about 109 flat over 3,000 tokens
MTP k=1 / k=3 (stock)
77.9 / 80.8 tok/s
TRITON_ATTN with a 2-line split-KV verify patch; stock vLLM 0.31 falls from 84 to 51 tok/s over 2,900 tokens at k=2

Speed at context

short
vLLM56.1 tok/s
vLLM + MTP k=292.3 tok/s

Where the time goes

From written records: the raw run files are on the R9700 machine, so there is no roofline or kernel breakdown yet.

Tested on

GPU
2× Radeon R9700 · 32 GB each
System RAM
128 GB
CPU threads
–
Memory, measured
–
Temp, max
see notes
Power
see notes
SM clock
–
Driver
–
Measured
2026-10-27 · written record: 4 text prompts x 256 tokens, greedy
  • Two Radeon AI PRO R9700 (gfx1201), 32 GB each, 128 GB of system RAM
  • Source: BoltBeam docs/models-tested.md (2026-11-13) and docs/evidence/mtp-27b-rtx5090-r9700-20261027 (r9700/sources.md). Written records: the raw run files stay on the R9700 machine.

Qwen3.8-27B on other GPUs