Qwen3.8-27B Quark W4A16 on 2× Radeon R9700
Quark W4A16 export of Qwen3.8-27B (exact file not recorded)
Engines
vLLM
0.31.0 ↗measured 2026-10-27
56.1 tok/s
No roofline yet: written record
More
- memory, measured
- –
- 4-bit output head (patched)
- 61.2 tok/s
--attention-backend TRITON_ATTN, GPU_MAX_HW_QUEUES=1 (vLLM's default ROCM_ATTN slows to 9.4 tok/s by 5,000 tokens)vLLM + MTP k=2
0.31.0 + split-KV verify patch ↗measured 2026-10-27
92.3 tok/s
No roofline yet: written record
More
- memory, measured
- –
- over 3,000 tokens
- holds about 90 tok/s (stock vLLM: 84 falling to 51)
- stock vLLM, no patch
- 88.9 tok/s
- with a 4-bit output head too (patched)
- 115.5 tok/s, about 109 flat over 3,000 tokens
- MTP k=1 / k=3 (stock)
- 77.9 / 80.8 tok/s
TRITON_ATTN with a 2-line split-KV verify patch; stock vLLM 0.31 falls from 84 to 51 tok/s over 2,900 tokens at k=2Speed at context
short
Where the time goes
From written records: the raw run files are on the R9700 machine, so there is no roofline or kernel breakdown yet.
Tested on
- GPU
- 2× Radeon R9700 · 32 GB each
- System RAM
- 128 GB
- CPU threads
- –
- Memory, measured
- –
- Temp, max
- see notes
- Power
- see notes
- SM clock
- –
- Driver
- –
- Measured
- 2026-10-27 · written record: 4 text prompts x 256 tokens, greedy
- Two Radeon AI PRO R9700 (gfx1201), 32 GB each, 128 GB of system RAM
- Source: BoltBeam docs/models-tested.md (2026-11-13) and docs/evidence/mtp-27b-rtx5090-r9700-20261027 (r9700/sources.md). Written records: the raw run files stay on the R9700 machine.
Qwen3.8-27B on other GPUs
RTX 5090vLLM + MTP, 4-bit draft layer · W4A16-AWQ · 4-bit head202.1 tok/s H200vLLM · W4A16-AWQ100.8 tok/s · 34%2× Radeon R9700vLLM + MTP k=2 · Quark W4A1692.3 tok/s H100 80GBvLLM · W4A16-AWQ87.8 tok/s · 41%H200 NVLllama.cpp · UD-Q4_K_M78.6 tok/s · 27%2× RTX 3090vLLM · W4A16-AWQ68.6 tok/s · 76%RTX 4090vLLM · W4A16-AWQ54.4 tok/s · 88%RTX 3090vLLM · W4A16-AWQ46.2 tok/s · 80%L40SvLLM · W4A16-AWQ43.6 tok/s · 89%
The fastest setup measured on each GPU, one user. The percentage is that GPU's own roofline.