Qwen3.8-27B UD-Q4_K_M on H100 80GB HBM3
GGUF IQ4_XS+Q5_K
The tested file was 16.46 GB. The repo's current file is 16.56 GB: it was re-uploaded after the run.
Engines
llama.cpp
b11571 · 097cf33d9 ↗measured 2026-11-09 18:22 UTCnewest release then: 0.9.0 (checked 2026-11-09)
83.6 tok/s
41% of the roofline205.7 tok/s possible
- now, a token
- 11.96 ms
- floor
- 4.86 ms
- read, a token
- 15.74 GB
More
- memory, measured
- 3237 GB/s
What was run
llama-bench -m Qwen3.8-27B-UD-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1Speed at context
128 tokens
512 tokens
Where the time goes
- Now, a token
- 11.96 ms
- Roofline floor, a token
- 4.86 ms
- Gap to the floor
- 7.10 ms
- Kernels a token
- 497
Launch heavy: 1914 kernel launches per token
Per role ms a token. floor (bytes ÷ measured memory speed) above the floor
Per kernel (8): ms now / ms at the roofline, launches a token
No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.
Run h100-qwen3.8-27b-q4km-llama
Tested on
- GPU
- H100 80GB HBM3 · 80 GB
- System RAM
- 443 GB
- CPU threads
- 52
- Memory, measured
- 3237 GB/s
- Temp, max
- 40 °C
- Power
- 147 W mean of 700 W
- SM clock
- 1980 MHz
- Driver
- 580.126.20
- Measured
- 2026-11-09 18:22 UTC · context 512 tokens · measured, in-model (nsys): 33 of 39 weight roles timed
Qwen3.8-27B on other GPUs
H200data center100.8 tok/s · 34%2× Radeon R9700consumer92.3 tok/s H100 80GB HBM3data center87.8 tok/s · 41%RTX 5090consumer87.5 tok/s · 80%H200 NVLdata center78.6 tok/s · 27%2× RTX 3090consumer68.6 tok/s · 76%RTX 4090consumer54.4 tok/s · 88%RTX 3090consumer46.2 tok/s · 80%L40Sdata center43.6 tok/s · 89%
Best engine and quant on each GPU, one user. The percentage is that GPU's own roofline.