Qwen3.8-27B speed on every GPU
11 measured runs across 6 GPUs, fastest first.
| GPU | Quant | Engine | tok/s | Roofline | Share |
|---|---|---|---|---|---|
| H100 80GB HBM3 | W4A16-AWQ | vLLM | 87.8 | 211.9 | 41% |
| RTX 5090 | W4A16-AWQ | vLLM | 87.5 | 109.7 | 80% |
| H100 80GB HBM3 | UD-Q4_K_M | llama.cpp | 83.6 | 205.7 | 41% |
| RTX 5090 | UD-Q4_K_M | llama.cpp | 81.9 | 108.7 | 75% |
| H200 NVL | UD-Q4_K_M | llama.cpp | 78.6 | 286.7 | 27% |
| RTX 4090 | W4A16-AWQ | vLLM | 54.4 | 61.7 | 88% |
| RTX 4090 | UD-Q4_K_M | llama.cpp | 50.4 | 61.1 | 82% |
| RTX 3090 | W4A16-AWQ | vLLM | 46.2 | 57.6 | 80% |
| L40S | W4A16-AWQ | vLLM | 43.6 | 48.9 | 89% |
| RTX 3090 | UD-Q4_K_M | llama.cpp | 41.5 | 57.0 | 73% |
| L40S | UD-Q4_K_M | llama.cpp | 40.0 | 48.4 | 83% |