Qwen3-14B speed on every GPU
11 measured runs across 6 GPUs, fastest first.
| GPU | Quant | Engine | tok/s | Roofline | Share |
|---|---|---|---|---|---|
| RTX 5090 | AWQ | vLLM | 153.0 | 200.9 | 76% |
| RTX 5090 | Q4_K_M | llama.cpp | 149.5 | 197.8 | 76% |
| H100 80GB HBM3 | AWQ | vLLM | 147.6 | 380.3 | 39% |
| H100 80GB HBM3 | Q4_K_M | llama.cpp | 144.4 | 374.3 | 39% |
| H200 NVL | Q4_K_M | llama.cpp | 135.0 | 521.8 | 26% |
| RTX 4090 | AWQ | vLLM | 97.7 | 113.0 | 86% |
| RTX 4090 | Q4_K_M | llama.cpp | 94.4 | 111.2 | 85% |
| RTX 3090 | AWQ | vLLM | 82.4 | 105.4 | 78% |
| L40S | AWQ | vLLM | 77.7 | 89.4 | 87% |
| RTX 3090 | Q4_K_M | llama.cpp | 76.4 | 103.8 | 74% |
| L40S | Q4_K_M | llama.cpp | 74.1 | 88.1 | 84% |