Every GPU measured
Pick a card to see every model, quant and engine measured on it, each against that GPU's roofline.
NVIDIA
RTX 3090consumer24 GB · 898 GB/s measured124.5 tok/s Qwen3-8B Q4_K_M3 models · 8 runs2× RTX 3090consumer24 GB each · 896 GB/s measured68.6 tok/s Qwen3.8-27B W4A16-AWQ2 models · 3 runsRTX 4090consumer24 GB · 962 GB/s measured97.7 tok/s Qwen3-14B AWQ2 models · 4 runsRTX 5090consumer32 GB · 1710 GB/s measured378.9 tok/s Nemotron-3.5-Lightning-30B NVFP45 models · 13 runsL40Sdata center45 GB · 762 GB/s measured77.7 tok/s Qwen3-14B AWQ2 models · 4 runsH200data center140 GB · 4662 GB/s measured166.9 tok/s Qwen3-14B AWQ3 models · 3 runsH200 NVLdata center140 GB · 4513 GB/s measured135.0 tok/s Qwen3-14B Q4_K_M2 models · 2 runsH100 80GBdata center80 GB · 3252 GB/s measured227.0 tok/s Qwen3-8B Q4_K_M3 models · 5 runs