Qwen2.5-7B-Instruct
30 records across 8 hardware.
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RTX 5090 | Qwen2.5-7B-Instruct 7B | BF16 | vLLM | 1404.7 | – | – | – | – | 2026-08 | community | – | source |
| RTX 5090 | Qwen2.5-7B-Instruct 7B | F16 | llama.cpp (CUDA) | 1045.8 | – | – | – | – | 2026-08 | community | – | source |
| RTX 5090 | Qwen2.5-7B-Instruct 7B | F16 | llama.cpp (CUDA) | 705.9 | – | – | – | – | 2026-08 | community | – | source |
| RTX 5090 | Qwen2.5-7B-Instruct 7B | F16 | llama.cpp (CUDA) | 700.9 | – | – | – | – | 2026-08 | community | – | source |
| RTX 5090 | Qwen2.5-7B-Instruct 7B | F16 | llama.cpp (CUDA) | 101.4 | – | – | – | – | 2026-08 | community | – | source |
| RTX 5090 | Qwen2.5-7B-Instruct 7B | F16 | llama.cpp (CUDA) | 100.1 | – | – | – | – | 2026-08 | community | – | source |
| RTX 5090 | Qwen2.5-7B-Instruct 7B | F16 | llama.cpp (CUDA) | 99.8 | – | – | – | – | 2026-08 | community | – | source |
| RTX 5090 | Qwen2.5-7B-Instruct 7B | BF16 | vLLM | 92.9 | – | – | – | – | 2026-08 | community | – | source |
| M5 Max | Qwen2.5-7B-Instruct | 4bit | oMLX | 87.7 | 2871.0 | – | – | 1024 | 2026-10-13 | community | – | source |
| M5 Max | Qwen2.5-7B-Instruct | 4bit | oMLX | 86.8 | 3091.0 | – | – | 4096 | 2026-10-13 | community | – | source |
| M2 Max | Qwen2.5-7B-Instruct | 4bit | oMLX | 73.5 | 615.3 | – | – | 1024 | 2026-10-09 | community | – | source |
| M2 Max | Qwen2.5-7B-Instruct | 4bit | oMLX | 68.0 | 600.3 | – | – | 4096 | 2026-10-09 | community | – | source |
| M5 Pro | Qwen2.5-7B-Instruct | 4bit | oMLX | 65.5 | 1695.0 | – | – | 1024 | 2026-10-10 | community | – | source |
| M5 Pro | Qwen2.5-7B-Instruct | 4bit | oMLX | 62.2 | 1896.0 | – | – | 4096 | 2026-10-10 | community | – | source |
| Arc Pro B70 | Qwen2.5-7B-Instruct 7B | Q4_K_M | llama.cpp (OpenVINO) | 59.7 | 5337.8 | – | – | 128 | 2026-09 | community | – | source |
| M5 Pro | Qwen2.5-7B-Instruct | 4bit | oMLX | 57.5 | 1891.0 | – | – | 8192 | 2026-10-10 | community | – | source |
| M4 Pro | Qwen2.5-7B-Instruct | 4bit | oMLX | 53.4 | 479.5 | – | – | 1024 | 2026-10-12 | community | – | source |
| M5 Pro | Qwen2.5-7B-Instruct | 4bit | oMLX | 51.8 | 1713.0 | – | – | 16384 | 2026-10-10 | community | – | source |
| M4 Pro | Qwen2.5-7B-Instruct | 4bit | oMLX | 50.4 | 459.1 | – | – | 4096 | 2026-10-12 | community | – | source |
| RTX PRO 4000 Blackwell | Qwen2.5-7B-Instruct | EXL2 6.0bpw H6 | ExLlamaV2 | 47 | – | – | – | – | 2026-03-16 | community | – | source |
| Arc Pro B70 | Qwen2.5-7B-Instruct 7B | Q4_K_M | llama.cpp (OpenVINO) | 44.1 | 5437.3 | – | – | 128 | 2026-09 | community | – | source |
| M4 Pro | Qwen2.5-7B-Instruct | 4bit | oMLX | 44.1 | 378.5 | – | – | 8192 | 2026-10-12 | community | – | source |
| Arc Pro B70 | Qwen2.5-7B-Instruct 7B | Q4_K_M | llama.cpp (OpenVINO) | 41.1 | 3043.0 | – | – | 4096 | 2026-09 | community | – | source |
| M4 | Qwen2.5-7B-Instruct | AWQ | oMLX | 21.8 | 256.3 | – | – | 1024 | 2026-10-14 | community | – | source |
| M4 | Qwen2.5-7B-Instruct | AWQ | oMLX | 21.1 | 247.4 | – | – | 4096 | 2026-10-16 | community | – | source |
| M4 | Qwen2.5-7B-Instruct | AWQ | oMLX | 20.9 | 247.5 | – | – | 4096 | 2026-10-14 | community | – | source |
| M4 | Qwen2.5-7B-Instruct | AWQ | oMLX | 20.2 | 235.4 | – | – | 8192 | 2026-10-16 | community | – | source |
| M4 | Qwen2.5-7B-Instruct | AWQ | oMLX | 19.8 | 235.5 | – | – | 8192 | 2026-10-14 | community | – | source |
| M4 | Qwen2.5-7B-Instruct | 4bit | oMLX | 18.7 | 206.9 | – | – | 16384 | 2026-10-17 | community | – | source |
| Arc Pro B70 | Qwen2.5-7B-Instruct 7B | Q4_K_M | llama.cpp (OpenVINO) | 9.6 | 2388.4 | – | – | 4096 | 2026-09 | community | – | source |