granite-4.0-h-tiny
8 records across 2 hardware.
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RTX PRO 6000 | granite-4.0-h-tiny | Q4_K_M | llama.cpp (CUDA) | 2541.9 | – | – | – | – | 2026-08 | community | – | source |
| RTX PRO 6000 | granite-4.0-h-tiny | BF16 | llama.cpp (CUDA) | 2287.6 | – | – | – | – | 2026-08 | community | – | source |
| RTX PRO 6000 | granite-4.0-h-tiny | Q4_K_M | llama.cpp (CUDA) | 2192.4 | – | – | – | – | 2026-08 | community | – | source |
| RTX PRO 6000 | granite-4.0-h-tiny | BF16 | llama.cpp (CUDA) | 1427.0 | – | – | – | – | 2026-08 | community | – | source |
| GB10 | granite-4.0-h-tiny | Q4_K_M | llama.cpp (CUDA) | 551.2 | – | – | – | – | 2026-08 | community | – | source |
| GB10 | granite-4.0-h-tiny | Q4_K_M | llama.cpp (CUDA) | 456.4 | – | – | – | – | 2026-08 | community | – | source |
| GB10 | granite-4.0-h-tiny | BF16 | llama.cpp (CUDA) | 390.4 | – | – | – | – | 2026-08 | community | – | source |
| GB10 | granite-4.0-h-tiny | BF16 | llama.cpp (CUDA) | 271.3 | – | – | – | – | 2026-08 | community | – | source |