H100
18 records across 7 models (quants: BF16, FP8, Q4_0, Ternary PQ2_0, Ternary PTQ1_0, as-published).
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| H100 | gpt-oss 120B | as-published | vLLM | 119092 | – | – | – | – | 2026-09-25 | community | – | source |
| H100 | gpt-oss 120B | as-published | vLLM | 102876 | – | – | – | – | 2026-09-25 | community | – | source |
| H100 | Llama 3.1 8B | FP8 | vLLM | 5005 | – | – | – | – | 2026-09-30 | community | – | source |
| H100 | Llama 3.1 8B | FP8 | vLLM | 4977 | – | – | – | – | 2026-09-30 | community | – | source |
| H100 | Llama 3.1 8B | FP8 | vLLM | 4821 | – | – | – | – | 2026-09-30 | community | – | source |
| H100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 328.7 | 11564.03 | – | – | tg128 | 2025-11-15 | community | – | source |
| H100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 303.08 | 10162.9 | – | – | tg128 | 2025-11-15 | community | – | source |
| H100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 280.74 | 11263.29 | – | – | tg128 | 2025-09-05 | community | – | source |
| H100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 267.81 | 9918.34 | – | – | tg128 | 2025-09-05 | community | – | source |
| H100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 229.85 | 8457.52 | – | – | tg128 | 2025-09-05 | community | contradiction | source |
| H100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 219.23 | 7350.98 | – | – | tg128 | 2025-09-05 | community | contradiction | source |
| H100 | Nemotron-Parse | BF16 | vLLM | 937.9 | – | – | – | – | 2026-10-07 | community | – | source |
| H100 | Nemotron-Parse | BF16 | vLLM | 772.7 | – | – | – | – | 2026-10-07 | community | – | source |
| H100 | Qwen3 8B 8B dense | FP8 | vLLM | 839.75 | – | – | – | – | 2026-07 | community | – | source |
| H100 | Qwen3 8B 8B dense | FP8 | vLLM | 809.40 | – | – | – | – | 2026-07 | community | – | source |
| H100 | Qwen3.5-35B-A3B 35B MoE (A3B active) | FP8 | vLLM | 195 | – | – | – | – | 2026-04 | community | – | source |
| H100 | Ternary-Bonsai-2-27B | Ternary PQ2_0 | llama.cpp (CUDA) | 113.9 | 2830 | – | – | tg128 | 2026-09-16 | community | – | source |
| H100 | Ternary-Bonsai-2-27B | Ternary PTQ1_0 | llama.cpp (CUDA) | 86.9 | 1237 | – | – | tg128 | 2026-09-16 | community | – | source |
All rows cite their source. Provenance: community 40018 sourced 1085