A100
46 records across 7 models (quants: BF16, Q4_0, Ternary PQ2_0, Ternary PTQ1_0, as-published).
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A100 | gpt-oss 20B 20B MoE | as-published | vLLM | 914.97 | – | – | – | – | 2026-09 | community | – | source |
| A100 | gpt-oss 20B 20B MoE | as-published | vLLM | 501.49 | – | – | – | – | 2026-09 | community | – | source |
| A100 | InternVL3.5-2B 2B VLM | as-published | vLLM | 6016.16 | – | – | – | – | 2025-09 | community | – | source |
| A100 | InternVL3.5-2B 2B VLM | as-published | vLLM | 5912.53 | – | – | – | – | 2025-09 | community | – | source |
| A100 | InternVL3.5-2B 2B VLM | as-published | vLLM | 4734.68 | – | – | – | – | 2025-09 | community | – | source |
| A100 | InternVL3.5-2B 2B VLM | as-published | vLLM | 4143.84 | – | – | – | – | 2025-09 | community | – | source |
| A100 | InternVL3.5-2B 2B VLM | as-published | vLLM | 1691.45 | – | – | – | – | 2025-09 | community | – | source |
| A100 | InternVL3.5-2B 2B VLM | as-published | vLLM | 1683.27 | – | – | – | – | 2025-09 | community | – | source |
| A100 | InternVL3.5-2B 2B VLM | as-published | vLLM | 202.74 | – | – | – | – | 2025-09 | community | – | source |
| A100 | InternVL3.5-2B 2B VLM | as-published | vLLM | 199.48 | – | – | – | – | 2025-09 | community | – | source |
| A100 | InternVL3.5-4B 4B VLM | as-published | vLLM | 2674.70 | – | – | – | – | 2025-09 | community | – | source |
| A100 | InternVL3.5-4B 4B VLM | as-published | vLLM | 2120.87 | – | – | – | – | 2025-09 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 215.11 | 5422.2 | – | – | tg128 | 2025-11-15 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 200.9 | 5285.96 | – | – | tg128 | 2025-09-05 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 200.05 | 4967.14 | – | – | tg128 | 2025-11-15 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 196.26 | 5295.94 | – | – | tg128 | 2025-09-05 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 190.88 | 4849.53 | – | – | tg128 | 2025-09-05 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 185.91 | 4883.69 | – | – | tg128 | 2025-09-05 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 176.88 | 5141.45 | – | – | tg128 | 2025-05-22 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 168.23 | 4865.98 | – | – | tg128 | 2025-05-22 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 155.72 | 3016.29 | – | – | tg128 | 2026-07-30 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 141.32 | 2698.95 | – | – | tg128 | 2026-07-30 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 128.41 | 1175.43 | – | – | tg128 | 2026-05-26 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (Vulkan) | 121.83 | 3103.32 | – | – | tg128 | 2025-05-22 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (Vulkan) | 120.53 | 3164.55 | – | – | tg128 | 2025-05-22 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (Vulkan) | 108.39 | 4040.77 | – | – | tg128 | 2026-08-14 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (Vulkan) | 102.46 | 3222.01 | – | – | tg128 | 2026-08-14 | community | – | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 65.91 | 653.01 | – | – | tg128 | 2026-05-13 | community | contradiction | source |
| A100 | llama 7B | Q4_0 | llama.cpp (CUDA) | 65.48 | 626.32 | – | – | tg128 | 2026-05-13 | community | contradiction | source |
| A100 | MiniCPM-V-4 VLM | as-published | vLLM | 3822.97 | – | – | – | – | 2025-09 | community | – | source |
| A100 | MiniCPM-V-4 VLM | as-published | vLLM | 3239.55 | – | – | – | – | 2025-09 | community | – | source |
| A100 | MiniCPM-V-4 VLM | as-published | vLLM | 2320.44 | – | – | – | – | 2025-09 | community | – | source |
| A100 | MiniCPM-V-4 VLM | as-published | vLLM | 1272.71 | – | – | – | – | 2025-09 | community | – | source |
| A100 | MiniCPM-V-4 VLM | as-published | vLLM | 1114.85 | – | – | – | – | 2025-09 | community | – | source |
| A100 | MiniCPM-V-4 VLM | as-published | vLLM | 1065.36 | – | – | – | – | 2025-09 | community | – | source |
| A100 | MiniCPM-V-4 VLM | as-published | vLLM | 126.22 | – | – | – | – | 2025-09 | community | – | source |
| A100 | MiniCPM-V-4 VLM | as-published | vLLM | 126.05 | – | – | – | – | 2025-09 | community | – | source |
| A100 | Qwen2.5-VL-7B 7B VLM | BF16 | vLLM | 2632.88 | – | – | – | – | 2025-09 | community | – | source |
| A100 | Qwen2.5-VL-7B 7B VLM | BF16 | vLLM | 2451.80 | – | – | – | – | 2025-09 | community | – | source |
| A100 | Qwen2.5-VL-7B 7B VLM | BF16 | vLLM | 1795.85 | – | – | – | – | 2025-09 | community | – | source |
| A100 | Qwen2.5-VL-7B 7B VLM | BF16 | vLLM | 1517.21 | – | – | – | – | 2025-09 | community | – | source |
| A100 | Qwen2.5-VL-7B 7B VLM | BF16 | vLLM | 713.45 | – | – | – | – | 2025-09 | community | – | source |
| A100 | Qwen2.5-VL-7B 7B VLM | BF16 | vLLM | 709.00 | – | – | – | – | 2025-09 | community | – | source |
| A100 | Qwen2.5-VL-7B 7B VLM | BF16 | vLLM | 76.46 | – | – | – | – | 2025-09 | community | – | source |
| A100 | Ternary-Bonsai-2-27B | Ternary PQ2_0 | llama.cpp (CUDA) | 73.9 | 1328 | – | – | tg128 | 2026-09-16 | community | – | source |
| A100 | Ternary-Bonsai-2-27B | Ternary PTQ1_0 | llama.cpp (CUDA) | 54.7 | 706 | – | – | tg128 | 2026-09-16 | community | – | source |
All rows cite their source. Provenance: community 40018 sourced 1085