Llama 3 8B Instruct
2 records across 1 hardware.
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RTX 4070 Ti SUPER | Llama 3 8B Instruct 8B | EXL2 8.0bpw | ExLlamaV2 | 64 | – | – | – | 8192 | 2024-05 | community | – | source |
| RTX 4070 Ti SUPER | Llama 3 8B Instruct 8B | Q8_0 | LM Studio | 56 | – | – | – | 8192 | 2024-05 | community | – | source |