llama-3-1-8b-instruct
101 records across 56 hardware.
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| M5 Max | llama-3-1-8b-instruct | 4bit | oMLX | 87.1 | 2860.0 | – | – | 1024 | 2026-10-11 | community | – | source |
| M3 Ultra (80-core GPU, 256 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 63.304543444444434 | 1062.184084222222 | – | 1.1019065137777775 | – | – | sourced | – | source |
| M3 Ultra (80-core GPU, 512 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 62.67566477777777 | 1109.4627288333336 | – | 1.060675240722222 | – | – | sourced | – | source |
| M5 Max (32-core GPU, 36 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 61.634733222222216 | 630.2577913333333 | – | 1.914077976777778 | – | – | sourced | – | source |
| M2 Ultra (60-core GPU, 64 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 59.480982 | 703.2592348888888 | – | 1.6398754306666665 | – | – | sourced | – | source |
| M4 Max (40-core GPU, 48 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 55.0887822962963 | 663.3646092592592 | – | 1.7937055694814816 | – | – | sourced | – | source |
| M1 Ultra (64-core GPU, 128 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 54.30790400000001 | 667.5689454444446 | – | 1.6929770138888889 | – | – | sourced | – | source |
| M4 Pro | llama-3-1-8b-instruct | 4bit | oMLX | 54.2 | 381.9 | – | – | 1024 | 2026-10-11 | community | – | source |
| M4 Pro | llama-3-1-8b-instruct | 4bit | oMLX | 53.3 | 378.9 | – | – | 1024 | 2026-10-11 | community | – | source |
| M4 Max (40-core GPU, 128 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 51.63987292592592 | 617.642594037037 | – | 1.9420189397222225 | – | – | sourced | – | source |
| M4 Pro | llama-3-1-8b-instruct | 4bit | oMLX | 48.9 | 462.4 | – | – | 1024 | 2026-10-06 | community | – | source |
| M1 Ultra (48-core GPU, 128 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 48.89361162962964 | 534.3503048148149 | – | 2.1556520740370373 | – | – | sourced | – | source |
| M4 Max (32-core GPU, 36 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 48.071329866666666 | 570.2711772 | – | 2.1545828000444445 | – | – | sourced | – | source |
| M4 Pro | llama-3-1-8b-instruct | 4bit | oMLX | 47.6 | 442.3 | – | – | 4096 | 2026-10-06 | community | – | source |
| M4 Max (40-core GPU, 64 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 47.095953944444446 | 557.1200590555555 | – | 2.1445155647222225 | – | – | sourced | – | source |
| M2 Max (38-core GPU, 96 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 46.41150744444444 | 484.057812 | – | 2.4422629954444446 | – | – | sourced | – | source |
| M3 Max (40-core GPU, 128 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 45.792675277777775 | 587.0791163333333 | – | 2.0508701621111114 | – | – | sourced | – | source |
| M2 Max (38-core GPU, 32 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 44.66334788888888 | 473.660562 | – | 2.4878861063333333 | – | – | sourced | – | source |
| M1 Max (32-core GPU, 64 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 37.75919747222222 | 380.7280225277778 | – | 3.12193795138889 | – | – | sourced | – | source |
| M3 Max (30-core GPU, 96 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 37.74030144444444 | 456.8050157777777 | – | 2.6877091111111118 | – | – | sourced | – | source |
| M3 Max (30-core GPU, 36 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 37.47979072222222 | 443.4985221666666 | – | 2.725718886444444 | – | – | sourced | – | source |
| M1 Max (32-core GPU, 32 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 35.40364952777778 | 359.8733261944444 | – | 3.4528548829722223 | – | – | sourced | – | source |
| M4 Pro (20-core GPU, 64 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 32.90144462962963 | 349.4138904444444 | – | 3.563006918222223 | – | – | sourced | – | source |
| M4 Pro (20-core GPU, 48 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 32.67357938271605 | 361.8889191604938 | – | 3.435018682604938 | – | – | sourced | – | source |
| M4 Pro (20-core GPU, 24 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 32.533236711111115 | 359.5577164 | – | 3.419087840755556 | – | – | sourced | – | source |
| M1 Max (24-core GPU, 64 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 32.11692033333334 | 306.18835666666666 | – | 3.9939120186666677 | – | – | sourced | – | source |
| M2 Max (30-core GPU, 32 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 31.185154777777775 | 325.001513 | – | 3.7657567592222216 | – | – | sourced | – | source |
| M6 | llama-3-1-8b-instruct | 4bit | oMLX | 30.9 | 998.1 | – | – | 1024 | 2026-10-06 | community | – | source |
| M4 Pro (16-core GPU, 24 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 30.508745533333336 | 298.0409531777778 | – | 4.201011161111111 | – | – | sourced | – | source |
| M4 Pro (16-core GPU, 48 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 30.176443111111105 | 302.390073 | – | 4.143549694333333 | – | – | sourced | – | source |
| M6 | llama-3-1-8b-instruct | 4bit | oMLX | 28.5 | 1077.0 | – | – | 4096 | 2026-10-06 | community | – | source |
| M5 Max | llama-3-1-8b-instruct | unknown | oMLX | 27.6 | 2651.0 | – | – | 1024 | 2026-10-11 | community | – | source |
| M2 Pro (19-core GPU, 32 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 26.311742333333328 | 261.761128 | – | 4.701914784666666 | – | – | sourced | – | source |
| M3 Max (40-core GPU, 64 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 25.448832000000003 | 377.04858177777777 | – | 3.3699854838888887 | – | – | sourced | – | source |
| M6 | llama-3-1-8b-instruct | 4bit | oMLX | 25.4 | 1046.0 | – | – | 8192 | 2026-10-06 | community | – | source |
| M2 Pro (16-core GPU, 16 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 24.331960740740744 | 224.56031674074077 | – | 5.574148605037037 | – | – | sourced | – | source |
| M2 Pro (16-core GPU, 32 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 23.809723111111115 | 208.68323722222223 | – | 6.06785997211111 | – | – | sourced | – | source |
| M5 (10-core GPU, 32 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 22.29346603703704 | 210.70696922222223 | – | 5.997633993703704 | – | – | sourced | – | source |
| M3 Pro (18-core GPU, 36 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 22.06217488888889 | 283.8059724444445 | – | 4.460495907444445 | – | – | sourced | – | source |
| M1 Pro (16-core GPU, 16 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 21.88205522222222 | 204.5403022222222 | – | 6.089909109518518 | – | – | sourced | – | source |
| M1 Pro (16-core GPU, 32 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 21.698664740740742 | 200.28393007407408 | – | 6.2272433256296305 | – | – | sourced | – | source |
| M3 Pro (14-core GPU, 36 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 21.455100444444447 | 223.43918544444443 | – | 5.651565199111111 | – | – | sourced | – | source |
| M3 Pro (18-core GPU, 18 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 20.80849266666667 | 279.21399711111115 | – | 4.529956342555556 | – | – | sourced | – | source |
| M1 Pro (14-core GPU, 16 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 20.14017855555555 | 176.96636692592594 | – | 6.991519057185186 | – | – | sourced | – | source |
| M1 Pro (14-core GPU, 32 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 19.971878777777775 | 173.08191844444443 | – | 7.230578513999999 | – | – | sourced | – | source |
| M3 Pro (14-core GPU, 18 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 19.10761944444444 | 199.51528166666668 | – | 6.427559875000002 | – | – | sourced | – | source |
| M2 (8-core GPU, 8 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 18.342969166666663 | 148.82072661111113 | – | 8.390452365666668 | – | – | sourced | – | source |
| M4 (10-core GPU, 32 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 16.809439277777777 | 166.14551366666666 | – | 7.554811377277778 | – | – | sourced | – | source |
| M4 Pro | llama-3-1-8b-instruct | unknown | oMLX | 16.2 | 405.5 | – | – | 1024 | 2026-10-11 | community | – | source |
| M4 Pro | llama-3-1-8b-instruct | unknown | oMLX | 16.2 | 405.5 | – | – | 1024 | 2026-10-11 | community | – | source |
| M4 (10-core GPU, 16 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 15.983290222222225 | 166.82581797222224 | – | 7.980498432833332 | – | – | sourced | – | source |
| M4 (10-core GPU, 24 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 15.935378370370373 | 148.96748985185187 | – | 8.236942598851853 | – | – | sourced | – | source |
| M4 Pro | llama-3-1-8b-instruct | unknown | oMLX | 15.8 | 399.0 | – | – | 4096 | 2026-10-11 | community | – | source |
| M4 Pro | llama-3-1-8b-instruct | unknown | oMLX | 15.8 | 399.0 | – | – | 4096 | 2026-10-11 | community | – | source |
| M4 (8-core GPU, 16 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 15.26954622222222 | 134.02995044444447 | – | 9.18359174066667 | – | – | sourced | – | source |
| M1 Ultra (GPU count not published, 128 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 15.24847544444444 | 71.55644355555556 | – | 17.47675613911111 | – | – | sourced | – | source |
| M2 (10-core GPU, 16 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 14.68716522222222 | 140.73035655555557 | – | 9.125759541666666 | – | – | sourced | – | source |
| M2 (10-core GPU, 24 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 14.668569888888888 | 141.73761555555558 | – | 9.023242666666668 | – | – | sourced | – | source |
| M1 (8-core GPU, 8 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 14.552260666666667 | 133.8709728888889 | – | 9.648145671333332 | – | – | sourced | – | source |
| M3 (10-core GPU, 16 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 13.499802861111112 | 138.71456447222224 | – | 9.024639849583332 | – | – | sourced | – | source |
| M1 (7-core GPU, 8 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 13.385236666666666 | 108.71905633333331 | – | 11.513906754444445 | – | – | sourced | – | source |
| M2 (8-core GPU, 16 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 12.906848888888891 | 113.95904577777776 | – | 10.946365963333333 | – | – | sourced | – | source |
| M3 (GPU count not published, 16 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 11.83115477777778 | 33.58097844444444 | – | 40.49901329166667 | – | – | sourced | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | 11.29 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | 11.27 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | 11.06 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | 11.03 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | 10.69 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | 10.64 | – | – | – | 128 | 2026-10 | community | – | source |
| M3 (10-core GPU, 24 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 10.22738411111111 | 109.8980422777778 | – | 11.670486025444445 | – | – | sourced | – | source |
| M1 (7-core GPU, 16 GB) | llama-3-1-8b-instruct | Q4_K - Medium | llamafile | 9.387483333333332 | 83.93792277777777 | – | 14.82420303238889 | – | – | sourced | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | 6.25 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | 6.21 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | 6.09 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | 6.07 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | 5.91 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | 5.86 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 53.84 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 209.1 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 99.37 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 366.7 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 176.76 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 571.95 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 51.5 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 177.31 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 95.36 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 315.43 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 168.85 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q8_0 | llama.cpp (CPU) | – | 496.83 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 99.07 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 205.03 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 177.18 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 365.14 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 305.82 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 580.2 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 91.36 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 174.46 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 164.27 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 312.12 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 283.17 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | llama-3-1-8b-instruct | Q4_0 | llama.cpp (CPU) | – | 504.99 | – | – | 4096 | 2026-10 | community | – | source |