All Benchmarked Models

62 models tested across real hardware. Click any model to see detailed benchmark results.

llama3.1:8b

llama · 8.0B · Q4_K_M

2 benchmarks avg 71/100 12.9 tok/s

mistral:latest

llama · 7.2B · Q4_K_M

2 benchmarks avg 73/100 38.4 tok/s

qwen3:0.6b

qwen3 · 751.63M · Q4_K_M

2 benchmarks avg 67/100 144.8 tok/s

gemma3:1b

gemma3 · 999.89M · Q4_K_M

2 benchmarks avg 68/100 39.4 tok/s

qwen2.5:7b

qwen2 · 7.6B · Q4_K_M

1 benchmarks avg 84/100 22.2 tok/s

qwen2.5:14b

qwen2 · 14.8B · Q4_K_M

1 benchmarks avg 83/100 11.2 tok/s

qwen2.5-7b-instruct

qwen2 · 7B · 4bit

1 benchmarks avg 82/100 22 tok/s

gemma3:12b

gemma3 · 12.2B · Q4_K_M

1 benchmarks avg 80/100 12.7 tok/s

qwen3.5:9b

qwen35 · 9.7B · Q4_K_M

1 benchmarks avg 80/100 11.8 tok/s

gemma3:4b

gemma3 · 4.3B · Q4_K_M

1 benchmarks avg 80/100 34.6 tok/s

qwen3.5:4b

qwen35 · 4.7B · Q4_K_M

1 benchmarks avg 80/100 16.6 tok/s

qwen2.5-coder:7b

qwen2 · 7.6B · Q4_K_M

1 benchmarks avg 79/100 22.1 tok/s

qwen2.5-coder-7b-instruct-mlx

qwen2 · 7B · 4bit

1 benchmarks avg 79/100 22.5 tok/s

yi:6b

llama · 6B · Q4_0

1 benchmarks avg 77/100 27.6 tok/s

gemma2:2b

gemma2 · 2.6B · Q4_0

1 benchmarks avg 77/100 55.1 tok/s

qwen2.5:3b

qwen2 · 3.1B · Q4_K_M

1 benchmarks avg 77/100 47.2 tok/s

gemma2:9b

gemma2 · 9.2B · Q4_0

1 benchmarks avg 76/100 17.5 tok/s

phi4:14b

phi3 · 14.7B · Q4_K_M

1 benchmarks avg 75/100 11 tok/s

qwen3.5:27b

qwen35 · 27.8B · Q4_K_M

1 benchmarks avg 75/100 4.4 tok/s

granite3.1-dense:8b

granite · 8.2B · Q4_K_M

1 benchmarks avg 74/100 18.8 tok/s

llama3.2:3b

llama · 3.2B · Q4_K_M

1 benchmarks avg 74/100 44.1 tok/s

qwen3.5:2b

qwen35 · 2.3B · Q8_0

1 benchmarks avg 73/100 30.4 tok/s

llama3.2:latest

llama · 3.2B · Q4_K_M

1 benchmarks avg 73/100 98.9 tok/s

mistral:7b

llama · 7.2B · Q4_K_M

1 benchmarks avg 72/100 22 tok/s

codegemma:7b

gemma · 9B · Q4_0

1 benchmarks avg 71/100 18.7 tok/s

mistral-nemo:12b

llama · 12.2B · Q4_0

1 benchmarks avg 71/100 14.6 tok/s

phi4-mini:latest

phi3 · 3.8B · Q4_K_M

1 benchmarks avg 71/100 36.2 tok/s

llama3:latest

llama · 8.0B · Q4_0

1 benchmarks avg 71/100 22.3 tok/s

qwen2.5:1.5b

qwen2 · 1.5B · Q4_K_M

1 benchmarks avg 70/100 84.1 tok/s

deepseek-v2:16b

deepseek2 · 15.7B · Q4_0

1 benchmarks avg 69/100 56.6 tok/s

granite3.1-dense:2b

granite · 2.5B · Q4_K_M

1 benchmarks avg 68/100 54.6 tok/s

nous-hermes2:latest

llama · 11B · Q4_0

1 benchmarks avg 68/100 15.9 tok/s

neural-chat:7b

llama · 7B · Q4_0

1 benchmarks avg 67/100 22.5 tok/s

aya:8b

command-r · 8.0B · F16

1 benchmarks avg 66/100 20 tok/s

qwen3.5:0.8b

qwen35 · 873.44M · Q8_0

1 benchmarks avg 65/100 48.2 tok/s

smollm2:1.7b

llama · 1.7B · Q8_0

1 benchmarks avg 64/100 51.8 tok/s

phi3:3.8b

phi3 · 3.8B · Q4_0

1 benchmarks avg 64/100 42 tok/s

starling-lm:7b

llama · 7B · Q4_0

1 benchmarks avg 63/100 23.7 tok/s

solar:10.7b

llama · 11B · Q4_0

1 benchmarks avg 63/100 15.8 tok/s

phi3:14b

phi3 · 14.0B · Q4_0

1 benchmarks avg 63/100 12.7 tok/s

codellama:7b

llama · 7B · Q4_0

1 benchmarks avg 63/100 23.6 tok/s

dolphin-phi:2.7b

phi2 · 3B · Q4_0

1 benchmarks avg 61/100 56.1 tok/s

phi:2.7b

phi2 · 3B · Q4_0

1 benchmarks avg 59/100 56.6 tok/s

wizardlm2:7b

llama · 7B · Q4_0

1 benchmarks avg 59/100 23.2 tok/s

vicuna:13b

llama · 13B · Q4_0

1 benchmarks avg 59/100 13.4 tok/s

vicuna:7b

llama · 7B · Q4_0

1 benchmarks avg 59/100 23.4 tok/s

text-embedding-nomic-embed-text-v1.5

nomic-bert · Q4_K_M

1 benchmarks avg 59/100 45.3 tok/s

qwen2.5:0.5b

qwen2 · 494.03M · Q4_K_M

1 benchmarks avg 57/100 165 tok/s

deepseek-coder:6.7b

llama · 7B · Q4_0

1 benchmarks avg 57/100 24.1 tok/s

llama2:13b

llama · 13B · Q4_0

1 benchmarks avg 56/100 13.5 tok/s

llama2:7b

llama · 7B · Q4_0

1 benchmarks avg 55/100 25.1 tok/s

orca-mini:7b

llama · 7B · Q4_0

1 benchmarks avg 55/100 24.4 tok/s

smollm2:360m

llama · 361.82M · F16

1 benchmarks avg 52/100 117.9 tok/s

qwen3:8b

qwen3 · 8.2B · Q4_K_M

1 benchmarks avg 47/100 18.7 tok/s

qwen3:4b

qwen3 · 4.0B · Q4_K_M

1 benchmarks avg 47/100 11.3 tok/s

orca-mini:3b

llama · 3B · Q4_0

1 benchmarks avg 47/100 30.6 tok/s

stablelm2:1.6b

stablelm · 2B · Q4_0

1 benchmarks avg 45/100 93.8 tok/s

tinyllama:1.1b

llama · 1B · Q4_0

1 benchmarks avg 43/100 121 tok/s

smollm2:135m

llama · 134.52M · F16

1 benchmarks avg 41/100 241 tok/s

starcoder2:3b

starcoder2 · 3B · Q4_0

1 benchmarks avg 39/100 49.4 tok/s

starcoder2:7b

starcoder2 · 7B · Q4_0

1 benchmarks avg 35/100 23.8 tok/s

deepseek-r1:14b

qwen2 · 14.8B · Q4_K_M

1 benchmarks avg 22/100 11.4 tok/s