All Benchmarked Models

61 models tested across real hardware. Click any model to see detailed benchmark results.

gemma3:1b

gemma3 · 999.89M · Q4_K_M

4 benchmarks avg 68/100 58.6 tok/s

llama3.1:8b

llama · 8.0B · Q4_K_M

3 benchmarks avg 70/100 15.3 tok/s

qwen3:0.6b

qwen3 · 751.63M · Q4_K_M

3 benchmarks avg 73/100 144.5 tok/s

qwen2.5:7b

qwen2 · 7.6B · Q4_K_M

2 benchmarks avg 80/100 21.3 tok/s

gemma3:4b

gemma3 · 4.3B · Q4_K_M

2 benchmarks avg 78/100 35.2 tok/s

qwen2.5-coder:7b

qwen2 · 7.6B · Q4_K_M

2 benchmarks avg 76/100 21.8 tok/s

qwen2.5:3b

qwen2 · 3.1B · Q4_K_M

2 benchmarks avg 76/100 42.9 tok/s

gemma3:12b

gemma3 · 12.2B · Q4_K_M

2 benchmarks avg 74/100 12.5 tok/s

gemma2:2b

gemma2 · 2.6B · Q4_0

2 benchmarks avg 76/100 54.4 tok/s

llama3.2:3b

llama · 3.2B · Q4_K_M

2 benchmarks avg 74/100 36 tok/s

gemma2:9b

gemma2 · 9.2B · Q4_0

2 benchmarks avg 74/100 17.2 tok/s

mistral:latest

llama · 7.2B · Q4_0

2 benchmarks avg 76/100 38.4 tok/s

mistral:7b

llama · 7.2B · Q4_K_M

2 benchmarks avg 68/100 17.8 tok/s

phi4:14b

phi3 · 14.7B · Q4_K_M

2 benchmarks avg 70/100 11.1 tok/s

codegemma:7b

gemma · 9B · Q4_0

2 benchmarks avg 70/100 18.9 tok/s

qwen2.5:1.5b

qwen2 · 1.5B · Q4_K_M

2 benchmarks avg 71/100 82.8 tok/s

llama3:latest

llama · 8.0B · Q4_0

2 benchmarks avg 71/100 21.8 tok/s

granite3.1-dense:2b

granite · 2.5B · Q4_K_M

2 benchmarks avg 70/100 54 tok/s

smollm2:1.7b

llama · 1.7B · Q8_0

2 benchmarks avg 66/100 51.1 tok/s

llama3.2:1b

llama · 1.2B · Q8_0

2 benchmarks avg 63/100 48.9 tok/s

qwen2.5:0.5b

qwen2 · 494.03M · Q4_K_M

2 benchmarks avg 61/100 163.9 tok/s

smollm2:360m

llama · 361.82M · F16

2 benchmarks avg 59/100 116.5 tok/s

stablelm2:1.6b

stablelm · 2B · Q4_0

2 benchmarks avg 51/100 78.3 tok/s

tinyllama:1.1b

llama · 1B · Q4_0

2 benchmarks avg 51/100 125.3 tok/s

qwen3:8b

qwen3 · 8.2B · Q4_K_M

2 benchmarks avg 46/100 18.4 tok/s

smollm2:135m

llama · 134.52M · F16

2 benchmarks avg 49/100 243 tok/s

phi4-mini

2 benchmarks avg 39/100 21.8 tok/s

deepseek-r1:14b

qwen2 · 14.8B · Q4_K_M

2 benchmarks avg 26/100 11.1 tok/s

qwen2.5:14b

qwen2 · 14.8B · Q4_K_M

1 benchmarks avg 82/100 11.2 tok/s

qwen3.5:4b

qwen35 · 4.7B · Q4_K_M

1 benchmarks avg 81/100 16.6 tok/s

yi:6b

llama · 6B · Q4_0

1 benchmarks avg 80/100 27.6 tok/s

llama3.2:latest

llama · 3.2B · Q4_K_M

1 benchmarks avg 77/100 98.9 tok/s

qwen3.5:2b

qwen35 · 2.3B · Q8_0

1 benchmarks avg 77/100 30.4 tok/s

granite3.1-dense:8b

granite · 8.2B · Q4_K_M

1 benchmarks avg 76/100 18.8 tok/s

phi4-mini:latest

phi3 · 3.8B · Q4_K_M

1 benchmarks avg 75/100 36.2 tok/s

mistral-nemo:12b

llama · 12.2B · Q4_0

1 benchmarks avg 73/100 14.6 tok/s

deepseek-v2:16b

deepseek2 · 15.7B · Q4_0

1 benchmarks avg 73/100 56.6 tok/s

neural-chat:7b

llama · 7B · Q4_0

1 benchmarks avg 71/100 22.5 tok/s

nous-hermes2:latest

llama · 11B · Q4_0

1 benchmarks avg 71/100 15.9 tok/s

qwen3:1.7b

qwen3 · 2.0B · Q4_K_M

1 benchmarks avg 70/100 37.9 tok/s

aya:8b

command-r · 8.0B · F16

1 benchmarks avg 70/100 20 tok/s

qwen3.5:0.8b

qwen35 · 873.44M · Q8_0

1 benchmarks avg 70/100 48.2 tok/s

phi3:3.8b

phi3 · 3.8B · Q4_0

1 benchmarks avg 69/100 42 tok/s

starling-lm:7b

llama · 7B · Q4_0

1 benchmarks avg 68/100 23.7 tok/s

codellama:7b

llama · 7B · Q4_0

1 benchmarks avg 68/100 23.6 tok/s

solar:10.7b

llama · 11B · Q4_0

1 benchmarks avg 66/100 15.8 tok/s

phi3:14b

phi3 · 14.0B · Q4_0

1 benchmarks avg 66/100 12.7 tok/s

dolphin-phi:2.7b

phi2 · 3B · Q4_0

1 benchmarks avg 66/100 56.1 tok/s

vicuna:7b

llama · 7B · Q4_0

1 benchmarks avg 65/100 23.4 tok/s

phi:2.7b

phi2 · 3B · Q4_0

1 benchmarks avg 65/100 56.6 tok/s

wizardlm2:7b

llama · 7B · Q4_0

1 benchmarks avg 65/100 23.2 tok/s

deepseek-coder:6.7b

llama · 7B · Q4_0

1 benchmarks avg 63/100 24.1 tok/s

vicuna:13b

llama · 13B · Q4_0

1 benchmarks avg 62/100 13.4 tok/s

orca-mini:7b

llama · 7B · Q4_0

1 benchmarks avg 61/100 24.4 tok/s

llama2:7b

llama · 7B · Q4_0

1 benchmarks avg 61/100 25.1 tok/s

llama2:13b

llama · 13B · Q4_0

1 benchmarks avg 60/100 13.5 tok/s

orca-mini:3b

llama · 3B · Q4_0

1 benchmarks avg 54/100 30.6 tok/s

qwen3:4b

qwen3 · 4.0B · Q4_K_M

1 benchmarks avg 48/100 11.3 tok/s

starcoder2:3b

starcoder2 · 3B · Q4_0

1 benchmarks avg 48/100 49.4 tok/s

falcon:7b

falcon · 7B · Q4_0

1 benchmarks avg 47/100 24.3 tok/s

starcoder2:7b

starcoder2 · 7B · Q4_0

1 benchmarks avg 44/100 23.8 tok/s