Compare Models
Select up to 4 models to compare benchmarks, pricing, and capabilities side by side.
Anthropic
Mistral AI
DeepSeek
Add Model
MMLU
Claude Haiku 3.5
85.2
Mistral Small
81.2
DeepSeek-R1-Distill-Llama-8B
73.0
HumanEval
Claude Haiku 3.5
88.1
Mistral Small
84.8
DeepSeek-R1-Distill-Llama-8B
74.0
GSM8K
Claude Haiku 3.5
91.6
Mistral Small
88.4
DeepSeek-R1-Distill-Llama-8B
87.0
GPQA
Claude Haiku 3.5
41.6
Mistral Small
37.5
DeepSeek-R1-Distill-Llama-8B
44.0
MGSM
Claude Haiku 3.5
88.5
Mistral Small
80.1
DeepSeek-R1-Distill-Llama-8B
0.0
ARC-Challenge
Claude Haiku 3.5
93.5
Mistral Small
89.5
DeepSeek-R1-Distill-Llama-8B
0.0
HellaSwag
Claude Haiku 3.5
89.5
Mistral Small
84.0
DeepSeek-R1-Distill-Llama-8B
78.0
MATH
Claude Haiku 3.5
69.2
Mistral Small
61.0
DeepSeek-R1-Distill-Llama-8B
80.0
SWE-bench
Claude Haiku 3.5
40.6
Mistral Small
18.5
DeepSeek-R1-Distill-Llama-8B
0.0
MMMLU
Claude Haiku 3.5
81.7
Mistral Small
73.2
DeepSeek-R1-Distill-Llama-8B
0.0
| Model | Input | Output | Blended* |
|---|---|---|---|
Claude Haiku 3.5 | $0.80 | $4.00 | $2.40 |
Mistral Small | $0.10 | $0.30 | $0.20 |
DeepSeek-R1-Distill-Llama-8B | $0.07 | $0.14 | $0.11 |
*Blended = average of input and output price
| Spec | Claude Haiku 3.5 | Mistral Small | DeepSeek-R1-Distill-Llama-8B |
|---|---|---|---|
| Context Window | 200K | 32K | 128K |
| Max Output | 8K | 4K | N/A |
| TTFT | 150ms | 140ms | N/A |
| Speed | 160 tok/s | 170 tok/s | N/A |
| Parameters | N/A | 24B | 8B |
| Architecture | Transformer | Transformer | Dense Transformer |
| Open Source | No | No | Yes |
| Tier | budget | budget | budget |
Quick Verdict
Best Performance
Claude Haiku 3.5
Best Value
DeepSeek-R1-Distill-Llama-8B
Fastest
Mistral Small