Compare Models
Select up to 4 models to compare benchmarks, pricing, and capabilities side by side.
Anthropic
Mistral AI
OpenAI
Add Model
MMLU
Claude Haiku 3.5
85.2
Mistral Small
81.2
GPT-4.1 Mini
84.5
HumanEval
Claude Haiku 3.5
88.1
Mistral Small
84.8
GPT-4.1 Mini
88.8
GSM8K
Claude Haiku 3.5
91.6
Mistral Small
88.4
GPT-4.1 Mini
94.0
GPQA
Claude Haiku 3.5
41.6
Mistral Small
37.5
GPT-4.1 Mini
45.0
MGSM
Claude Haiku 3.5
88.5
Mistral Small
80.1
GPT-4.1 Mini
88.0
ARC-Challenge
Claude Haiku 3.5
93.5
Mistral Small
89.5
GPT-4.1 Mini
94.5
HellaSwag
Claude Haiku 3.5
89.5
Mistral Small
84.0
GPT-4.1 Mini
90.5
MATH
Claude Haiku 3.5
69.2
Mistral Small
61.0
GPT-4.1 Mini
72.0
SWE-bench
Claude Haiku 3.5
40.6
Mistral Small
18.5
GPT-4.1 Mini
28.0
MMMLU
Claude Haiku 3.5
81.7
Mistral Small
73.2
GPT-4.1 Mini
80.2
| Model | Input | Output | Blended* |
|---|---|---|---|
Claude Haiku 3.5 | $0.80 | $4.00 | $2.40 |
Mistral Small | $0.10 | $0.30 | $0.20 |
GPT-4.1 Mini | $0.40 | $1.60 | $1.00 |
*Blended = average of input and output price
| Spec | Claude Haiku 3.5 | Mistral Small | GPT-4.1 Mini |
|---|---|---|---|
| Context Window | 200K | 32K | 1.0M |
| Max Output | 8K | 4K | 33K |
| TTFT | 150ms | 140ms | 170ms |
| Speed | 160 tok/s | 170 tok/s | 150 tok/s |
| Parameters | N/A | 24B | N/A |
| Architecture | Transformer | Transformer | Transformer |
| Open Source | No | No | No |
| Tier | budget | budget | budget |
Quick Verdict
Best Performance
Claude Haiku 3.5
Best Value
Mistral Small
Fastest
Mistral Small