A
Quality benchmark
Use standardized tasks to detect broad capability differences, but verify the exact evaluation protocol before comparing results from different sources.
Benchmark notebook
Numbers need context.
The tables below reproduce selected scores reported by model creators. They are useful for orientation, but benchmark setup, prompting, sampling and evaluation harnesses can differ between releases.
Reasoning distillation
DeepSeek R1 Distill
Selected results reported in the original DeepSeek-R1 release. Higher is better for all four metrics shown.
| Model | AIME 2024 | MATH-500 | GPQA Diamond | LiveCodeBench |
|---|---|---|---|---|
| R1 Distill Qwen 7B | 55.5 | 92.8 | 49.1 | 37.6 |
| R1 Distill Qwen 14B | 69.7 | 93.9 | 59.1 | 53.1 |
| R1 Distill Qwen 32B | 72.6 | 94.3 | 62.1 | 57.2 |
AIME 2024 pass@1
Source note: DeepSeek-R1 official release, January 2025. Scores are publisher-reported results, not measurements performed by this site.
Coding agents
Qwen3.6 27B release snapshot
Selected coding-agent results reported by the Qwen Team in April 2026. The same release table included previous Qwen models and Gemma 4 31B for comparison.
| Model | SWE-bench Verified | SWE-bench Pro | Terminal-Bench 2.0 | SkillsBench |
|---|---|---|---|---|
| Qwen3.5 27B | 75.0 | 51.2 | 41.6 | 27.2 |
| Gemma 4 31B | 52.0 | 35.7 | 42.9 | 23.6 |
| Qwen3.6 27B | 77.2 | 53.5 | 59.3 | 48.2 |
Source note: Qwen3.6-27B official release, April 2026. Cross-vendor benchmark tables should be read as one publisher's evaluation snapshot, not as an independent universal ranking.
B
Throughput benchmark
Measure prompt tokens per second and generated tokens per second separately. Long prompts can stress memory bandwidth very differently from decoding.
C
Application benchmark
The most useful test is your own workload: the same prompts, context sizes, structured outputs and quality checks you expect in production.