Benchmarks · coding
SWE-bench Verified (Vals)
Solving production software engineering tasks Scores are the values published by VALS AI. Source benchmark version 1.
53 models · 53 results · updated 2026-09-01 · vals.ai ↗
Leaderboard
Opus 5 max
Heavy line: best score at each price. Thin lines join one model's effort levels. Cost uses each model's median subscription plan, or API price with no plan data. 38 more models under All models in settings.
More from this benchmark
5 published measuresTime per task87 results · lower is better
37 more models under All models in settings.
Score vs any published measure4 measures: sub-scores, splits, counts
Breakdown
darker is better within each column · click a column to sort| Model | ||||
|---|---|---|---|---|
| Opus 5 max | 90.5 | 96.9 | 98.5 | 100.0 |
| DeepSeek V4 Pro 0813 max | 100.0 | 95.0 | 97.4 | 100.0 |
| GPT-5.6 Sol max | 97.6 | 95.0 | 97.4 | 100.0 |
| Grok 4.6 high | 92.9 | 95.8 | 95.9 | 100.0 |
| GLM 5.3 max | 90.5 | 94.6 | 97.4 | 100.0 |
| GPT-5.6 Terra max | 97.6 | 93.9 | 96.9 | 100.0 |
| Fable 5 max | 92.9 | 94.6 | 95.9 | 100.0 |
| Kimi K3 max | 92.9 | 94.6 | 92.3 | 66.7 |
| GPT-5.6 Luna max | 85.7 | 92.0 | 96.4 | 66.7 |
| GLM 5.3 Flash max | 83.3 | 92.3 | 93.3 | 100.0 |
| DeepSeek V4 Flash 0731 high | 90.5 | 87.4 | 90.7 | 66.7 |
| Opus 4.8 max | 73.8 | 88.1 | 92.8 | 66.7 |
| Grok 4.5 high | 73.8 | 83.5 | 93.8 | 66.7 |
| Muse Spark 1.2 xhigh | 64.3 | 86.6 | 92.3 | 33.3 |
| Qwen 3.8 27B xhigh | 69.0 | 85.8 | 90.2 | 66.7 |
| Qwen 3.8 Max | 64.3 | 83.5 | 93.3 | 66.7 |
| GLM 5.2 max | 66.7 | 81.6 | 88.1 | 66.7 |
| GPT-5.5 xhigh | 50.0 | 81.2 | 91.8 | 66.7 |
| Inkling Small max | 42.9 | 80.5 | 93.8 | 33.3 |
| Opus 4.7 max | 64.3 | 78.9 | 90.2 | 66.7 |
| Muse Spark 1.1 xhigh | 52.4 | 82.4 | 88.1 | 66.7 |
| Gemini 3.7 Flash high | 57.1 | 80.5 | 87.1 | 33.3 |
| Gemini 3.8 Flash high | 52.4 | 77.8 | 89.7 | 33.3 |
| Sonnet 5 max | 76.2 | 77.4 | 83.5 | 66.7 |
| Composer 2.5 none | 52.4 | 78.9 | 86.6 | 66.7 |
All results
53 rows| # | Model | Effort | Score | Subscription $ / task | API $ / task | Time | Harness |
|---|---|---|---|---|---|---|---|
| 1 | Opus 5 | max | 97.0% | $0.022 | $1.29 | 9.6 min | |
| 2 | DeepSeek V4 Pro 0813 | max | 96.4% | $0.10 | $0.10 | 4.0 min | |
| 3 | GPT-5.6 Sol | max | 96.2% | $0.032 | $1.15 | 3.0 min | |
| 4 | Grok 4.6 | high | 95.6% | $0.081 | $0.78 | 10 min | |
| 5 | GLM 5.3 | max | 95.4% | $0.035 | $0.34 | 14 min | |
| 6 | GPT-5.6 Terra | max | 95.4% | $0.011 | $0.40 | 3.0 min | |
| 7 | Fable 5 | max | 95.0% | $0.23 | $2.05 | 5.9 min | |
| 8 | Kimi K3 | max | 93.4% | $0.069 | $0.76 | 10 min | |
| 9 | GPT-5.6 Luna | max | 93.0% | $0.002 | $0.043 | 3.4 min | |
| 10 | GLM 5.3 Flash | max | 92.0% | free | $0.018 | 1.1 h | |
| 11 | DeepSeek V4 Flash 0731 | high | 88.8% | $0.010 | $0.010 | 3.4 min | |
| 12 | Opus 4.8 | max | 88.6% | $0.033 | $1.92 | 9.4 min | |
| 13 | Grok 4.5 | high | 86.6% | $0.085 | $0.54 | 3.3 min | |
| 14 | Muse Spark 1.2 | xhigh | 86.6% | $0.28 | $0.55 | 8.1 min | |
| 15 | Qwen 3.8 27B | xhigh | 86.0% | $0.079 | $1.08 | 10.0 min | |
| 16 | Qwen 3.8 Max | max | 85.6% | $0.13 | $1.13 | 42 min | |
| 17 | GLM 5.2 | max | 82.8% | $0.021 | $0.71 | 11 min | |
| 18 | GPT-5.5 | xhigh | 82.6% | $0.038 | $1.36 | 7.1 min | |
| 19 | Inkling Small | max | 82.2% | $0.11 | $0.13 | 4.1 min | |
| 20 | Opus 4.7 | max | 82.0% | $2.42 | $2.42 | 7.4 min | |
| 21 | Muse Spark 1.1 | xhigh | 82.0% | $0.35 | $0.35 | 3.5 min | |
| 22 | Gemini 3.7 Flash | high | 80.8% | $0.19 | $1.44 | 5.8 min | |
| 23 | Gemini 3.8 Flash | high | 80.0% | $0.089 | $2.19 | 11 min | |
| 24 | Sonnet 5 | max | 79.6% | $0.026 | $1.49 | 16 min | |
| 25 | Composer 2.5 | none | 79.6% | Cursor CLI | |||
| 26 | Gemini 3.6 Flash | high | 79.6% | $0.038 | $1.19 | 13 min | |
| 27 | Gemini 3.5 Flash | high | 78.8% | $0.95 | $0.95 | 4.2 min | |
| 28 | Kimi K2.7 Code | max | 78.2% | $0.058 | $0.27 | 15 min | |
| 29 | GPT-5.3 Codex | xhigh | 78.0% | $0.46 | $0.46 | 4.1 min | |
| 30 | Inkling | max | 77.6% | $0.054 | $0.59 | 6.6 min |