Benchmarks · agentic
Terminal-Bench Science (Vals)
Research workflow tasks contributed by practicing scientists across five domains
36 models · 37 results · updated 2026-11-05 · vals.ai ↗
Leaderboard
GPT-6 Astra max
Heavy line: best score at each price. Thin lines join one model's effort levels. Cost uses each model's median subscription plan, or API price with no plan data. 24 more models under All models in settings.
More from this benchmark
1 published measuresTime per task44 results · lower is better
22 more models under All models in settings.
All results
37 rows| # | Model | Effort | Score | Subscription $ / task | API $ / task | Time | Harness |
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | max | 62.9% | $0.89 | $21 | 1.8 h | Mini-SWE-agent |
| 2 | GPT-6.1 Sol | max | 52.9% | $0.20 | $3.44 | 1.4 h | Mini-SWE-agent |
| 3 | Opus 5.5 | max | 47.1% | $0.29 | $19 | 2.5 h | Mini-SWE-agent |
| 4 | Sonnet 5.5 | max | 47.1% | $0.30 | $20 | 2.8 h | Mini-SWE-agent |
| 5 | Gemini 4 Argon | high | 44.3% | $32 | $45 | 1.8 h | Mini-SWE-agent |
| 6 | Fable 5.1 | max | 40.0% | $5.93 | $38 | 2.6 h | Mini-SWE-agent |
| 7 | GPT-6 Sol | max | 30.0% | $0.18 | $5.82 | 1.4 h | Mini-SWE-agent |
| 8 | Opus 5 | max | 27.1% | $0.56 | $33 | 2.2 h | Mini-SWE-agent |
| 9 | GPT-5.6 Sol | max | 20.0% | $0.20 | $7.32 | 1.7 h | Mini-SWE-agent |
| 10 | Fable 5 | max | 15.7% | $5.86 | $52 | 2.0 h | Mini-SWE-agent |
| 11 | Qwen 3.8 Max | max | 12.9% | $1.97 | $17 | 3.4 h | Mini-SWE-agent |
| 12 | Grok 4.6 | high | 11.4% | $0.50 | $4.82 | 1.3 h | Mini-SWE-agent |
| 13 | Haiku 5.5 | max | 10.0% | $0.096 | $4.52 | 1.6 h | Mini-SWE-agent |
| 14 | GPT-5.6 Terra | max | 10.0% | $0.14 | $5.19 | 2.1 h | Mini-SWE-agent |
| 15 | Grok 4.7 | xhigh | 10.0% | $1.58 | $17 | 1.5 h | Mini-SWE-agent |
| 16 | Muse Spark 1.3 | max | 10.0% | $3.05 | $6.09 | 1.4 h | Mini-SWE-agent |
| 17 | Gemini 3.8 Flash | high | 8.6% | $0.23 | $5.64 | 55 min | Mini-SWE-agent |
| 18 | Gemini 3.7 Flash | high | 7.1% | $0.93 | $7.19 | 1.3 h | Mini-SWE-agent |
| 19 | Sonnet 5 | max | 5.7% | $0.53 | $30 | 3.1 h | Mini-SWE-agent |
| 20 | Gemini 3.5 Flash | high | 5.7% | $3.80 | $3.80 | 1.0 h | Mini-SWE-agent |
| 21 | GLM 5.3 | max | 5.7% | $1.62 | $16 | 2.7 h | Mini-SWE-agent |
| 22 | Opus 4.8 | max | 4.3% | $0.40 | $23 | 1.9 h | Mini-SWE-agent |
| 23 | DeepSeek V4 Flash 0731 | high | 4.3% | $1.10 | $1.10 | 1.6 h | Mini-SWE-agent |
| 24 | DeepSeek V4 Pro 0813 | max | 4.3% | $4.63 | $4.63 | 2.3 h | Mini-SWE-agent |
| 25 | Gemini 3.6 Flash | high | 4.3% | $0.14 | $4.39 | 1.9 h | Mini-SWE-agent |
| 26 | GPT-6 Luna | max | 4.3% | $0.018 | $0.24 | 1.3 h | Mini-SWE-agent |
| 27 | MiMo V2.6 Flash | high | 4.3% | free | $0.24 | 4.0 h | Mini-SWE-agent |
| 28 | Muse Spark 1.3 | xhigh | 4.3% | $2.20 | $4.41 | 1.1 h | Mini-SWE-agent |
| 29 | GLM 5.3 Flash | max | 2.9% | free | $0.52 | 3.5 h | Mini-SWE-agent |
| 30 | Inkling 2 Small | max | 2.9% | $5.55 | $5.55 | 2.5 h | Mini-SWE-agent |