Benchmarks · agentic
AA-Briefcase v1.1
AA-Briefcase: agentic office-work tasks over real files (xlsx/pptx/docx/pdf) judged pairwise; reported as Elo.
105 models · 192 results · data as of 2026-11-06 · artificialanalysis.ai ↗
Leaderboard
Sonnet 5.5 max
Heavy line: best score at each price. Thin lines join one model's effort levels. Cost uses each model's median subscription plan, or API price with no plan data. 27 more models under All models in settings.
More from this benchmark
43 published measuresScore vs any published measure43 measures: sub-scores, splits, counts
82 more models under All models in settings.
Breakdown
darker is better within each column · click a column to sort| Model | ||||||
|---|---|---|---|---|---|---|
| Sonnet 5.5 max | 1,843 | 5,202 | 16,795 | 13,505 | 9,030 | 7,403 |
| Opus 5.5 max | 1,274 | 3,308 | 9,093 | 7,144 | 5,235 | 4,158 |
| Kimi K3.1 max | 598.0 | 2,611 | 5,707 | 4,952 | 4,014 | 2,107 |
| Fable 5.1 max | 625.0 | 2,010 | 5,302 | 5,383 | 3,430 | 2,209 |
| Muse Wonder (Max) | 642.0 | 1,374 | 4,848 | 5,383 | 4,632 | 904.0 |
| Opus 5 max | 937.0 | 2,274 | 5,349 | 5,347 | 4,100 | 1,978 |
| Grok 4.7 xhigh | 634.0 | 1,605 | 3,735 | 4,613 | 3,155 | 1,766 |
| Gemini 4 Carbon high | 448.0 | 2,341 | 2,488 | 3,059 | 2,444 | 406.0 |
| Muse Spark 1.3 max | 509.0 | 1,665 | 3,869 | 4,642 | 3,869 | 471.0 |
| Haiku 5.5 max | 1,234 | 1,153 | 8,021 | 5,002 | 3,948 | 5,309 |
| Qwen 3.8 Flash | 1,278 | 1,881 | 12,482 | 13,582 | 11,125 | 4,223 |
| DeepSeek V4.1 Flash 1017 max | 881.0 | 2,393 | 6,988 | 6,971 | 5,605 | 3,040 |
| GPT-6 Astra max | 279.0 | 1,058 | 2,308 | 1,853 | 1,568 | 589.0 |
| GPT-6.1 Sol max | 315.0 | 1,068 | 2,568 | 2,069 | 1,859 | 874.0 |
| Grok 4.6 xhigh | 363.0 | 1,284 | 2,378 | 3,346 | 2,021 | 704.0 |
| Fable 5 max, fallback | 643.0 | 1,945 | 3,726 | 3,749 | 2,702 | 1,445 |
| Inkling 2 Small | 702.0 | 4,096 | 5,647 | 5,596 | 4,928 | 1,829 |
| MiMo V2.6 Pro | 879.0 | 1,239 | 5,454 | 5,592 | 4,304 | 2,299 |
| GLM 5.3 max | 984.0 | 0 | 7,293 | 7,421 | 5,531 | 2,868 |
| Kimi K3 max | 744.0 | 2,311 | 4,953 | 4,879 | 3,598 | 1,802 |
| MiMo V2.6 Flash | 984.0 | 1,574 | 5,223 | 5,749 | 4,309 | 2,342 |
| Gemini 4 Argon high | 423.0 | 1,976 | 2,712 | 4,309 | 2,523 | 218.0 |
| GPT-5.6 Sol max | 677.0 | 1,023 | 3,697 | 2,976 | 2,298 | 1,347 |
| GPT-6 Sol max | 492.0 | 1,139 | 3,307 | 2,915 | 2,119 | 1,088 |
| GLM 5.3 Flash max | 1,097 | 1,631 | 7,372 | 7,769 | 6,166 | 3,474 |
All results
192 rows| # | Model | Effort | AA-Briefcase v1.1 | Subscription $ / task | API $ / task | Harness |
|---|---|---|---|---|---|---|
| 1 | Sonnet 5.5 | max | 1813 ±11 | $0.31 | $20 | Stirrup |
| 2 | Opus 5.5 | max | 1800 ±11 | $0.32 | $21 | Stirrup |
| 3 | Opus 5.5 | xhigh | 1760 ±11 | $0.19 | $12 | Stirrup |
| 4 | Sonnet 5.5 | xhigh | 1742 ±12 | $0.11 | $6.98 | Stirrup |
| 5 | Opus 5.5 | high | 1682 ±9 | $0.095 | $6.27 | Stirrup |
| 6 | Kimi K3.1 | max | 1675 ±11 | $7.22 | $7.22 | Stirrup |
| 7 | Fable 5.1 | max | 1669 ±8 | $3.54 | $23 | Stirrup |
| 8 | Muse Wonder (Max) | max | 1659 ±10 | $5.79 | $8.68 | Stirrup |
| 9 | Opus 5 | max | 1655 ±8 | $0.31 | $18 | Stirrup |
| 10 | Fable 5.1 | xhigh | 1649 ±8 | $2.77 | $18 | Stirrup |
| 11 | Grok 4.7 | xhigh | 1637 ±9 | $0.74 | $8.03 | Stirrup |
| 12 | Sonnet 5.5 | high | 1634 ±10 | $0.050 | $3.27 | Stirrup |
| 13 | Grok 4.7 | high | 1629 ±9 | $0.59 | $6.38 | Stirrup |
| 14 | Opus 5 | xhigh | 1629 ±8 | Stirrup | ||
| 15 | Opus 5.5 | medium | 1620 ±9 | $0.067 | $4.40 | Stirrup |
| 16 | Gemini 4 Carbon | high | 1611 ±10 | $2.91 | $8.16 | Stirrup |
| 17 | Gemini 4 Carbon | medium | 1602 ±11 | $2.52 | $7.06 | Stirrup |
| 18 | Muse Spark 1.3 | max | 1581 ±9 | $1.64 | $3.28 | Stirrup |
| 19 | Fable 5.1 | high | 1577 ±7 | $1.78 | $11 | Stirrup |
| 20 | Haiku 5.5 | max | 1574 ±10 | $0.014 | $0.67 | Stirrup |
| 21 | Qwen 3.8 Flash | 1574 ±8 | Stirrup | |||
| 22 | DeepSeek V4.1 Flash 1017 | max | 1570 ±10 | free | $0.68 | Stirrup |
| 23 | GPT-6 Astra | max | 1569 ±8 | $0.41 | $9.50 | Stirrup |
| 24 | Gemini 4 Carbon | low | 1567 ±11 | $2.57 | $7.20 | Stirrup |
| 25 | Opus 5 | high | 1557 ±7 | Stirrup | ||
| 26 | GPT-6.1 Sol | max | 1555 ±9 | $0.13 | $2.31 | Stirrup |
| 27 | GPT-6 Astra | xhigh | 1544 ±9 | $0.28 | $6.56 | Stirrup |
| 28 | Grok 4.6 | xhigh | 1541 ±7 | $0.50 | $4.88 | Stirrup |
| 29 | Fable 5 | max | 1535 ±7 | Stirrup | ||
| 30 | Grok 4.6 | high | 1528 ±7 | Stirrup |