Benchmarks · agentic
Ramp Accounting Bench
Long-horizon accounting work across realistic financial systems: 137 tasks, 3 attempts each in one shared agent harness (Mercor Archipelago), graded by GPT-5.6 Luna against per-task criteria. All models ran at high reasoning effort. Value is the mean share of criteria met across the three attempts. Cost is per attempt.
18 models · 18 results · updated 2026-10-31T01:02:02.814141Z · labs.ramp.com ↗
Leaderboard
Gemini 4 Argon high
Heavy line: best score at each price. Thin lines join one model's effort levels. Cost uses each model's median subscription plan, or API price with no plan data. 6 more models under All models in settings.
More from this benchmark
10 published measuresTime per task18 results · lower is better
7 more models under All models in settings.
Output tokens per task18 results · lower is better
Score vs any published measure8 measures: sub-scores, splits, counts
Breakdown
darker is better within each column · click a column to sort| Model | |||||||
|---|---|---|---|---|---|---|---|
| Gemini 4 Argon high | 23.4 | 26.3 | 16.8 | 2.92 | 9.49 | 76.4 | 39.7 |
| Opus 5.5 high | 10.9 | 19.7 | 8.03 | 8.76 | 11.7 | 31.4 | 20.2 |
| Fable 5.1 high | 12.4 | 19.7 | 6.57 | 7.3 | 13.1 | 34.3 | 18.5 |
| Muse Wonder (Max) high | 14.6 | 19.7 | 10.2 | 5.11 | 9.49 | 58.9 | 38.2 |
| GPT-6 Astra high | 11.7 | 19.0 | 8.03 | 7.3 | 10.9 | 33.3 | 16.8 |
| GPT-6.1 Sol high | 10.9 | 16.8 | 5.84 | 5.84 | 10.9 | 36.1 | 17.1 |
| Gemini 3.8 Flash high | 10.9 | 14.6 | 5.84 | 3.65 | 8.76 | 95.3 | 94.3 |
| Grok 4.7 high | 9.49 | 19.0 | 5.84 | 9.49 | 13.1 | 90.8 | 35.6 |
| GLM 5.3 high | 11.7 | 21.2 | 2.92 | 9.49 | 18.3 | 56.4 | 33.7 |
| Muse Spark 1.3 high | 9.49 | 16.1 | 4.38 | 6.57 | 11.7 | 65.7 | 55.3 |
| Sonnet 5.5 high | 8.03 | 12.4 | 2.92 | 4.38 | 9.49 | 31.3 | 20.5 |
| Gemini 3.7 Flash high | 9.49 | 15.3 | 6.57 | 5.84 | 8.76 | 57.3 | 56.5 |
| Haiku 5.5 high | 7.3 | 13.9 | 1.46 | 6.57 | 12.4 | 49.4 | 22.5 |
| Qwen 3.8 Max high | 12.4 | 19.7 | 1.46 | 7.3 | 18.3 | 62.0 | 39.3 |
| DeepSeek V4.1 Flash high | 12.4 | 16.8 | 2.19 | 4.38 | 14.6 | 70.7 | 35.4 |
| Kimi K3 high | 10.2 | 13.9 | 1.46 | 3.65 | 12.4 | 50.2 | 27.0 |
| GLM 5.3 Flash high | 7.3 | 13.9 | 1.46 | 6.57 | 12.4 | 43.9 | 29.2 |
| GPT-6 Luna high | 7.3 | 10.9 | 0.73 | 3.65 | 10.2 | 51.3 | 23.5 |
All results
18 rows| # | Model | Effort | Ramp Accounting Bench | Subscription $ / task | API $ / task | Time | Output tokens | Harness |
|---|---|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon | high | 61.1% | $2.16 | $3.03 | 17 min | 86k | Mercor Archipelago |
| 2 | Opus 5.5 | high | 50.5% | $0.020 | $1.34 | 3.4 min | 20k | Mercor Archipelago |
| 3 | Fable 5.1 | high | 49.7% | $0.54 | $3.48 | 5.7 min | 23k | Mercor Archipelago |
| 4 | Muse Wonder (Max) | high | 48.7% | $1.76 | $2.65 | 4.5 min | 24k | Mercor Archipelago |
| 5 | GPT-6 Astra | high | 48.7% | $0.099 | $2.31 | 5.0 min | 12k | Mercor Archipelago |
| 6 | GPT-6.1 Sol | high | 46.4% | $0.024 | $0.42 | 4.9 min | 11k | Mercor Archipelago |
| 7 | Gemini 3.8 Flash | high | 45.7% | $0.060 | $1.47 | 8.9 min | 83k | Mercor Archipelago |
| 8 | Grok 4.7 | high | 44.1% | $0.29 | $3.11 | 15 min | 55k | Mercor Archipelago |
| 9 | GLM 5.3 | high | 43.1% | $0.22 | $2.16 | 37 min | 140k | Mercor Archipelago |
| 10 | Muse Spark 1.3 | high | 42.5% | $0.78 | $1.56 | 3.8 min | 31k | Mercor Archipelago |
| 11 | Sonnet 5.5 | high | 42.5% | $0.010 | $0.67 | 2.6 min | 20k | Mercor Archipelago |
| 12 | Gemini 3.7 Flash | high | 42.3% | $0.11 | $0.85 | 5.9 min | 44k | Mercor Archipelago |
| 13 | Haiku 5.5 | high | 42.3% | $0.004 | $0.21 | 4.2 min | 52k | Mercor Archipelago |
| 14 | Qwen 3.8 Max | high | 41.2% | $0.34 | $2.85 | 15 min | 63k | Mercor Archipelago |
| 15 | DeepSeek V4.1 Flash | high | 40.8% | $0.009 | $0.12 | 12 min | 62k | Mercor Archipelago |
| 16 | Kimi K3 | high | 39.2% | $0.14 | $1.55 | 11 min | 31k | Mercor Archipelago |
| 17 | GLM 5.3 Flash | high | 35.7% | free | $0.18 | 17 min | 112k | Mercor Archipelago |
| 18 | GPT-6 Luna | high | 34.0% | $0.004 | $0.051 | 5.4 min | 30k | Mercor Archipelago |