Benchmarks · reliability
Tool-call error rate
Volume-weighted mean tool-call error rate across OpenRouter providers (3d).
175 models · 175 results · data as of 2026-11-06 · openrouter.ai ↗
Leaderboard
GPT-5 Pro high
All results
175 rows| # | Model | Effort | Tool-call error rate |
|---|---|---|---|
| 1 | GPT-5 Pro | high | 0.0% |
| 2 | GPT-5.2 Pro | xhigh | 0.0% |
| 3 | GPT-5.4 Pro | xhigh | 0.0% |
| 4 | Mistral Small 3.1 | none | 0.0% |
| 5 | GPT-6 Astra Pro | max | 0.0% |
| 6 | Kimi K3.1 | max | 0.1% |
| 7 | Kimi K3 | max | 0.1% |
| 8 | GPT-6 Sol | max | 0.1% |
| 9 | Grok 4.3 | xhigh | 0.1% |
| 10 | GPT-6 Astra | max | 0.1% |
| 11 | Hy4 | high | 0.1% |
| 12 | Hy4 preview | high | 0.2% |
| 13 | MiMo V2.5 | xhigh | 0.2% |
| 14 | Grok Build 0.1 | 0.2% | |
| 15 | GLM 4.5 | xhigh | 0.2% |
| 16 | Opus 4.6 | max | 0.2% |
| 17 | Grok 4.5 | max | 0.2% |
| 18 | Devstral 2 | none | 0.2% |
| 19 | GPT-5.5 Pro | xhigh | 0.3% |
| 20 | Grok 4.7 | max | 0.3% |
| 21 | GPT-6.1 Sol Pro | 0.3% | |
| 22 | GPT-6.1 Sol | max | 0.3% |
| 23 | GPT-5.6 Terra Pro | 0.3% | |
| 24 | Grok 4.6 | max | 0.3% |
| 25 | Muse Spark Open | xhigh | 0.3% |
| 26 | GPT-5.1 Codex Mini | high | 0.3% |
| 27 | GPT-6 Sol Pro | max | 0.3% |
| 28 | Qwen 3.5 Plus | none | 0.3% |
| 29 | Qwen 3 30B-A3B | none | 0.4% |
| 30 | Mistral Medium 3.5 | high | 0.4% |