Benchmarks · reasoning
LLM2014 Logic
A private, monthly-rotating bank of about 28 logic and reasoning questions (interleaved text, rule induction, log analysis, puzzles), three runs each; the derivation must be correct, guesses earn nothing. Median score out of 100.
26 models · 27 results · updated 2026-11 · llm2014.github.io ↗
Leaderboard
GPT-6 Astra xhigh
More from this benchmark
3 published measuresTime per task27 results · lower is better
13 more models under All models in settings.
Output tokens per task27 results · lower is better
Score vs any published measure1 measures: sub-scores, splits, counts
All results
27 rows| # | Model | Effort | LLM2014 Logic | Time | Output tokens |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | xhigh | 86.5 | 5.2 min | 13k |
| 2 | Opus 5.5 | xhigh | 77.7 | 4.9 min | 33k |
| 3 | Fable 5.1 | xhigh | 70.5 | 8.0 min | 41k |
| 4 | Gemini 4 Argon | high | 69.1 | 7.5 min | 24k |
| 5 | GPT-6.1 Sol | xhigh | 67.8 | 10 min | 13k |
| 6 | Kimi K3.1 | max | 61.6 | 22 min | 51k |
| 7 | Muse Wonder (Max) | xhigh | 59.6 | 11 min | 54k |
| 8 | Sonnet 5.5 | xhigh | 57.4 | 6.1 min | 40k |
| 9 | DeepSeek V4.1 Flash 1017 | max | 56.0 | 9.9 min | 77k |
| 10 | Grok 4.7 | xhigh | 52.5 | 24 min | 81k |
| 11 | MiniMax M3.1 | max | 51.8 | 12 min | 53k |
| 12 | Gemini 3.8 Flash | high | 51.3 | 6.5 min | 37k |
| 13 | GLM 5.3 | max | 47.7 | 26 min | 68k |
| 14 | DeepSeek V4 Pro 0813 | max | 47.4 | 28 min | 62k |
| 15 | Qwen 3.8 Max | xhigh | 45.3 | 27 min | 72k |
| 16 | GLM 5.3 Flash | max | 44.6 | 22 min | 51k |
| 17 | Qwen 3.8 Flash | xhigh | 37.6 | 17 min | 68k |
| 18 | Haiku 5.5 | xhigh | 32.3 | 5.3 min | 59k |
| 19 | Qwen 3.8 27B | xhigh | 31.2 | 38 min | 73k |
| 20 | Hy3 | high | 28.1 | 19 min | 48k |
| 21 | Ling 3.1 Flash | xhigh | 25.1 | 17 min | 88k |
| 22 | GPT-6 Luna | xhigh | 23.9 | 5.7 min | 28k |
| 23 | Gemini 3.5 Flash-Lite | high | 11.9 | 3.1 min | 16k |
| 24 | GPT-5.5 Instant | 9.4 | 48s | 1.9k | |
| 25 | Mistral Large 4 | high | 6.4 | 9.3 min | 28k |
| 26 | DeepSeek V4.1 Flash | max | 3.4 | 29s | 3.6k |
| 27 | Qwen 3.8 27B | max | 2.9 | 3.7 min | 8.1k |