Benchmarks
Every benchmark has its own leaderboard and everything its publisher releases beyond the score. Where one measures cost per task, Priors adjusts it for subscription plans, so you see what a model costs to actually use.
326 benchmarksup to 445 models each114 publishers
/
How scores work326 benchmarks
- Terminal-Bench v4-AAAA's 66-task Terminal-Bench v4.0, three repeats, average pass@1. Distinct from the upstream leaderboard and v2.1.
artificialanalysis.aiTop model: Kimi K3.1154 models32 mixesQuality 100
- HLE-DiamondHLE-Diamond: the refined 1,000-question version of Humanity's Last Exam from CAIS and Scale AI, split evenly between reasoning and knowledge questions; accuracy.
- FrontierCode 2.0-AAAA's run of Cognition's FrontierCode 2.0 (132 private tasks in real open-source repositories, three attempts each): would a maintainer merge the change? An attempt failing any blocking check scores 0. Separate from Cognition's own leaderboard, which uses different harnesses.
- Terminal-Bench-Science-AATerminal-Bench-Science run by AA: 70 expert-curated scientific research workflows across five domains in a terminal; a task passes only if every one of its tests passes. Pass@1 over three repeats.
- FrontierMath Tier 4 (v2)1 sub-boardEpoch AI's FrontierMath Tier 4, version 2: expert-written mathematics problems evaluated with Python tools. Accuracy on the private set, not the public sample questions. Data: Epoch AI, CC BY.
epoch.aiTop model: GPT-6.1 Sol68 models10 mixesQuality 90
- GDP.pdfGDP.pdf: document-heavy professional tasks over PDFs graded against rubric criteria; share of tasks passing all criteria.
- AutomationBench-AAAutomationBench-AA: building workflow automations; partial-credit score.
- ProgramBench v1 Fully Resolved2 sub-boardsShare of public ProgramBench v1 tasks where the implementation passes every hidden behavioral test.
vals.aiTop model: Opus 5.563 models4 mixesQuality 100
- AA-Briefcase v1.1AA-Briefcase: agentic office-work tasks over real files (xlsx/pptx/docx/pdf) judged pairwise; reported as Elo.
- DeepSWE v1.1Pass@1 on 113 original long-horizon software engineering tasks across 91 repos and 5 languages, run with mini-swe-agent in isolated environments (v1.1 fixes dependency drift and flaky tasks).
deepswe.datacurve.aiTop model: GPT-6 Astra28 models8 mixesQuality 76
- ARC-AGI-3ARC-AGI-3 semi-private set: interactive game-like environments where the agent must learn rules on the fly; percent of possible score.
arcprize.orgTop model: GPT-6 Astra24 models7 mixesQuality 78
- BullshitBench v2Share of 100 nonsense prompts (software, finance, legal, medical, physics) where the model clearly calls out the nonsense instead of confidently answering; 3-judge panel, mean aggregation. Reported per reasoning level.
petergpt.github.ioTop model: Opus 4.8108 models5 mixesQuality 79
- Terminal-Bench Science (Vals)Research workflow tasks contributed by practicing scientists across five domains
- MLCR-AAMedical Long Context Reasoning (MLCR-AA): reasoning over long medical records; overall score.
- Arena WebDev3 sub-boardsHuman preference between agentic front-end web development outputs; Bradley-Terry rating.
arena.aiTop model: Opus 5.5113 models13 mixesQuality 62
- SimpleBench200+ multiple-choice questions on spatio-temporal reasoning, social intelligence and adversarial trick questions where unspecialized humans outperform frontier models; accuracy averaged over 5 runs.
simple-bench.comTop model: Opus 5.596 models10 mixesQuality 66
- FrontierCode 2.0Mergeability of coding-agent changes on private maintainer-authored tasks, graded by unit tests, rubrics and verifiers (Cognition).
cognition.comTop model: Opus 5.528 models5 mixesQuality 76
- Terminal-Bench 4.0Resolution rate of agent+model systems on 66 professional computer-work tasks in a terminal (5 trials per task, 8-hour timeout). Rows are harness+model pairs.
tbench.aiTop model: Opus 5.523 models9 mixesQuality 67
- Agent Arena6 sub-boardsArena.ai Agent Arena: causal 'net improvement' a model contributes to real-world agentic sessions (task success, praise vs complaint, steerability, shell commands, tool hallucination), estimated from millions of live Agent Mode sessions. Percent with 95% CI; not an Elo.
- FrontierSWE v2Cognition's ultra-long-horizon software engineering benchmark: 34 tasks graded 0-100 on implementation, performance and research; mean@5 with a 20-hour budget per task in the Proximus harness at max reasoning effort.
frontierswe.comTop model: GPT-6 Astra22 models5 mixesQuality 67
- GDPval-AA v2.1GDPval-AA: economically valuable knowledge-work tasks across occupations, graded pairwise by AA; reported as Elo.
- WeirdML v3Fully agentic: 11 hand-made ML and data-analysis tasks run in coding-agent harnesses, where models must explore unfamiliar data and build pipelines with limited data, unspecified goals or little feedback. Overall effective score (95% interval).
htihle.github.ioTop model: GPT-6 Astra25 models5 mixesQuality 65
- AutomationBench (Zapier)Zapier's agent benchmark (v1.0.6): end-to-end business workflows across 47 real tools in six functions, graded strictly on the final environment state (every assertion must pass). Private held-out set.
zapier.comTop model: Gemini 4 Carbon40 models4 mixesQuality 68
- AA Intelligence Index v5.0Artificial Analysis' composite intelligence score (v5.0), combining 9 evaluations: AA-Briefcase v1.1, AutomationBench-AA, Terminal-Bench 4.0, FrontierCode 2.0, AA-Omniscience v1.1, GDP.pdf, HLE-Diamond, CritPt v2 (Hard Subset), Terminal-Bench-Science 0.1.
- Terminal-Bench 4.0 (Vals)Vals AI's Terminal-Bench 4.0 aggregate scores from its external-runner leaderboard. Vals publishes a separate system set and score series from the tbench.ai harness-plus-model leaderboard, so the two sources remain distinct.
- AA Intelligence Index v5.0.1Artificial Analysis' composite intelligence score (v5.0.1), combining 9 evaluations: AA-Briefcase v1.1, AutomationBench-AA, Terminal-Bench 4.0, FrontierCode 2.0, AA-Omniscience v1.1, GDP.pdf, HLE-Diamond, CritPt v2 (Hard Subset), Terminal-Bench-Science 0.1.
- Blueprint-Bench 2Spatial reasoning: reconstruct 2D floor plans from ~20 apartment photos, scored on room connectivity and scale (random baseline normalized to 0).
andonlabs.comTop model: Gemini 4 Carbon30 models2 mixesQuality 72
- AA-3DAA-3D: agents in an offline sandbox turn a written brief into an interactive three.js scene; Elo from blind pairwise votes by AA's recruited panel.
- Harvey LAB-AA v1.1Harvey LAB-AA v1.1: 120 private agentic legal tasks built by Harvey. Hallucination-Gated All-Pass Rate: a task counts only when a three-judge panel passes every rubric criterion and the deliverables contain no material hallucination. Not comparable with v1.0 scores.
- AA-Omniscience v1.19 sub-boardsAA-Omniscience v1.1 Index (-100..100): knowledge accuracy minus a hallucination penalty. v1.1 drops saturated and public questions, adds new ones and revises ambiguous ones; scores run about 2 points below v1.
- ClutterBench1 sub-boardClutter as a share of drawn area in generated graphics, using the raw prompt. Red area plus half the orange area, averaged over 6 scored posts. Lower is better.
clutterbench.adamholter.chatgpt.siteTop model: Sonnet 5.512 models8 mixesQuality 50
- T3 Code usage shareShare of all turns sent in T3 Code (2026-10-07 to 2026-11-05), from T3's own telemetry via Slopalytics. Revealed preference among T3 Code users, not a quality measure: price, plans and harness defaults drive it too. Model-level, no reasoning effort.
slopalytics.comTop model: Opus 5.589 models1 mixQuality 76
- Code2Video-AACode2Video-AA: AA's run of HeyGen's Code2Video Bench. Models turn 114 design briefs into short product and brand videos in HyperFrames HTML; Elo from blind pairwise votes.
- MineBenchModels turn a text prompt into raw Minecraft-style block coordinates; humans vote pairwise on the rendered builds. Value is the conservative rankScore (rating − 2·RD), not raw Elo.
minebench.aiTop model: GPT-6 Astra Pro77 models5 mixesQuality 56
- AA-LCR v1.1AA-LCR: long-context reasoning over ~100k-token document sets; pass rate.
- VoxelBench (text)Models build 3D voxel scenes from text prompts; humans vote head-to-head on the renders. Glicko-2 rating.
voxelbench.aiTop model: GPT-6 Astra54 models4 mixesQuality 58
- Chess Puzzles (Epoch AI)Epoch AI's chess puzzles: the model finds the best move in positions with a single correct line. Score is the share of puzzles solved. Data: Epoch AI, CC BY.
- KernelBench CUDA (kernelbench.com)Coding agents write four CUDA-only GPU kernels (GLM-5.2 fused MoE, DeepSeek sparse attention, MegaQwen decode, grid MinGRU scan) where Triton and other DSLs fail, one unlimited session per problem on an RTX PRO 6000, each run audited for reward hacking. Mean per-problem score over the full deck (peak fraction of roofline; sparse attention against a reference latency, so it can exceed 100%); failed, flagged or unaudited problems count 0. Independent of Stanford's KernelBench.
kernelbench.comTop model: Sonnet 5.522 models3 mixesQuality 60
- Legal Research BenchEvaluating agents on legal research tasks across diverse areas of US law
- Vals IndexGDP-weighted measure of agentic model performance across finance, coding, and legal tasks.
- Vending-Bench 2Year-long simulated vending-machine business; agents handle suppliers, inventory, pricing, and refunds. Score is the final bank balance.
- AA Intelligence Index v4.3.2Artificial Analysis' composite intelligence score (v4.3.2), combining 10 evaluations: AA-Briefcase v1.1, GDPval-AA v2.1, AutomationBench-AA, Terminal-Bench 4.0, SciCode, Humanity's Last Exam, GDP.pdf, CritPt, AA-Omniscience, and AA-LCR v1.1.
- FrontierMath ErdősEpoch AI's 68 open Erdős problems, formulated in Lean. Models must supply complete formal proofs or disproofs. Published solved fraction; distinct from FrontierMath difficulty tiers. Data: Epoch AI, CC BY.
- ProofBench v1.1Automated theorem proving benchmark
- CritPt v21 sub-boardCritPt v2: the authors' revision of CritPt's 70 research-level physics challenges, solved with a code-execution tool in an offline Python sandbox; accuracy.
- DAYJOB: HealthcareUses Surge AI's rubric-graded healthcare assignment success rate. Score: score. Collected by Jake Boggs' benchmark dashboard.
surgehq.aiTop model: Opus 5.536 models1 mixQuality 68
- GDPxSemiAnalysis's benchmark of agents doing frontier work in 206 long tasks across 11 domains: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement, hardware design, materials science, alignment and software engineering. Each model runs in its own harness where it has one, up to 8 hours a task. The score averages 3 attempts per task, with every domain weighted equally and each graded the way its tasks intend, pass or fail or partial. Most tasks are private. SemiAnalysis says safety classifiers lower some models' scores in cybersecurity, biology and alignment.
gdpx.semianalysis.comTop model: Opus 5.520 models1 mixQuality 68
- Furniture AssemblyEpoch AI's spatial-reasoning benchmark: from photos of IKEA builds (Ställ shoe cabinet, Tonstad bed, Gullaberg dresser) and the official manual, the model decides whether the build is correct and, if not, identifies and describes the mistaken step. Mean score. Data: Epoch AI, CC BY.
- AA-AnalystAgentAA-AnalystAgent: multi-step financial/data analysis agent tasks; pass rate.
- MazeBenchUses gems collected on the populated first-party ASCII-with-tools leaderboard. Score: share of MazeBench's 100 gems collected. Collected by Jake Boggs' benchmark dashboard.
mazebench.comTop model: GPT-6 Astra27 models1 mixQuality 67
- Code MigrationCan language models reimplement real-world programs in another language? Vals AI's overall score for Code Migration.
- SciCodeSciCode: implementing scientific research code from paper descriptions; sub-problem accuracy. Artificial Analysis marks this evaluation under review, so its scores may change. The reason is SciCode-Verified (arXiv 2608.04975), an unreviewed preprint from the authors of the proposed fixes, which reports dataset defects that reject correct solutions. AA removed SciCode from its Intelligence Index in v5.0.
- Vibe Code Bench v1.1Can models build web applications from scratch? Vals AI's overall score for Vibe Code Bench v1.1.
- MMMU-ProMMMU-Pro: multimodal college-level understanding; accuracy.
- Vals RSI Index v1.1: Post-training: Finance AgentCan a model do the research that builds the next model? Autonomous LLM R&D scored against frozen published, human, and frontier-model references. Published Post-training: Finance Agent score in percentage points.
- Epoch Capabilities IndexEpoch AI's Capabilities Index: a single score per model fitted across many benchmarks (a Bayesian item-response model since Nov 4, 2026), so scores are comparable across models tested on different sets. Claude 3.5 Sonnet is set to 130 and GPT-5 to 150. Intervals are 90% credible intervals. Epoch keeps each model's best score per benchmark across settings, so a row is the model's best measured setting, not one reasoning effort. Data: Epoch AI, CC BY.
- threejsevalCoding models build a Three.js scene from the same prompt; humans vote blind on pairs of finished scenes. Value is Bradley-Terry Elo (base 1500), the mean across prompts.
threejseval.comTop model: Opus 5.530 models2 mixesQuality 56
- BioMysteryBenchAnthropic's benchmark of whether AI agents can solve real-world bioinformatics mysteries from raw data
- Humanity's Last Exam (text only)Scale's run of Humanity's Last Exam on its text-only subset (the questions without images): expert-written frontier questions across disciplines, accuracy with a 95% confidence interval. Models evaluated after HLE became public carry a contamination warning.
scale.comTop model: GPT-6 Astra46 models1 mixQuality 63
- Excel Modeling BenchmarkEvaluating agents on Excel-based financial modeling tasks
- IOIVals AI's overall score for version 2 of the International Olympiad in Informatics benchmark.
- ChartographyCan a model read the charts professionals rely on — Kaplan-Meier curves, candlesticks, contour maps, Sankey diagrams, Bode plots, wind roses — from the image alone, to an expert-set precision? Maintained by Surge AI. Score: The board's accuracy: the share of answers inside the expert-set acceptable range of the golden answer, judged per answer by Gemini 3.5 Flash, averaged over the 100 tasks and ten runs per task.
- GDP.xlsxUses Surge AI's first-party spreadsheet-task leaderboard score. Score: score. Collected by Jake Boggs' benchmark dashboard.
- MedScribeCan models support doctors with their administrative work?
- Vals RSI Index v1.1: Pre-training: CompressionCan a model do the research that builds the next model? Autonomous LLM R&D scored against frozen published, human, and frontier-model references. Published Pre-training: Compression score in percentage points.
- Tax Agent BenchEvaluating agents on research-grade US tax questions using the Tax Agent Bench harness
- DuelLab GameBench 2Models write game-playing programs from a rules description; the programs are compiled and play head-to-head tournaments on hidden abstract strategy games. 0-100 score, penalised for programs that fail to run.
benchmarks.duellab.orgTop model: Opus 5.577 models1 mixQuality 60
- Harvey's Legal Agent BenchmarkTests an agent's ability to complete legal work using documents, spreadsheets, presentations, and file-system tools
- AA-RoboticsAA-Robotics: a model in its lab's coding-agent harness directly controls a 6-axis robot arm (5 joints and a gripper) through everyday tabletop tasks such as pressing a button, pouring a drink and writing "hello world". Mean score per task (0 to 100, partial credit for progress).
- CursorBench 4.0 (Cursor)Agent evaluation on ambiguous, multi-file tasks drawn from real Cursor sessions. Maintained by Cursor. Score: CursorBench score.
cursor.comTop model: Opus 5.518 models2 mixesQuality 52
- Arena Text13 sub-boardsCrowdsourced pairwise human preference on open-ended text prompts (style control on); Bradley-Terry rating.
- FrontierCS 2Uses the published pass rate against the human reference on every hidden metric, not the separate FCI index. Score: pass rate beating the human reference on every metric. Collected by Jake Boggs' benchmark dashboard.
frontier-cs.orgTop model: Opus 5.517 models1 mixQuality 59
- FrontierMLUses the published mean normalized task score across wallet-budget settings; the index selects each model's highest-scoring setting. Score: mean normalized task score. Collected by Jake Boggs' benchmark dashboard.
frontierml.orgTop model: Opus 5.518 models1 mixQuality 59
- SimpleQA Verified1,000 short fact-seeking questions with human-verified answer keys (DeepMind's relabeling of OpenAI's SimpleQA); accuracy on the verified set.
kaggle.comTop model: Gemini 3.1 Pro49 models1 mixQuality 59
- Public Benefits Bench v1.1Accuracy answering questions about SNAP benefits and eligibility.
- Vals RSI Index v1.1: Pre-training: LM TrainingCan a model do the research that builds the next model? Autonomous LLM R&D scored against frozen published, human, and frontier-model references. Published Pre-training: LM Training score in percentage points.
- MathArenaUses MathArena's website-wide IRT expected-performance estimate. Score: website-wide IRT expected performance. Collected by Jake Boggs' benchmark dashboard.
matharena.aiTop model: GPT-6.1 Sol88 models1 mixQuality 58
- Senior SWE-BenchUses the first-party tasteful solve rate (pass@1) at each model's published effort. Score: tasteful solve rate (pass@1). Collected by Jake Boggs' benchmark dashboard.
senior-swe-bench.snorkel.aiTop model: Opus 5.523 models1 mixQuality 58
- SRE BenchAgent reverse engineering of real-world binary programs without access to source code.
- Terminal-Bench 2.1 (Vals)State-of-the-art set of difficult terminal-based tasks Scores are the values published by VALS AI. Source benchmark version 2.1.
- AA Progress Index v1Artificial Analysis Progress Index: general capability on one scale that holds across years, fitted with item response theory over every evaluation AA has run since late 2022, so models tested on different benchmarks are still comparable. Built for trends over time; the Intelligence Index measures today's frontier.
- Arena Vision8 sub-boardsHuman preference on prompts with image inputs (multimodal understanding); Bradley-Terry rating.
- Remote Labor IndexUses the dashboard.safe.ai RLI automation-rate API. Score: automation rate. Collected by Jake Boggs' benchmark dashboard.
remotelabor.aiTop model: Opus 5.516 models1 mixQuality 56
- Mystery Game Puzzles (Epoch AI)Epoch AI's deduction puzzles: work out the hidden rules of a mystery game from its observations. Score is the share solved. Data: Epoch AI, CC BY.
- AA Intelligence Index Cost per TaskMeasured USD per task, weighted across the Intelligence Index evaluations.
- Cost to Run AA Intelligence IndexTotal USD AA spent running the full Intelligence Index evaluation suite for the model (measured input+output token cost, incl. reasoning), not list token price.
- Output Speed (tokens/s)Output tokens per second over a request's full duration, so time to first token and reasoning tokens count. AA averages its workloads (1k, 10k and 100k prompts, and multi-turn agent sessions) over the past 5 days, and a request that fails on the endpoint's side counts as 0. Read from the lab's own API where AA has one, otherwise the median across providers.
- τ³-Bankingτ³-Bench banking domain: multi-turn tool-using agent tasks with a simulated user; pass rate.
- Latency (Time to First Token)Seconds from request to first token as measured by AA, averaged across its workloads over the past 5 days. It may be a reasoning token, so this is not time to first answer.
- CritPt v2Uses Artificial Analysis accuracy on the revised research-level physics benchmark, evaluated with code execution. Score: accuracy. Collected by Jake Boggs' benchmark dashboard.
- DAYJOB: FinanceUses Surge AI's rubric-graded finance assignment success rate. Score: score. Collected by Jake Boggs' benchmark dashboard.
- Mercor APEXScore: best pass@1 (loop_truncated_tools_agent). Collected by Jake Boggs' benchmark dashboard.
mercor.comTop model: Gemini 4 Carbon61 models0 mixesQuality 67
- Ramp Accounting BenchLong-horizon accounting work across realistic financial systems: 137 tasks, 3 attempts each in one shared agent harness (Mercor Archipelago), graded by GPT-5.6 Luna against per-task criteria. All models ran at high reasoning effort. Value is the mean share of criteria met across the three attempts. Cost is per attempt.
labs.ramp.comTop model: Gemini 4 Argon18 models1 mixQuality 55
- GDPxUses partial credit, averaged equally across domains after averaging partners within each domain; excludes biology, cybersecurity, and alignment. Score: domain-balanced partial-credit score without safety classifiers. Collected by Jake Boggs' benchmark dashboard.
- HANDBOOK.md AgentsUses Surge AI's first-party MCP-based enterprise policy-following agent score. Score: score. Collected by Jake Boggs' benchmark dashboard.
- ALE-BenchUses each model's best all-scenario performance mean, normalized to the current leaderboard maximum. Score: best all-scenario performance mean. Collected by Jake Boggs' benchmark dashboard.
sakanaai.github.ioTop model: GPT-6 Astra122 models0 mixesQuality 65
- FrontierPhysicsEnd-to-end physics research by agents: 56 tasks across nine domains (theory, computation and experiment), written by 42 physicists from 26 institutions, 3 trials each. A pass needs every deterministic verifier test and every blocker rubric. Value is the overall pass rate averaged over the trials. Cost is per rollout, one attempt at one task.
benchflow.aiTop model: Opus 5.522 models0 mixesQuality 65
- Provider latencyMedian provider latency across OpenRouter endpoints, in seconds (3d).
openrouter.aiTop model: Llama 3.2 3b183 models0 mixesQuality 65
- Provider throughputMedian provider output throughput in tokens per second across OpenRouter endpoints (3d).
- Finance Agent v2Evaluating agents on core financial analyst tasks using the FAB v2 harness
- MedCodeCan models support the medical billing process?
- SkillsBenchHow important are skills for agents? Vals AI's overall SkillsBench score.
- CWE-Bench-AA Refusal RateShare of CWE-Bench-AA tasks the model refused on safety grounds. Lower means fewer refusals on defensive security work.
- CheatBenchUses the published cheating-attempt rate; lower is better. Score: cheating rate. Collected by Jake Boggs' benchmark dashboard.
cheatbench.aiTop model: Gemini 4 Argon10 models2 mixesQuality 45
- FrontierCode 2.0Uses the 2.0 Main suite's weighted rubric score at each model's best reasoning effort, including published historical model results. Score: Main rubric score at best reasoning effort. Collected by Jake Boggs' benchmark dashboard.
- ZeroBenchUses main-question pass@5 because current ZeroBench evaluations no longer report pass@1. Score: main questions pass@5. Collected by Jake Boggs' benchmark dashboard.
zerobench.github.ioTop model: GPT-6 Astra61 models0 mixesQuality 64
- SWE-bench VerifiedA 500-instance human-filtered subset of SWE-bench created with OpenAI, served as the default Verified leaderboard view. Maintained by SWE-bench Team. Score: % Resolved: percentage of task instances solved.
swebench.comTop model: Opus 4.859 models1 mixQuality 52
- ChartographySurge AI's 100 professional chart-reading prompts across 12 domains: specialized graphics (Kaplan-Meier curves, candlesticks, contour maps, Sankey diagrams, Bode plots) that need perception, domain knowledge and multi-step reasoning. Accuracy.
- SimpleQA Verified (Epoch AI run)Google DeepMind's cleaned-up SimpleQA short-answer factuality set, run by Epoch AI. Share answered correctly. Data: Epoch AI, CC BY.
- MysteryMechanismCan agents rediscover sealed mathematical mechanisms through bounded experiments?
- Vals RSI Index v1.1: Harness Engineering: JudgeCan a model do the research that builds the next model? Autonomous LLM R&D scored against frozen published, human, and frontier-model references. Published Harness Engineering: Judge score in percentage points.
- ZeroBench (main)ZeroBench's 100 main questions: visual problems built to be impossible for contemporary multimodal models. Official leaderboard, pass@5 (solved in at least one of five tries).
- CWE-BenchCWE-Bench: security-weakness detection; score.
- GDP.pdfCan a model answer real professional questions about the PDFs work runs on — manuals, wiring diagrams, filings, forms — so that every rubric criterion an expert wrote is met? Maintained by Surge AI. Score: The board's all-pass rate: the share of responses that satisfy every rubric criterion, judged by Gemini 3.5 Flash, over the 100 held-out tasks and five runs per task.
- SWE Atlas-QnAAgents explore real repositories with shell tools and answer deep codebase questions; rubric-graded Task Resolve Rate (all rubric items met). Best harness per model/effort.
labs.scale.comTop model: Opus 522 models1 mixQuality 51
- WeirdML v217 ML tasks where models write Python pipelines from scratch (shape inference, chess winners, Kolmogorov shuffles); mean accuracy across tasks.
- DAYJOB: TaxUses the published attempt pass rate at the 95% rubric threshold across 50 professional tax assignments. Score: score. Collected by Jake Boggs' benchmark dashboard.
- FrontierChallengeUses the first-party pass rate across 97 scientific workflow tasks. Score: pass rate. Collected by Jake Boggs' benchmark dashboard.
apodexai.github.ioTop model: GPT-5.6 Sol11 models1 mixQuality 50
- Vals RSI Index v1.1Can a model do the research that builds the next model? Autonomous LLM R&D scored against frozen published, human, and frontier-model references. Overall score in percentage points.
- benchmarks.bio Expected PerformanceMean IRT-predicted pass rate across automatically discovered evaluations, excluding refusal. Uses the best mapped harness per model and evaluation and requires observed coverage in every domain; the aggregate counts as one benchmark. Score: IRT expected performance across evaluations. Collected by Jake Boggs' benchmark dashboard.
benchmarks.bioTop model: Opus 5.526 models0 mixesQuality 61
- ClockBenchUses the first-party analog-clock accuracy leaderboard. Score: overall accuracy. Collected by Jake Boggs' benchmark dashboard.
clockbench.aiTop model: Gemini 4 Argon37 models0 mixesQuality 61
- PPBenchScore: max success rate. Collected by Jake Boggs' benchmark dashboard.
ppbench.comTop model: Fable 553 models0 mixesQuality 61
- UGIA private-question suite combining sensitive-topic knowledge with willingness to follow controversial instructions. Maintained by DontPlanToEnd. Score: UGI 🏆.
huggingface.coTop model: Grok 4.20167 models0 mixesQuality 61
- UGI NatIntA private-question evaluation of general knowledge and reasoning across textbook, popular-culture and world-model topics. Maintained by DontPlanToEnd. Score: NatInt 💡.
- UGI WillingnessA private-question evaluation of how far a model follows challenging instructions before refusing or deviating. Maintained by DontPlanToEnd. Score: W/10 👍.
- UGI WritingA maintainer-owned writing evaluation considering intelligence, style, repetition and length adherence, informed by human preferences. Maintained by DontPlanToEnd. Score: Writing ✍️.
- ClockBenchReading analog clocks. Trivial for people (90.7% average, 95% best) and hard for frontier models. Accuracy on the official leaderboard.
- CyberBench v1.1Agent performance reproducing OSS-Fuzz vulnerabilities with proof-of-concept inputs and patching the underlying bugs.
- Vibe Code Bench 1-100Can models extend a working web application across a long sequence of dependent requests? Vals AI's overall score for Vibe Code Bench 1-100.
- Agents' Last ExamSnorkel AI's benchmark of long-horizon, economically valuable professional workflows across 55 sub-industries, run in each model's agent harness with verifiable outcomes. Score is the share of rubric points earned (the pass rate, all checks met, is lower). Best harness per model and effort.
snorkel.aiTop model: Gemini 4 Carbon13 models1 mixQuality 48
- HiL-BenchUses Scale Labs combined Pass@3 with ask_human tool. Score: combined Pass@3 with ask_human tool. Collected by Jake Boggs' benchmark dashboard.
- sudo L7Surge AI's 60-task staff-level software engineering benchmark. Expert-authored rubrics assess correctness, architecture, production risk, verification and communication, using each model's published agent harness and reasoning setting.
- Real-SWE (Specific Labs)Resolution rate of coding agents on private, licensed enterprise codebases, published as pass@1 over eight runs per task on a ten-task sample. Maintained by Specific Labs. Score: Resolution rate = pass@1 correctness averaged over eight independent runs per task, with a published 95% confidence interval.
withspecific.comTop model: Fable 5.18 models1 mixQuality 48
- AI Chess Leaderboard (Dubesor)LLMs play full chess games against each other from the move list, in reasoning and continuation modes combined; Elo within this LLM-only pool, models with 10+ games.
dubesor.deTop model: Gemini 3.1 Pro203 models0 mixesQuality 60
- DeepFinanceJulius Research's ultra long-horizon benchmark of real Private Equity and Investment Banking workflows: 21 tasks that take a professional about 27.1 hours each, scored against 84 rubric criteria per task, 10 attempts per task. Value is the mean share of rubric criteria met. Cost is mean API cost per task.
juliusresearch.aiTop model: Opus 5.520 models0 mixesQuality 60
- Roboflow Visual ReasoningRoboflow Vision Evals: image questions requiring arithmetic, comparisons, spatial relations and multi-step inference (the answer is never written in the image). LLM-judge accuracy, mean of runs.
playground.roboflow.comTop model: Gemini 4 Argon59 models0 mixesQuality 60
- APEX-Agents-AAAPEX-Agents-AA: long-horizon professional-services agent tasks; pass rate.
- DeepsecBench-AADeepsecBench-AA (based on Vercel's DeepsecBench), part of the AA Cyber Index: finding and fixing vulnerabilities in real codebases; score.
- Endpoint Error Rate (lab's own API)Share of requests to the lab's own API that failed with an error, a rate limit or a timeout over the past 5 days, across AA's single-request and multi-turn workloads. Failures on AA's side are left out. AA reports this per endpoint; models it measures only across other providers are not listed.
- Terminal-Bench Hard-AATerminal-Bench Hard (AA subset of Terminal-Bench 2.0): agentic terminal tasks; pass rate.
- EnterpriseBench CoreCraftUses Surge AI's first-party enterprise-task leaderboard score. Score: score. Collected by Jake Boggs' benchmark dashboard.
- EBR-bench (Earthborne Rangers, Epoch AI)Epoch AI's agentic game benchmark built on the Earthborne Rangers card game: long-horizon planning across a campaign. Score is the share of the scenario completed. Data: Epoch AI, CC BY.
- OTIS Mock AIME 2024-2025The OTIS Mock AIME problems (2024 and 2025), run by Epoch AI. Accuracy over repeated samples. Data: Epoch AI, CC BY.
- RWE-BenchUses average pass@1 across real-world engineering tasks, not pass@5; publication/export dates do not determine model release dates. Score: average pass@1 across engineering tasks. Collected by Jake Boggs' benchmark dashboard.
xplore.aiTop model: GPT-6 Astra11 models0 mixesQuality 58
- EQ-Bench Longform Creative WritingLLM-judged benchmark in which models plan and write a short story/novella over 8x 1000-word turns, scored 0-100 across 14 rubric dimensions. Maintained by EQ-Bench. Score: Score (0-100): average of all chapter scores plus the final scored piece across 14 rubric dimensions, with the forced poetry/metaphor criterion weighted 5x and a long-context degradation penalty applied.
eqbench.comTop model: Opus 5106 models0 mixesQuality 58
- MCP Atlas (Scale AI)How reliably a model completes realistic multi-step workflows over real Model Context Protocol servers: discovering the right tool in a noisy tool menu, calling it with correct parameters, recovering from errors and synthesising the results into an accurate final answer. Maintained by Scale AI. Score: Pass rate in percent over all 1,000 tasks; the published ± is the confidence half-width.
- Lichess Puzzle Elo (Dubesor)LLMs solve rated Lichess puzzles (100+ of a 500-puzzle set); the value is a maximum-likelihood performance rating against the puzzles' Lichess ratings, first-try solves only.
- GPQA Diamond (Epoch AI run)Graduate-level science multiple choice (the diamond subset), run by Epoch AI. Accuracy. Data: Epoch AI, CC BY.
- Humanity's Sixth SenseScale AI and Elorian's intuitive visual reasoning benchmark: 522 image and silent-video tasks covering temporal, spatial, social and abstract inference. Pass@1 averaged over three attempts per task; all rubric criteria must pass. Published intervals are 95% task-cluster bootstrap intervals.
- LLM ChessLLMs play chess through a text tool interface against a random-move player and the Komodo Dragon engine at set skill levels; the value is the Elo estimated from those games.
maxim-saplin.github.ioTop model: GPT-6 Astra103 models0 mixesQuality 58
- Kaggle Game ArenaUses Kaggle's pooled Bradley-Terry leaderboard rating across strategic games. Score: Bradley-Terry rating scaled expected win rate vs Gemini 4 Argon. Collected by Jake Boggs' benchmark dashboard.
- EQ-Bench Creative Writing v3A LLM-judged creative writing benchmark with vocabulary and GPT-slop controls. Maintained by EQ-bench. Score: Elo Score column in the leaderboard.
- Butter-BenchLLMs orchestrate a household robot through six 'pass the butter' subtasks; average task completion rate.
- SpireClimb (Epoch AI)Epoch AI's Slay the Spire agent benchmark: climb the spire by playing cards and choosing routes. Score is progress up the spire. Data: Epoch AI, CC BY.
- Biopharma Bench v0.1Uses mean task criteria score across 71 regulated biopharma workflows. Score: mean task criteria score. Collected by Jake Boggs' benchmark dashboard.
eval.raycaster.aiTop model: GPT-6 Astra12 models1 mixQuality 44
- Riemann-BenchUses Surge AI's first-party extreme-math leaderboard score. Score: score. Collected by Jake Boggs' benchmark dashboard.
- LLM Creative Story-Writing BenchmarkPairwise comparison of short stories written to the same constrained creative briefs, with LLM evaluator choices combined into a relative comparison score. Maintained by Lech Mazur. Score: Relative comparison score from pairwise LLM evaluator judgments (Thurstone-style rating; zero is near the middle of the comparison set).
raw.githubusercontent.comTop model: Fable 5.152 models0 mixesQuality 56
- VulcanBench Frontier v423 behavioural-reconstruction tasks: the model must repair a replacement implementation of a legacy program until hidden tests confirm it reproduces the program’s real drift from its written spec; the combined score weights functional correctness 50%, lint/complexity 8.5%, security 8.5% and judged Code quality 33%. Maintained by Morgan Linton. Score: Combined score across the 23 tasks, one column per model × reasoning-effort level: 100 × (0.50 × partial-credit functional hidden-test score + 0.085 × lint/complexity
vulcanbench.comTop model: Sonnet 5.513 models1 mixQuality 44
- LLM2014 LogicA private, monthly-rotating bank of about 28 logic and reasoning questions (interleaved text, rule induction, log analysis, puzzles), three runs each; the derivation must be correct, guesses earn nothing. Median score out of 100.
llm2014.github.ioTop model: GPT-6 Astra26 models0 mixesQuality 56
- CUA-benchComputer-use agents playing commercial video games through keyboard and mouse input, with half the titles held out.
- EnterpriseOps-Gym-AAEnterpriseOps-Gym-AA: multi-turn enterprise operations tasks over simulated business systems; success rate.
- PLE-BenchUses the overall hidden-test rubric score, selecting each model's best harness on the native 0-1 scale. Score: mean rubric score on the hidden test set. Collected by Jake Boggs' benchmark dashboard.
ple-bench.comTop model: Opus 5.518 models0 mixesQuality 55
- Hearing BenchModels see only a spectrogram image of a sound clip, never the audio, and must name the sound: everyday sounds, instrument notes, vocal sounds, spoken words and sentences. Strict grading; value is % of 100 clips correct.
hearing-bench.adamholter.chatgpt.siteTop model: Gemini 4 Argon10 models0 mixesQuality 55
- Uptime (7d)Mean provider uptime over the trailing week on OpenRouter — share of time endpoints can serve the model.
- PostTrainBench v1.1Agents act as post-training engineers: given a base model and a compute budget, run a full fine-tuning pipeline, scored on downstream benchmark gains averaged over seeds.
posttrainbench.comTop model: Fable 511 models0 mixesQuality 55
- GPQA DiamondGPQA Diamond: graduate-level science multiple choice; accuracy.
- ITBench-AAITBench-AA (SRE scenarios): diagnosing and remediating incidents in live Kubernetes environments; pass rate.
- Olam Poker EloUses Olam Labs' Bradley-Terry Elo ratings from multi-agent social poker and excludes human players. Score: Poker Elo scaled expected win rate vs Claude Fable 5. Collected by Jake Boggs' benchmark dashboard.
olamlabs.aiTop model: Fable 534 models0 mixesQuality 54
- ApprenticeBench API (NeoCognition)An agent completes the same 100 sequential accounts-payable tasks in the Odoo ERP with diminishing mentoring, using dedicated application API calls instead of the screen interface. Maintained by NeoCognition. Score: Cumulative success rate.
apprenticebench.comTop model: Fable 5.127 models0 mixesQuality 54
- Toolathlon, original release (HKUST NLP, archived board)An agent completes 108 real-world tool-use tasks across MCP servers and local toolkits, graded by executing the resulting state against each task's ground truth — the original task definitions, before the June 2026 Verified repair. Maintained by HKUST NLP. Score: Pass@1 in percent as published on the archived pre-Verified board.
toolathlon.xyzTop model: Gemini 3.5 Flash33 models0 mixesQuality 54
- Tool-call error rateVolume-weighted mean tool-call error rate across OpenRouter providers (3d).
- Terminal-Bench 2.0Terminal-Bench 2.0: task resolution rate for agent+model pairs on 89 terminal tasks. Kept for time travel.
hub.harborframework.comTop model: GPT-5.540 models0 mixesQuality 54
- Arena SearchHuman preference among LLMs with integrated web search; Bradley-Terry rating.
- HalluHardExcludes web-search variants. Score: turn-1 hallucination rate without web search. Collected by Jake Boggs' benchmark dashboard.
halluhard.comTop model: GPT-6 Astra29 models0 mixesQuality 53
- JobBenchUses the first-party weighted-score leaderboard. Score: weighted score. Collected by Jake Boggs' benchmark dashboard.
job-bench.github.ioTop model: Opus 5.525 models0 mixesQuality 53
- TasteValUses experimental-compute efficiency relative to human experts, divided by the current highest model multiplier. Retains the native multiplier and 95% source interval. Score: compute multiplier / current leaderboard maximum. Collected by Jake Boggs' benchmark dashboard.
pzeroresearch.comTop model: Opus 5.520 models0 mixesQuality 53
- GSO software optimization, Opt@1 (UC Berkeley)An agent gets a real codebase and a performance test and must make the code as fast as an expert developer's optimization; 102 tasks across 10 codebases and 5 languages. Maintained by GSO team. Score: Opt@1: share of tasks where a single attempt reaches at least 95 % of the human speedup and passes the correctness tests.
gso-bench.github.ioTop model: Opus 4.826 models0 mixesQuality 53
- Elimination Game BenchmarkMulti-player tournament where 8 LLM players converse, form alliances, and vote to eliminate each other, with final rankings scored via TrueSkill. Maintained by Lech Mazur. Score: TrueSkill mean (μ), with σ uncertainty reported separately.
- ResearchClawBenchAn AI agent gets the raw data and related literature behind a published scientific paper — the paper itself hidden — and must redo the research end to end, from analysis to a written report, across 40 tasks in 10 scientific domains. Maintained by ResearchClawBench team. Score: Mean rubric score over the tasks a model has a scored run for, 0–100 (an LLM acting as a strict peer reviewer grades each report against an expert-curated weighted rubric; 50 = the report matches the original paper).
internscience.github.ioTop model: Opus 4.821 models0 mixesQuality 53
- LegalBenchEvaluating language models on a wide range of open source legal reasoning tasks.
- LiveBenchA periodically refreshed suite of objectively graded tasks with separate category scores and an overall category mean. Maintained by LiveBench team. Score: Overall mean of category averages, with source-published exceptions retained.
livebench.aiTop model: Fable 5.156 models0 mixesQuality 52
- Chess LLM BenchmarkLLMs play rated chess games in a pool anchored by chess engines and Maia bots; the value is the Glicko rating, with a FIDE-scale estimate in details, for models with 10+ games.
chessbenchllm.onrender.comTop model: Gemini 3.1 Pro53 models0 mixesQuality 52
- Drone-BenchModels write drone-surveillance code across five tasks (reconstruction, localization, navigation, detection, following); progress toward the human-authored baseline.
- AA Cyber IndexArtificial Analysis Cyber Index: cyber defense in real repositories (find, reproduce and patch vulnerabilities), the equal-weight mean of CWE-Bench-AA, DeepsecBench-AA and CyberGym-E2E-AA. Safety refusals score zero. Includes trusted-access models that are not publicly available.
- CyberGym-E2E-AACyberGym-E2E-AA (based on Berkeley RDI's CyberGym-E2E), part of the AA Cyber Index: end-to-end vulnerability reproduction and patching; score. The component with the most safety refusals.
- AA Openness IndexArtificial Analysis Openness Index (0 to 100): how open a model is, from weight availability and license to transparency about training data and methodology. Not a capability measure.
- AGI-Pilled BenchMedian forecast in years after 2026 for 8 AI milestones. Lower means more AGI-pilled. 5 runs per milestone; the IMO control is excluded.
agi-pilled-bench.adamholter.chatgpt.siteTop model: Haiku 5.514 models0 mixesQuality 51
- Aider PolyglotEvaluates LLMs on 225 Exercism coding exercises across six languages, testing code editing without human intervention. Maintained by Paul Gauthier. Score: Percent of the 225 Exercism exercises completed with a correct edit.
aider.chatTop model: GPT-537 models0 mixesQuality 51
- ApprenticeBench CUA (NeoCognition)A computer-use agent operates the Odoo ERP through its screens across 100 sequential accounts-payable tasks with diminishing mentoring. Maintained by NeoCognition. Score: Cumulative success rate.
- ChessBenchLLMs play rated chess games against each other, one rating per model and reasoning effort; the value is a Glicko-2 rating within this benchmark's pool, qualified setups only.
chessbench.orgTop model: Gemini 4 Argon18 models0 mixesQuality 51
- IFBenchIFBench: precise instruction-following with verifiable constraints; accuracy.
- ComplexConstraintsUses Surge AI's first-party leaderboard score. Score: score. Collected by Jake Boggs' benchmark dashboard.
- HELM LiteUses HELM Lite's core-scenarios Accuracy table mean win rate. Score: core-scenarios mean win rate. Collected by Jake Boggs' benchmark dashboard.
crfm.stanford.eduTop model: GPT-4o40 models0 mixesQuality 50
- SlopCodeBenchUses the published best-version leaderboard and Iso Solve score. Score: isolated checkpoints solved. Collected by Jake Boggs' benchmark dashboard.
scbench.aiTop model: GPT-5.513 models0 mixesQuality 50
- Apex 2025 (MathArena)Twelve final-answer problems from 2025 competitions that MathArena selected because strong models of the time failed them; the selection is biased against the four models used to choose it (Grok 4, GPT-5, Gemini 2.5 Pro, GLM 4.5). Maintained by MathArena. Score: Accuracy (average performance on the competition) with a 95% confidence interval (normal approximation).
- MirrorCode (Epoch AI, with METR)Reimplement a large existing program from its behavior alone. Built by Epoch AI with METR; score is the share of tests passed. Data: Epoch AI, CC BY.
- MathArena Apex1 sub-boardMathArena Apex: 12 final-answer competition problems that earlier frontier models (Grok 4, GPT-5, Gemini 2.5 Pro, GLM 4.5) failed, so it is biased against them. Accuracy over repeated attempts.
- Time Horizon Index: KSPProgress through 30 Kerbal Space Program missions during a five-day autonomous run, with partial credit within a mission.
- AlpacaEval 2.0 VerifiedUses Tatsu Lab's verified/minimal AlpacaEval 2.0 length-controlled win-rate leaderboard. Score: length-controlled win rate. Collected by Jake Boggs' benchmark dashboard.
tatsu-lab.github.ioTop model: GPT-4o20 models0 mixesQuality 49
- EQ-Bench 3Uses the published Elo leaderboard. Score: Elo scaled expected win rate vs claude-fable-5. Collected by Jake Boggs' benchmark dashboard.
- GameDevBenchUses first-party pass@1 across all 333 game-development tasks. Score: pass@1 across all 333 tasks. Collected by Jake Boggs' benchmark dashboard.
waynechi.comTop model: Opus 5.518 models0 mixesQuality 49
- NanoGPT SpeedrunUses Prime Intellect's source-reported share of the human record gap closed for each model's best validated optimizer-speedrun result. Score: share of the human record gap closed. Collected by Jake Boggs' benchmark dashboard.
primeintellect.aiTop model: Fable 518 models0 mixesQuality 49
- Opus Magnum BenchUses each model variant's best pooled human-normalized reward across the full 36-puzzle campaign. Score: mean best human-normalized puzzle reward. Collected by Jake Boggs' benchmark dashboard.
opusmagnumbench.comTop model: GPT-5.6 Sol20 models0 mixesQuality 49
- BU Bench V1 (Browser Use)A browser agent completes 100 hand-selected web tasks: custom page-interaction challenges plus tasks drawn from WebBench, Mind2Web 2, GAIA and BrowseComp. Maintained by Browser Use. Score: Task success rate.
github.comTop model: Opus 4.87 models1 mixQuality 37
- EQ-Bench 4Measures active emotional and social intelligence of LLMs in multi-turn roleplay chats with simulated personas, ranked by an LLM-judge panel Elo. Maintained by EQ-Bench. Score: Ranked Elo score from blind, bidirectional pairwise judging of multi-turn persona-chat transcripts by a three-judge LLM panel (soft Bradley–Terry fit, normalised against anchor models).
- Long-Horizon Terminal-BenchAn agent works in a terminal for up to 90 minutes per task on 46 long jobs — building systems, migrating frameworks, playing games move by move — and earns partial credit for real progress. Maintained by Long-Horizon Terminal-Bench authors. Score: Mean reward over the 46 tasks (hidden verifiers pay continuous partial credit from 0 to 1; errors count 0).
zli12321.github.ioTop model: Grok 4.521 models0 mixesQuality 49
- MMMUA multimodal benchmark of 11.5K college-level questions across six disciplines and 30 subjects, with 30 heterogeneous image types. Maintained by MMMU Team. Score: Zero-shot accuracy (percent) on multimodal college-level questions, reported overall and per discipline for the Val and Test splits.
mmmu-benchmark.github.ioTop model: GPT-5.146 models0 mixesQuality 49
- The Slop IndexLeaderboard scoring how much each model's writing reads like chatbot slop versus pre-AI human writing, blending live arena votes with four mechanical axes. Maintained by The Slop Index. Score: Slop Score 0-100 = 40% live arena Elo votes + 15% each for concise, templating, rhythm, and tells measured against pre-AI human writing; higher = sloppier.
theslopindex.comTop model: Nemotron 3.5 Lightning53 models0 mixesQuality 49
- MATH Level 5 (Epoch AI)The hardest tier (level 5) of the MATH competition set, run by Epoch AI. Accuracy. Data: Epoch AI, CC BY.
- KernelBench Hard (kernelbench.com)Coding agents write six GPU kernels (FP8 and W4A16 GEMMs, KDA, paged attention, top-k, MoE) in CUDA or Triton, one unlimited session per problem on an RTX PRO 6000, each run audited for reward hacking. Mean peak fraction of roofline over the full deck; failed, flagged or unaudited problems count 0. Independent of Stanford's KernelBench.
- OSWorld 2.0Computer use: agents complete long, realistic desktop tasks across apps in a VM. Binary accuracy (task fully completed) on the newest task release, full set, default step budget.
osworld-v2.xlang.aiTop model: Opus 52 models1 mixQuality 37
- Vending-Bench ArenaMulti-agent Vending-Bench 2: models run competing vending machines at one location, may trade or collude, and are scored individually on final balance.
- CyberGym-E2E-AA Refusal RateShare of CyberGym-E2E-AA tasks the model refused on safety grounds. Lower means fewer refusals on defensive security work.
- AA-PersonalityBench ImaginationAA-PersonalityBench Imagination index: creative, open-ended answers. Percentile rank (1 to 100) across the models AA evaluated, from pairwise LLM-judged trait Elos. Describes style, not correctness.
- AA-PersonalityBench SlopAA-PersonalityBench Slop index: how much the writing reads as generic AI filler; lower is better. Percentile rank (1 to 100) across the models AA evaluated, from pairwise LLM-judged trait Elos. Describes style, not correctness.
- GBA EvalUses the first-party leaderboard JSON overall emulator score. Score: overall emulator score. Collected by Jake Boggs' benchmark dashboard.
gbaeval.comTop model: Opus 523 models0 mixesQuality 48
- PostTrainBench v1Uses the archived v1 weighted aggregate; subtasks do not count as separate benchmarks. Score: weighted aggregate score. Collected by Jake Boggs' benchmark dashboard.
- SciUniverseUses overall Pass@1 averaged equally across 17 task families. Score: Pass@1 across 17 equally weighted task families. Collected by Jake Boggs' benchmark dashboard.
c5r.netTop model: Fable 5.16 models0 mixesQuality 48
- CharXiv ReasoningCan a multimodal model reason over the real, messy charts in scientific papers — combining values, trends and labels across a figure — rather than read off templated plots? Maintained by Princeton Language and Intelligence. Score: Reasoning accuracy in percent on the validation split: the share of its 1,000 reasoning questions (one per chart) answered correctly, each final answer extracted and scored 0/1 against the ground truth by GPT-4o.
charxiv.github.ioTop model: o324 models0 mixesQuality 48
- Judgemark v4Meta-evaluation that rates LLM judges on how discriminatively they score fixed creative-writing samples. Maintained by EQ-Bench. Score: Composite judge-separation score on the reported 100-point scale (unclamped bounds not verified).
- LongBench v2503 challenging multiple-choice questions with contexts from 8k to 2M words assessing deep understanding and reasoning on realistic long-context multitasks. Maintained by THUDM. Score: Accuracy on 503 multiple-choice questions across six task categories.
longbench2.github.ioTop model: Qwen 3.8 Max28 models0 mixesQuality 48
- SWE Atlas Refactoring (Scale AI)Whether a coding agent restructures production code while preserving its behaviour, graded by tests and rubrics. Maintained by Scale AI. Score: Leaderboard score (percent); ± is the published confidence half-width.
- SWE-Bench Pro PublicA software-engineering evaluation of long-horizon tasks in public repositories, scored by patches passing both new and regression tests. Maintained by Scale AI. Score: Resolve rate on the public set.
- VitaBenchCan an agent finish a real-life errand that spans food delivery, restaurant bookings and travel — 66 tools, a customer whose wishes shift mid-conversation, and no domain policy to lean on? Maintained by Meituan LongCat team. Score: Cross-Scenarios Avg @4 on the 100 cross-scenario tasks (the paper's main result), graded by the benchmark's rubric-based sliding-window LLM evaluator; the page does not define @4 in words — by the column name it is the average over four trials.
vitabench.github.ioTop model: Gemini 3 Flash12 models0 mixesQuality 48
- SAGEStudent Assessment with Generative Evaluation: accuracy grading student math work against established rubrics.
- τ²-Bench Telecomτ²-Bench telecom domain: dual-control tool-using agent tasks; pass rate.
- HumanEvalUses LLM Stats / ZeroEval HumanEval pass@1 leaderboard. Score: HumanEval pass@1. Collected by Jake Boggs' benchmark dashboard.
llm-stats.comTop model: Kimi K242 models0 mixesQuality 47
- SWE-bench Verified (Epoch AI)The human-verified SWE-bench subset: resolve real GitHub issues in a repository. Run by Epoch AI with its own scaffold; resolve rate. Data: Epoch AI, CC BY.
- Hermes BenchUses mean reward across 150 Hermes Bench tasks in the Hermes Agent harness. Score: mean task reward. Collected by Jake Boggs' benchmark dashboard.
portal.nousresearch.comTop model: Opus 5.516 models0 mixesQuality 46
- Production SimulationUses mean normalized score across twenty production-planning scenarios. Score: mean normalized scenario score. Collected by Jake Boggs' benchmark dashboard.
haladir.comTop model: Fable 56 models0 mixesQuality 46
- τ^τ-bench (Hyper-τ), release v1 (Sierra)A coding agent builds a working customer-service agent from realistic evidence (policies, transcripts, recordings, a client API), and is scored by how well that agent then serves simulated customers on 53 held-out tasks. Maintained by Sierra. Score: Overall: mean reward across all 53 release tasks of the agent the Developer built (τ³-bench inner loop).
- Japanese-RP-Bench (Aratako)Benchmark measuring LLM Japanese roleplay ability via 10-turn two-model dialogues scored by a four-judge ensemble across eight criteria. Maintained by Aratako. Score: Overall Average = mean evaluation value across the 8 criteria, averaged over 4 judge models (gpt-4o-2024-08-06, o1-mini-2024-09-12, Claude 3.5 Sonnet, Gemini 1.5 Pro).
- MLS-Bench-LiteAI agents get a fixed ML research scaffold with strong baselines and must invent one algorithmic change — a loss, optimizer, sampler or training procedure — whose gain holds across settings, seeds, datasets and scales; Lite is the 30-task subset over all 12 research domains. Maintained by MLS-Bench authors. Score: The board's MLS-Bench-Lite score: the paper's normalized task metric, averaged (arithmetic mean) over the 30 Lite tasks, each agent run under Harbor with a 5-hour exploration budget.
mls-bench.comTop model: Fable 5.114 models0 mixesQuality 46
- RSI-Exam 0.1 (aiming-lab)An agent spends a long, budgeted run improving an inherited, working-but-weak executable research method or harness on a visible split, and the single artifact it submits is re-run once on a sealed hidden split across 88 executable research tasks in six domains. Maintained by RSI-Exam Team. Score: Mean frozen normalised hidden-set score (0–1) over the 88 active tasks (Full board).
rsi-exam.aiTop model: GPT-6 Astra Codex13 models0 mixesQuality 46
- Spiral-BenchA LLM-judged benchmark measuring sycophancy and delusion reinforcement in simulated multi-turn chats. Maintained by sam-paech. Score: Safety Score: weighted average of protective vs risky behaviours, scaled 0-100.
- METR Time Horizon (50%)METR's time horizon: the length of software and reasoning task, measured in minutes a skilled human needs, that the model completes successfully half the time. Longer is better; it has roughly doubled every few months.
metr.orgTop model: Mythos Preview21 models0 mixesQuality 46
- AutoResearchExamUses mean hidden-test AUARC across 24-hour research tasks. Score: hidden-test AUARC. Collected by Jake Boggs' benchmark dashboard.
benchmarks.bespokelabs.aiTop model: Opus 5.511 models0 mixesQuality 45
- Slop ScoreLeaderboard and analysis tool scoring how much model writing resembles AI slop via overused words, contrast patterns, and trigrams. Maintained by Sam Paech. Score: Weighted composite of slop-word frequency (60%), 'not-x-but-y' contrast patterns (25%), and slop trigram frequency (15%) on outputs from a standardized prompt set; lower is better.
- SWE-bench MultimodalA 480-instance SWE-bench variant whose issue descriptions include visual elements. Maintained by SWE-bench Team. Score: % Resolved: percentage of task instances solved.
- BoardBenchModels play 30 unseen puzzles across ten board games (Xiangqi, chess, Go, backgammon, shogi, Scrabble, Kalah, Gomoku, Othello, Connect Four). An engine scores each legal move from 0 to 1 and any illegal move 0; the value is the mean score × 100.
boardbench.devTop model: GPT-6 Astra8 models0 mixesQuality 45
- BabyVisionUniPat AI's visual-reasoning test of basic perception a young child handles (tracking, pattern completion, visual puzzles): 388 items, 22 task types. Accuracy without tools; humans score 94.1%.
unipat.aiTop model: Qwen 3.8 Max11 models0 mixesQuality 44
- CADBenchUses Seldon's first-party mean verifier score. Score: mean verifier score. Collected by Jake Boggs' benchmark dashboard.
seldon.globalTop model: Fable 512 models0 mixesQuality 44
- MathVistaUses Kaggle's MathVista testmini accuracy leaderboard. Random-chance accuracy is 17.9%. Score: MathVista testmini accuracy. Collected by Jake Boggs' benchmark dashboard.
- Japanese-RP-Bench v2 (tegnike fork)Fork of Japanese-RP-Bench adding v2 metrics for role fidelity, conversation quality, persona stability, robustness, and recovery in Japanese roleplay. Maintained by tegnike. Score: Challenge RP Summary: macro-average of five criteria; leaderboard first sorts Major-free rate descending, then Major rate ascending.
- ArXivMath 06/2026 (MathArena)Research-level math problems with a checkable final answer, drawn from arXiv papers submitted in June 2026; each edition is restricted to papers published within the last month to reduce training-data contamination. Maintained by MathArena. Score: Accuracy (average performance on the competition) with a 95% confidence interval (normal approximation).
- BrokenArXiv 06/2026 (MathArena)Plausible but false proof statements taken from June 2026 arXiv papers; a model scores by refusing to prove them and saying the statement is false as written. Maintained by MathArena. Score: Accuracy (average performance on the competition) with a 95% confidence interval (normal approximation).
- SWE-bench MultilingualA 300-instance SWE-bench variant with tasks from 42 repositories across 9 programming languages. Maintained by SWE-bench Team. Score: % Resolved: percentage of task instances solved.
- Toolathlon-Verified (HKUST NLP)An agent completes 108 real-world tool-use tasks across 32 MCP servers and 7 local toolkits, and each task is graded by executing the resulting state against the task's ground truth. Maintained by HKUST NLP. Score: Pass@1 in percent, the mean over three runs (Verified series).
- OEIS Open (Epoch AI)Find the rule behind open integer sequences from the OEIS and extend them. Share solved. Data: Epoch AI, CC BY.
- ExploitBench v0.1Agentic memory-corruption exploitation on real V8 CVEs; mean share of the 16-flag capability ladder (coverage → arbitrary read/write → code execution) across seeds.
exploitbench.aiTop model: Mythos Preview4 models0 mixesQuality 44
- AIME 2025AIME 2025: competition math; accuracy.
- DeepsecBench-AA Refusal RateShare of DeepsecBench-AA tasks the model refused on safety grounds. Lower means fewer refusals on defensive security work.
- Global-MMLU-LiteGlobal-MMLU-Lite: multilingual MMLU subset; accuracy.
- LiveCodeBenchLiveCodeBench: competitive programming problems; pass rate.
- CAD-EvalUses Epoch AI's external CAD-Eval overall pass-rate leaderboard. Score: CAD-Eval overall pass rate. Collected by Jake Boggs' benchmark dashboard.
- ITSMBenchUses live first-party pass@1 across 89 enterprise IT service-management tasks. Score: pass@1. Collected by Jake Boggs' benchmark dashboard.
simulation.ressl.aiTop model: Opus 5.56 models0 mixesQuality 43
- PerceptionBenchUses the first-party overall atomic visual-perception accuracy. Score: overall atomic visual-perception accuracy. Collected by Jake Boggs' benchmark dashboard.
kimi.comTop model: GPT-5.6 Sol15 models0 mixesQuality 43
- BrokenArXiv 08/2026 (MathArena)Plausible but false statements taken from August 2026 arXiv papers; a model scores by refusing to prove them and saying the statement is false as written. Maintained by MathArena. Score: Accuracy (average performance on the competition) with a 95% confidence interval (normal approximation).
- LLM Divergent Thinking Creativity BenchmarkModels generate 25 words maximally distinct from each other and from 50 seed words under starting-letter constraints, scored by LLM-judged word divergence. Maintained by Lech Mazur. Score: Average LLM-judged score of minimum divergences between each generated word and other words, on a 0 to 10 scale (percentage of repeated words also reported).
- PingPong English (unadjusted mean)Role-playing benchmark where LLM interrogators emulate users in multi-turn character conversations, and LLM judges rate in-character, entertaining, and fluent qualities. Maintained by Ilya Gusev. Score: Unadjusted Avg score with character:entertain:fluency weights 1:1:1.
ilyagusev.github.ioTop model: Llama 3126 models0 mixesQuality 43
- SWE-rebench, issues 15 May – 1 Jul 2026 (Nebius)A model resolves 111 fresh GitHub issues from 65 repositories, created between 15 May and 1 July 2026, inside one fixed minimal agent scaffold, run five times. Maintained by Nebius. Score: Resolved rate (mean over five runs) on the task window's issues, fixed ReAct-style scaffold, 128K context.
swe-rebench.comTop model: Fable 513 models0 mixesQuality 43
- EnigmaEval1,184 long-form multimodal puzzles from puzzle hunts such as the MIT Mystery Hunt, graded by exact string match; accuracy over three runs. Private test set (Scale Labs, CAIS, MIT).
- NL2Repo-BenchBuild a complete, installable Python repository from a natural-language spec and an empty workspace (104 tasks, graded by the upstream pytest suites). Average test pass rate; scores are vendor and paper reports, gathered by LLMLearner.
llmlearner.comTop model: Opus 518 models0 mixesQuality 43
- BabyVisionUses UniPat AI's first-party visual-reasoning leaderboard score and excludes the human baseline. Score: score. Collected by Jake Boggs' benchmark dashboard.
- SWE Atlas Test Writing (Scale AI)Whether a coding agent writes production-grade tests for real repositories, graded with rubrics and LLM judges. Maintained by Scale AI. Score: Leaderboard score (percent); ± is the published confidence half-width.
- OEIS Open Lite (Epoch AI)The smaller Lite split of Epoch's OEIS Open sequence benchmark. Share solved. Data: Epoch AI, CC BY.
- MMLU-ProMMLU-Pro: broad knowledge multiple choice; accuracy.
- EdgeBenchUses the leaderboard's overall score at the longest (12-hour) environment-learning horizon. Score: 12-hour-horizon overall score. Collected by Jake Boggs' benchmark dashboard.
edge-bench.orgTop model: Opus 4.85 models0 mixesQuality 41
- Epoch Automation ReportsUses Epoch's equal-weight average of category rubric scores, including human-intervention deductions. Score: mean category rubric score. Collected by Jake Boggs' benchmark dashboard.
- GSM8KUses Epoch AI's external GSM8K exact-match leaderboard. Score: GSM8K exact match. Collected by Jake Boggs' benchmark dashboard.
- Context Arena — MRCR v2 (8-needle)The model must find and reproduce the right one of eight near-identical earlier replies hidden in a long conversation, averaged over context lengths from 8k to 128k tokens. Maintained by Context Arena. Score: Cumulative average score up to 128k tokens (unweighted mean of the 8k–128k bins), percent.
contextarena.aiTop model: GPT-5.6 Sol64 models0 mixesQuality 41
- ArXivMath 08/2026 (MathArena)Research-level math problems with a checkable final answer, drawn from arXiv papers submitted in August 2026; each edition is restricted to papers published within the last month to reduce training-data contamination. Maintained by MathArena. Score: Accuracy (average performance on the competition) with a 95% confidence interval (normal approximation).
- OSWorld-VerifiedComputer use on 361 verified Ubuntu desktop tasks (Chrome, LibreOffice, GIMP, VS Code…). Success rate of general models, best across step budgets.
os-world.github.ioTop model: Fable 513 models0 mixesQuality 41
- ARC AI2Uses Epoch AI's external ARC Challenge leaderboard. Random baseline is treated as 25% for multiple choice. Score: ARC AI2 Challenge score. Collected by Jake Boggs' benchmark dashboard.
- Arena-Hard-Auto v0.1Uses LMArena's Arena-Hard-Auto v0.1 leaderboard CSV; GPT-4-0314 is the anchor model. Score: Arena-Hard-Auto score. Collected by Jake Boggs' benchmark dashboard.
- BIG-Bench HardUses Epoch AI's external BIG-Bench Hard aggregate leaderboard. Score: BBH average. Collected by Jake Boggs' benchmark dashboard.
- CAD ArenaUses the published mean CAD Arena score across all CAD platforms, on its native 0-1 scale. Score: mean CAD Arena score across all platforms. Collected by Jake Boggs' benchmark dashboard.
normal.aiTop model: Gemini 4 Argon13 models0 mixesQuality 40
- WinoGrandeUses Epoch AI's external WinoGrande accuracy leaderboard. Random baseline is 50% for binary choice. Score: WinoGrande accuracy. Collected by Jake Boggs' benchmark dashboard.
- BuzzBenchAn LLM-judged evaluation of humour explanation and predicted audience response to Never Mind the Buzzcocks introductions. Maintained by Samuel Paech / EQ-Bench. Score: LLM judge score against human-authored gold responses; headline Score column.
- Vals Multimodal Index v1.2Benchmark consisting of a weighted performance across finance, coding, and education tasks. Showing the potential impact that LLM's can have on the economy. Scores are the values published by VALS AI. Source benchmark version 1.2.
- Web Search IndexCompares native model-provider search with independent web search tools on finance and legal research tasks.
- VoxelBench (image)Models recreate a reference image as a 3D voxel build; humans vote head-to-head on the renders. Glicko-2 rating.
- HellaSwagUses Epoch AI's external HellaSwag leaderboard. Random baseline is 25% for four-way multiple choice. Score: HellaSwag accuracy. Collected by Jake Boggs' benchmark dashboard.
- OpenBookQAUses Epoch AI's external OpenBookQA accuracy leaderboard. Random baseline is 25% for four-way multiple choice. Score: OpenBookQA accuracy. Collected by Jake Boggs' benchmark dashboard.
- Apex Shortlist 2025 (MathArena)47 final-answer problems from 2025 competitions on which at least one of two reference models (Grok 4 Fast, GPT-5 mini) got at least one of four attempts wrong, run by MathArena. Maintained by MathArena. Score: Accuracy (average performance on the competition) with a 95% confidence interval (normal approximation).
- RP-Bench Community EloA roleplay leaderboard reporting Bayesian community Elo alongside separate multi-turn judge and behavioural evaluations. Maintained by LeviTheWeasel / lazyweasel. Score: Bayesian community Elo.
- RULERNVIDIA synthetic benchmark that evaluates long-context LLMs with configurable sequence length and task complexity across 13 tasks in 4 categories. Maintained by NVIDIA. Score: Per-length scores at 4K-128K plus Avg. across the 13 RULER tasks (wAvg. inc/dec also reported); performance above the 4K Llama-2-7b threshold (85.6%) is underlined.
- AA-PersonalityBench ConvictionAA-PersonalityBench Conviction index: direct, confident, opinionated, and willing to push back. Percentile rank (1 to 100) across the models AA evaluated, from pairwise LLM-judged trait Elos. Describes style, not correctness.
- AA-PersonalityBench OrganizationAA-PersonalityBench Organization index: structured, scannable answers that lead with the bottom line. Percentile rank (1 to 100) across the models AA evaluated, from pairwise LLM-judged trait Elos. Describes style, not correctness.
- DROPUses LLM Stats / ZeroEval DROP F1 leaderboard. Score: DROP F1. Collected by Jake Boggs' benchmark dashboard.
- TriviaQAUses Epoch AI's external TriviaQA exact-match leaderboard. Score: TriviaQA exact match. Collected by Jake Boggs' benchmark dashboard.
- Scale SEAL: SWE-Bench Pro V2 Full1 sub-boardScale AI’s refreshed public SWE-Bench Pro split (642 long-horizon tasks in 11 copyleft repositories) run under a locked protocol, scored by resolve rate. Maintained by Scale AI. Score: Resolve rate on the full V2 public split (percent); ± is the published confidence half-width.
- VoiceCodeBenchSpeech-to-text task success on English workplace recordings that contain exact structured values such as addresses, email, dates, and file paths.
- AA-PersonalityBench ExpressivenessAA-PersonalityBench Expressiveness index: rich, varied, well-turned language. Percentile rank (1 to 100) across the models AA evaluated, from pairwise LLM-judged trait Elos. Describes style, not correctness.
- AA-PersonalityBench InformativenessAA-PersonalityBench Informativeness index: dense, evidence-based, specific, actionable answers. Percentile rank (1 to 100) across the models AA evaluated, from pairwise LLM-judged trait Elos. Describes style, not correctness.
- AA-PersonalityBench WarmthAA-PersonalityBench Warmth index: friendly, humorous, sincere, emotionally attuned. Percentile rank (1 to 100) across the models AA evaluated, from pairwise LLM-judged trait Elos. Describes style, not correctness.
- MortgageTax (Vals)Evaluating reading and understanding tax certificates as images Scores are the values published by VALS AI. Source benchmark version 1.
- Terminal-Bench 2.0 (Vals)State-of-the-art set of difficult terminal-based tasks Scores are the values published by VALS AI. Source benchmark version 2.0.
- AA-PersonalityBench TransparencyAA-PersonalityBench Transparency index: truthful, non-sycophantic answers that admit uncertainty and flag assumptions and nuance. Percentile rank (1 to 100) across the models AA evaluated, from pairwise LLM-judged trait Elos. Describes style, not correctness.
- TaxEval (Vals)A Vals-created set of questions and responses to tax questions Scores are the values published by VALS AI. Source benchmark version 2.
- BabyVision (with tools)BabyVision with image and Python tools available to the model. Accuracy.
- HMMT February 2026 (MathArena)The 33 final-answer problems of the February 2026 Harvard-MIT Mathematics Tournament, one of the largest high-school math competitions in the United States, run by MathArena on the contest problems. Maintained by MathArena. Score: Accuracy (average performance on the competition) with a 95% confidence interval (normal approximation).
- MCPMark VerifiedAI agents complete realistic multi-step tasks through Model Context Protocol servers for Notion, GitHub, a filesystem, Postgres and a Playwright browser, each checked by the task's own verification script; Verified is the maintainers' version-pinned, stabilized 127-task set. Maintained by EVAL SYS. Score: Pass@1 in percent: the share of the 127 Verified tasks a single run of the model passes, each task decided by its automated verify.py.
mcpmark.aiTop model: Kimi K38 models0 mixesQuality 32
- SlopBenchRuns 53 chat-style prompts and reports the percentage of model responses containing classic AI slop patterns, detected deterministically without an LLM judge. Maintained by Dan Cleary. Score: Slop Rate: percentage of responses with at least one slop-pattern hit (lower is better).
slop-bench.vercel.appTop model: GPT-5.3 Codex25 models0 mixesQuality 32
- LiveCodeBench (Vals)Our Implementation of the LiveCodeBench benchmark Scores are the values published by VALS AI. Source benchmark version 1.
- React Native EvalsCoding agents build React Native app features — navigation, animation, async state, lists, native APIs and Skia graphics — and an LLM judge checks the generated code against each task's written requirements. Maintained by Callstack. Score: The board's overall score in percent: the share of requirements met, summed over all 91 evals and every repeat run (an LLM judge decides each requirement against the generated files).
rn-evals.vercel.appTop model: Opus 523 models0 mixesQuality 31
- CaseLaw (Vals)Private question-answer benchmark over Canadian court-cases. Scores are the values published by VALS AI. Source benchmark version 2.
- CorpFin (Vals)A private benchmark evaluating understanding of long-context credit agreements Scores are the values published by VALS AI. Source benchmark version 2.
- ARC-AGI-2ARC-AGI-2 semi-private evaluation set: harder compositional grid puzzles designed to resist memorization; percent solved with verified cost per task.
- Arena DocumentHuman preference on document analysis and long-content reasoning; Bradley-Terry rating.
- AIME 2026 (MathArena)The 30 integer-answer problems of the 2026 American Invitational Mathematics Examination (AIME I and II), the qualifying exam for the USA Mathematical Olympiad, run by MathArena on the contest problems. Maintained by MathArena. Score: Accuracy (average performance on the competition) with a 95% confidence interval (normal approximation).
- MMLU Pro (Vals)Academic multiple-choice benchmark covering 14 subjects including STEM, humanities, and social sciences. Scores are the values published by VALS AI. Source benchmark version 1.
- MMMU Pro (Vals)Multimodal Multi-task Benchmark Scores are the values published by VALS AI. Source benchmark version 1.
- SWE-bench Verified (Vals)Solving production software engineering tasks Scores are the values published by VALS AI. Source benchmark version 1.
- AIME (Vals)Challenging national math exam given to top high-school students Scores are the values published by VALS AI. Source benchmark version 1.
- GPQA Diamond (Vals)Graduate-level Google-Proof Q&A benchmark evaluating models on questions that require deep reasoning. Scores are the values published by VALS AI. Source benchmark version 1.
- MedQA (Vals)Evaluating language model bias in medical questions. Scores are the values published by VALS AI. Source benchmark version 1.
- HMMT November 2025 (MathArena)The 30 final-answer problems of the November 2025 Harvard-MIT Mathematics Tournament, one of the largest high-school math competitions in the United States, run by MathArena on the contest problems. Maintained by MathArena. Score: Accuracy (average performance on the competition) with a 95% confidence interval (normal approximation).
- MATH 500 (Vals)Academic math benchmark on probability, algebra, and trigonometry Scores are the values published by VALS AI. Source benchmark version 1.
- USAMO 2026 (MathArena)The six proof problems of the 2026 USA Mathematical Olympiad, the final round of the American Mathematics Competitions, with each model's proofs graded by MathArena's LLM judges. Maintained by MathArena. Score: Accuracy: average score on the six proofs with a 95% confidence interval (normal approximation); LLM judges grade each proof against a rubric that a model writes from the official solution, mapping a proof to an integer score from 0 to 7.
- MGSM (Vals)A multilingual benchmark for mathematical questions. Scores are the values published by VALS AI. Source benchmark version 1.
- PrinzBenchA lawyer's private benchmark: 25 legal research questions in a niche area of U.S. law (find the statutes, regulations and guidance online, then reach the right conclusion) and 8 needle-in-the-haystack web search questions. Each is asked three times and hand-graded by the author at pass@1, not blind; score out of 99. The author considers it saturated since June 2026 and no longer tests OpenAI's Pro models.
- ARC-AGI-1ARC-AGI-1 semi-private evaluation set: novel grid-transformation puzzles measuring fluid intelligence; percent of tasks solved, with verified cost per task.
- Public Benefits Bench v1 (Vals)Can AI help people navigate SNAP benefits? Scores are the values published by VALS AI. Source benchmark version 1.
- Poker AgentTrueSkill rating for language models playing multiplayer no-limit Texas Hold'em; Vals scales ratings by 40 on a 1000-point baseline.
- MATH-500MATH-500: competition math subset; accuracy.