AA v5 Looks Balanced at the Top. The Test Got Harder.
Artificial Analysis replaced Agents with Knowledge Work, added two new benchmarks, and put 60% of the score on private questions. Open-weight models fell.

Artificial Analysis has updated its leaderboard again.
The top looks familiar. Claude Opus 5.5 is first. GPT-6 Astra is second. GPT-6.1 Sol is third. Anthropic and OpenAI still hold the top five. A quick look at the total scores makes the update seem like another balancing act: small gaps, stable leaders, no clear blowout.
The top three draw attention. The test design matters more. AA changed its categories, sub-benchmarks, question sources, and weights. The ranking moved. The evaluation logic moved further.
1. Agents becomes Knowledge Work
The four categories used to be Agents, Coding, Science, and General. They are now Knowledge Work, Coding, Science, and General. The weights changed from 30/20/20/30 to 25% each.
The old Agents category was hard to explain. Many tasks looked like agents. Many also looked like coding. A model’s Agents score was difficult to interpret. Knowledge Work points to concrete tasks: AA-Briefcase, AutomationBench-AA, and GDP.pdf ask models to process business materials and complete professional work. Coding means writing code, changing repositories, and passing tests. Science means research tasks and complex reasoning in a terminal. General still tests world knowledge.
Clear categories make scores easier to explain. If a leaderboard mixes categories, a busy ranking still tells you little. AA now lists the sub-benchmarks and weights under each category. Disputes can land on specific tasks.
2. New sub-benchmarks aim to stop gaming
AA added FrontierCode 2.0 and Terminal-Bench-Science.
FrontierCode 2.0 comes from Cognition. It contains 132 tasks based on real open-source repositories. The score depends on whether maintainers would merge the code, not whether tests pass. That standard moves evaluation from “can it run” to “can the code enter a real project.” Terminal-Bench-Science contains 70 research tasks run in a terminal. Experts designed and reviewed them. The benchmark tests how models handle complex scientific workflows.
AA removed GDPval-AA, SciCode, and AA-LCR. SciCode had too many flawed questions and too little separation between models. GDPval-AA and AA-LCR overlapped with Knowledge Work tasks. Too many tasks of the same kind made targeted training too easy. The earlier tau3 banking benchmark belonged to the same long-text, professional-work group. AA consolidated and replaced them to cut noise.
Three upgrades follow the same logic. HLE became HLE-Diamond, refined to 1,000 questions split between reasoning and knowledge. CritPt became CritPt v2, a private 40-question high-difficulty subset revised by the original authors. AA-Omniscience moved to V1.1, with saturated questions and public-set questions removed and ambiguous questions revised. These changes are quiet. They filter out flawed questions, easy questions, and public questions. What remains can separate models.
3. Private weight rises, open-weight scores fall
The top held. The middle and lower ranks moved sharply. For the first time, v5 puts 60% of the evaluation weight on private datasets or held-out answers.
Meta Muse Wonder Full fell from 56 to 50. Kimi K3.1 Pro fell from 55 to 46. DeepSeek V4.1 Flash fell from 45 to 32. Muse Spark Open from Meta fell from 39 to 25.
Kimi K3.1 Pro is still the highest-scoring open-weight model. Its gap from the top widened from 3 points to 12. Among models with released weights, DeepSeek V4.1 Flash and GLM-5.3 score 32. That is 26 points behind the leader and below Claude Haiku 5.5 at 36.
Some readers will see bias in that result. Another reading: private datasets lower the payoff of gaming public questions. Once questions are public, they get studied and optimized against. A higher score does not always mean higher capability. AA put 60% of the weight on private sets. Benchmark gaming became less worthwhile. Open-weight scores fell. That does not prove their capabilities dropped overnight. Part of the old score may have come from adapting to public questions. The reading is uncomfortable. It also supports the leaderboard’s credibility.
4. AA has more benchmarks in reserve
AA’s reserve goes beyond v5. Code2Video-AA is already a standalone sub-benchmark for code-generated video. AA-3D tests 3D modeling. MLCR-AA tests long-text medical processing. Harvey LAB-AA tests legal work. These benchmarks are still developing. After the market adapts to v5, AA can bring in new tests to keep separation between models.
Public leaderboards often fail because updates lag behind model iteration. Questions get gamed. Weights stay fixed. Categories go stale. Scores turn into a standard-answer contest for a public exam. AA does two things: it replaces questions when they saturate, and it restructures categories when they stop explaining model behavior. Its methods are transparent. Its update pace is steady. Call it a wild leaderboard if you want. Among wild leaderboards, it is less wild than most.
5. The next version will reshuffle again
The v5 score sheet will not last. FrontierCode 2.0 and Terminal-Bench-Science separate models today. They can be studied through tomorrow. If AA replaces them in the next version, or moves more weight to private sets, today’s ranking will change again.
The concrete facts from this update: Agents became Knowledge Work. The four categories each take 25%. FrontierCode 2.0 scores by whether maintainers merge code. Terminal-Bench-Science contains 70 terminal research tasks. GDPval-AA, SciCode, and AA-LCR were removed. HLE-Diamond, CritPt v2, and AA-Omniscience v1.1 went online. 60% of the weight comes from private datasets or held-out answers. Kimi K3.1 Pro fell from 55 to 46. DeepSeek V4.1 Flash fell from 45 to 32.
The top looks balanced. The questions got harder.
About the Creator
Jin
Writer of reamstories
https://reamstories.com/jin
Enjoyed the story? Support the Creator.
Subscribe for free to receive all their stories in your feed. You could also become a paid subscriber, letting them know you appreciate their work.
Comments
There are no comments for this story
Be the first to respond and start the conversation.