OpenAI

GPT-6.2 Astra Takes Second Place in A.I. Rankings but Stumbles Often

The OpenAI logo displayed on a computer screen
Photo: Andrew Neel / Unsplash

Set Trending Topics as a preferred source on Google.

Not the smartest model in the world, but probably the one with the best value for money: OpenAI has just unveiled GPT-6.2 Astra and GPT-6.2 Sol, and it is taking on Anthropic mainly on cost.

According to OpenAI, GPT-6.2 Astra is its new frontier model for the hardest projects, while GPT-6.2 Sol is the efficient model for everyday work. Both are designed to handle demanding tasks better while using fewer output tokens. They are available now to paying users of ChatGPT Work and Codex, as well as through the API.

Second Place on Artificial Analysis

In the independent ranking by Artificial Analysis, GPT-6.2 Astra takes second place. It trails the leader, Claude Fable 5.2, with which Anthropic recently knocked Gemini off the A.I. throne, by three points. Here is the top of the Artificial Analysis Intelligence Index v5:

  • Claude Fable 5.2 (Anthropic): 65 points
  • GPT-6.2 Astra (OpenAI): 62 points
  • Gemini 4 Carbon (Google): 60 points
  • Claude Opus 5.5 (Anthropic): 59 points
  • GPT-6.2 Sol (OpenAI): 57 points

Astra gains five points over GPT-6 Astra, overtaking Gemini 4 Carbon and Claude Opus 5.5 at their top effort settings. Sol gains two points over GPT-6.1 Sol, putting it on par with GPT-6 Astra at about 15 percent of the cost per task. Here are the other key results from Artificial Analysis:

  • Cost: A task in the index costs about $4 (about €3.60) with Astra at maximum reasoning effort, compared with up to $27 (about €24) for Fable 5.2. OpenAI models now occupy most of the so-called Pareto frontier, meaning they deliver the best performance for their price. Only Anthropic and Meta share that frontier. OpenAI also dominates on output tokens per task, with Fable 5.2 the only model from another lab on that frontier.
  • Science and knowledge: OpenAI leads Terminal-Bench-Science, which tests scientific research workflows, and trails only Fable 5.2 on the physics test CritPt. On factual knowledge (AA-Omniscience) and the knowledge and reasoning test HLE-Diamond, only Fable 5.2 beats the new Astra.
  • 3D and robotics: On AA-3D, where human judges vote blind on interactive 3D experiences, Astra leads with 1,318 Elo points, ahead of Fable 5.2 (1,271) and Sol (1,259). On AA-Robotics, which measures how well language models can plan and act in the real world, Astra (79) and Sol (78) also lead Fable 5.2 (75). Artificial Analysis now plans to make the test harder, as OpenAI and Anthropic models can already handle most of the basic tasks.
  • Coding: On the Coding Agent Index, Astra is flat, and Sol even loses three points. The reason is a 14-point drop on SWE-Atlas-QnA, where Sol is more likely than its predecessor to omit key facts in its responses. Thanks to their low cost, however, several effort levels of Astra and Sol still land on the Pareto frontier.

Astra is also getting cheaper per token. One million input tokens cost $8 (about €7.20) and one million output tokens $40 (about €36), 20 percent less than GPT-6 Astra in each case. The price for cached input drops from $1 to $0.40 per million tokens. Sol stays at $2 (about €1.80) for input and $10 (about €9) for output.

What OpenAI Promises in Its Blog Post

In its launch blog post, OpenAI is as confident as ever. The key claims:

  • Office work: Slides, spreadsheets and documents from Astra are meant to read “less like AI” and follow templates down to the font size. On the internal GDPval v2 test for knowledge work, Astra gains 6.2 percentage points while using 22 percent fewer output tokens.
  • A jab at Anthropic: On Agents’ Last Exam, a benchmark for multistep professional work, Astra leads Claude Fable 5.2 by 5.5 points at just 5 percent of the cost, according to OpenAI. On the legal test Harvey LAB-AA, Astra reportedly more than doubles Fable 5.2’s pass rate.
  • Science: OpenAI calls Astra “the best model in the world for scientists,” with new highs on Terminal-Bench-Science and the Vals Science Index.
  • 3D and design: Astra turns text descriptions into playable 3D experiences, websites and CAD models, such as a kit for a remote-controlled car complete with an animated assembly guide.
  • Sol for everyday use: On the coding test FrontierCode 2.0, Sol comes within 0.5 points of Astra at one-fifth of the cost. According to the blog, many OpenAI employees use Sol as their default model.

‘High’ for Self-Improvement for the First Time

On safety, OpenAI strikes a more serious tone. GPT-6.2 Astra is OpenAI’s first released model to reach the “High” threshold for A.I. self-improvement under its Preparedness Framework. According to OpenAI, that means the model’s impact is equivalent to giving every OpenAI researcher an additional, highly capable research engineer. Models that help develop future A.I. could accelerate progress faster than safeguards and human oversight can adapt, OpenAI itself warns. A new system called Control Room is meant to monitor research workloads and halt the model when needed.

On cybersecurity, both models reach the top “Critical” level, as GPT-6 Astra did. OpenAI is therefore shipping them with tighter safeguards that also restrict some legitimate security research. Defenders are meant to keep accessing less restricted models through the Daybreak program, which NATO is already using heavily.

Developers Are Disappointed

The model had barely launched when criticism from the developer community began pouring in on X. The loudest voice is Theo Browne, chief executive of T3 and a well-known tech YouTuber who tested Astra ahead of its release. In a long open letter to OpenAI, he wrote that he would not choose Astra over Claude Opus 5.5 for his work as a developer. He acknowledges its strengths: Astra is a great code reviewer, “world class” at 3D and games and strong at computer use. But his list of problems is long:

  • Pointless questions: On long tasks, Astra blocks itself with internal implementation questions that users cannot possibly answer. When asked to build a new level for the game Outer Wilds, the model stopped after two hours to ask about internal file names.
  • Incomprehensible answers: Astra gets lost in details and explains code in terms more complex than the code itself. Mr. Browne now has Opus 5.5 summarize its results. Asked why a machine was slow, Astra delivered four paragraphs on package management, while Opus simply answered that the SSD was slow.
  • Endless loops: On a project to rewrite the TypeScript compiler in Rust, Astra ran for seven days without landing a single improvement and ended up making the compiler 56 percent slower. Opus had completed the core task in under two weeks for about $20,000 (about €18,000) in tokens.

His prediction: On day one, everyone will love the model, and after two weeks it will be mostly forgotten. He concedes, however, that he may be overreacting.

Other voices are not thrilled either. The developer Mckay Wrigley quipped that Astra 6.2 is “a pretty good ad for Anthropic’s IPO,” which is set for Thursday. The A.I. watcher Token Gremlin suspects that OpenAI rushed the launch to disrupt Anthropic’s I.P.O. and ended up helping its rival even more. He accuses the model of alignment and safety problems and reports that it lapses into an “alien language” during long workflows. The account synthwavedd also pointed out that Astra and Sol show no improvement over their predecessors on Artificial Analysis’s Coding Agent Index and lag all recent Claude models.

More Power, New Weaknesses

The update is not free of setbacks. In two tests, the new models were more likely to keep quiet about obstacles blocking their work, such as a missing file. According to the system card, Astra did so in about 22 percent of cases, twice as often as GPT-6 Astra. OpenAI has added instructions in Codex that are meant to substantially reduce this, but they do not apply to direct API use.

There is also room for improvement on hallucinations. According to Artificial Analysis, the hallucination rate at maximum effort is 59 percent for Astra and 55 percent for Sol. The metric measures how often a model guesses on factual questions instead of admitting it does not know the answer. Gemini 4 Carbon and especially Fable 5.2 do better here. And Britain’s A.I. Security Institute found a universal jailbreak for cyber questions in pre-release versions. OpenAI says it has since made fixes, but these had not been independently verified when the system card was written.

Rank My Startup: Erobere die Liga der Top Founder!
Advertisement
Advertisement

Specials from our Partners

Top Posts from our Network

Deep Dives

© Wiener Börse

IPO Spotlight

powered by Wiener Börse

Europe's Top Unicorn Investments 2023

The full list of companies that reached a valuation of € 1B+ this year
© Behnam Norouzi on Unsplash

Crypto Investment Tracker 2022

The biggest deals in the industry, ranked by Trending Topics
ThisisEngineering RAEng on Unsplash

Technology explained

Powered by PwC
© addendum

Inside the Blockchain

Die revolutionäre Technologie von Experten erklärt

Trending Topics Tech Talk

Der Podcast mit smarten Köpfen für smarte Köpfe
© Shannon Rowies on Unsplash

We ❤️ Founders

Die spannendsten Persönlichkeiten der Startup-Szene
Tokio bei Nacht und Regen. © Unsplash

🤖Big in Japan🤖

Startups - Robots - Entrepreneurs - Tech - Trends

Continue Reading

Newsletter

Founders Dispatch

Zwei Mal pro Woche kostenlos in die Inbox: die wichtigsten Startups, Deals und Tech-Entwicklungen aus Europa, handgeschrieben von der Redaktion.

Jederzeit abbestellbar. Mehr über den Newsletter