ARC Prize has reported new ARC-AGI-3 results for OpenAI’s GPT-6 Astra that shift attention beyond the base model and toward the systems wrapped around it.
On ARC-AGI-3 Semi-Private, Astra scored 62.7% at a reported cost of roughly $26,100 using the benchmark’s Standard harness. In a separate Provider Adapter harness, it scored 99.9% at about $18,800. ARC-AGI-3 measures how agents explore unfamiliar turn-based environments, infer their mechanics and goals, and execute a plan—not simply whether they return a correct answer.
The figures are striking, but the comparison between the two setups is the more useful takeaway for builders and technology leaders.
The harness is part of the capability
The Standard harness lets a model retain notes that it chooses to carry across an environment. The Provider Adapter, by contrast, preserves opaque reasoning state between requests and compacts long conversations, allowing the system to reuse prior work.
That distinction produced a very large performance gap in the reported tests. It is a reminder that an agent’s operating environment—persistent state, context management, tool access, and the control loop around the model—can materially change outcomes.
For companies evaluating AI agents, benchmark claims should therefore trigger a set of implementation questions: What memory is retained between steps? Is the model receiving summarized state, raw history, or provider-managed reasoning state? What tools can it call? And how much of the result comes from the model versus the orchestration layer?
Those questions matter in production settings such as support operations, software engineering, research workflows and back-office automation, where tasks often span many steps and depend on maintaining an accurate model of the current situation.
Efficiency, not only completion
ARC Prize also says Astra, in the Provider Adapter setup at maximum reasoning effort, used fewer actions than its median human baseline on 96% of levels and 51.7% fewer actions per level on average.
That metric is important because agent economics are driven not just by the price of a model call, but by how many interactions are needed to discover a task’s rules and complete it. Fewer exploratory actions can mean lower latency, fewer tokens and less exposure to execution errors.
ARC Prize’s cost table illustrates the point: its higher-reasoning configurations sometimes cost less overall because they solve games in fewer actions, reducing total calls and tokens. The implication for operators is not that maximum reasoning is always the correct production choice. It is that teams should optimize end-to-end task cost and reliability, rather than choose a model setting based only on per-token pricing or nominal speed.
Symbolic notes and task-specific software
In published replays, ARC Prize observed Astra creating compact, code-like notes to represent game state, mechanics, control locations and multi-step plans. The model used shorthand for objects and coordinates, then translated that working model into action sequences.
In a separate PRO-LONG evaluation harness, where Astra had access to a sandbox for custom code, ARC Prize reports that it created game-specific parsers, state models, planners and search tools. Those runs represent combined performance of the model and its available tools, rather than a direct comparison to the controlled human testing, the organization notes.
For builders, that is still meaningful. Mature agent architectures may increasingly look less like a single model answering a prompt and more like a system that forms durable task representations, writes narrow tools when needed, validates predictions against observed state, and keeps a recoverable audit trail.
What to watch next
ARC-AGI-3 is designed around abstract environments, so its results should not be treated as direct evidence of broad workplace autonomy. The reported top score also depends on a specific harness, making reproducibility and disclosure of evaluation conditions essential.
The next tests are practical: whether the same combination of persistent state and tool use remains reliable on messy business systems, whether failures can be bounded and reviewed, and whether the economics hold at production scale. Astra’s result suggests that the race is no longer only about smarter models. It is also about building the right memory and execution systems around them.
