Somewhere out there, a frontier AI model — possibly worth hundreds of billions of dollars — just failed to figure out how to play a simple turn-based puzzle game. Not failed badly. Failed completely. Scored below 1%. Meanwhile, random humans off the street solved 100% of the same environments.

Welcome to ARC-AGI-3.

Key result: Humans solve 100% of ARC-AGI-3 environments. Frontier AI systems, as of March 2026, score below 1%.

What the Paper Does

Published on arXiv on March 24, 2026, ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence introduces the third generation of the Abstraction and Reasoning Corpus benchmark series from François Chollet and the ARC Prize Foundation.

While ARC-AGI-1 tested pattern inference from static grid examples, and ARC-AGI-2 pushed those same grids into harder multi-step reasoning, ARC-AGI-3 fundamentally changes the game. Literally. It moves from static input-output tasks to interactive, turn-based game environments where an agent must:

  1. Explore — actively probe the environment to gather information
  2. Model — build a world model from observations
  3. Set its own goals — infer what "winning" looks like without being told
  4. Plan and execute — navigate from current state to win condition efficiently

No instructions are given. No language. No cultural symbols. No "press X to continue." You're dropped into a 64×64 grid of colored cells and you figure it out — or you don't.

The headline result, sourced directly from the paper: humans solve 100% of ARC-AGI-3 environments. Frontier AI systems, as of March 2026, score below 1%.

Why It Matters

To appreciate why this number is so jarring, you need context on the arc (pun intended) of this benchmark series.

ARC-AGI-1 launched in 2019 and was considered a durable test of fluid intelligence. It resisted base LLM scaling entirely — models trained on more text didn't get better at it. Then test-time reasoning came along. OpenAI's o1 and o3 broke through, and by the 2024 ARC Prize competition, test-time training had hit 53.5% accuracy. ARC-AGI-1 was effectively cracked.

ARC-AGI-2 arrived in March 2025 as a harder static reasoning challenge. NVIDIA's team won with 24% accuracy. Progress was real, if incomplete.

But both benchmarks had a problem the ARC team couldn't ignore: they were potentially being memorized at scale. The paper includes damning evidence — Gemini 3 Deep Think used the correct ARC color-integer mapping in its reasoning chains without being told what it was, strongly suggesting the model had trained on ARC task data. When your benchmark is on the internet, it ends up in training data.

ARC-AGI-3 solves this with interactive environments. You can't memorize your way through a novel turn-based puzzle. You have to actually reason about what's happening in real time.

The paper frames this through Chollet's original definition of intelligence: skill-acquisition efficiency on novel tasks. It's not about what you know. It's about how fast you can figure out something you've never seen before. By that definition, frontier LLMs remain deeply limited — bounded by their training distribution even when they appear to reason fluently.

How It Works (Simplified)

Each ARC-AGI-3 environment is a series of levels built around a shared set of mechanics. Agents view a 64×64 grid where each cell is one of 16 colors. The action space is deliberately small: five directional keys, an undo, and the ability to click a specific cell. Complexity comes from the logic of the environment, not the difficulty of the controls.

Critically, the environments use only Core Knowledge priors — the kind of intuitive physics and object understanding that any human toddler has:

  • Objects persist and move
  • Basic geometry: symmetry, rotation, inside/outside
  • Intuitive physics: gravity, bouncing, momentum
  • Agentness: some things have goals

No numbers. No letters. No familiar game mechanics (each environment is validated for novelty against existing games). The first level of any environment is always a tutorial designed to communicate the core interaction pattern — but later levels require full mechanical understanding built up through play.

The Scoring System: RHAE

The benchmark introduces a new metric called RHAE (Relative Human Action Efficiency) — pronounced "Ray." It works like this:

For each level completed, the AI's action count is compared to the second-best human performance on that level. The score is the square of the efficiency ratio, capped at 1.0:

S(level) = min(1.0, human_actions / ai_actions)²

The squaring is intentional. Under a linear metric, an AI taking 2× the human action count would still score 50% — deceptively high for what is genuinely poor performance. The quadratic penalty makes the scoring honest: an AI that needs 10× the human actions scores just 1%.

Levels are weighted linearly — later levels count more than tutorial levels. Environments are scored as the weighted average across their five levels. The overall score is the mean across all environments.

The benchmark also applies a hard cutoff: if an AI is burning 5× more actions than humans on a given level, the run is terminated. No infinite token budgets exploiting brute-force strategies.

Private vs. Public Sets

Unlike ARC-AGI-2 (which had a 10:1 public-to-private ratio), ARC-AGI-3 inverts this to protect evaluation integrity. The public set is a small demonstration interface. The private set — the real benchmark — is OOD from the public set, covers broader mechanics, and is tightly guarded for the official ARC Prize competition.

The ARC team is releasing a "harness" that scores 100% on all public environments using human replay. The message is clear: don't bother optimizing for the public set. It is not a valid measure of progress.

Key Results

The paper's results are blunt:

  • Humans: 100% solve rate across all ARC-AGI-3 environments tested
  • Frontier AI (March 2026): below 1% on the benchmark
  • Human baseline data collected from 10 untimed members of the public per environment, in controlled in-person testing
  • The "second-best human" score is used as the normalization baseline, removing outlier performance while maintaining a strong human capability bar

For comparison:

  • ARC-AGI-1: Frontier models hit 53.5% (with test-time training) by late 2024
  • ARC-AGI-2: Best result was 24% as of the ARC Prize 2025 competition

ARC-AGI-3 represents a clean reset. The gap is not a difference in degree — it's categorical. Current AI simply cannot do what ARC-AGI-3 requires.

What This Means for Practitioners

If you're building agents today, this paper has several uncomfortable implications.

Your agent can't explore. The environments in ARC-AGI-3 require actively probing the world to gather information. LRM-based agents are optimized to consume and generate text based on what they already know. Active, efficient information gathering from an unknown environment is genuinely outside their design.

Your agent can't set its own goals. This is the deepest failure mode. In every production agent system today, goals are specified by humans in the prompt. ARC-AGI-3 requires autonomous goal inference — figuring out what "winning" means from environmental observations alone. No current frontier model can reliably do this.

Test-time reasoning is necessary but not sufficient. The LRM revolution (o1, o3, DeepSeek R1, Gemini Deep Think) showed that test-time reasoning could unlock fluid intelligence on static tasks. ARC-AGI-3 shows this isn't enough. Interactive environments demand something qualitatively different — dynamic world models and real-time strategic adaptation.

Memorization tricks won't work here. The reason ARC-AGI-3 uses interactive environments isn't aesthetic. It's adversarial. You cannot generate synthetic ARC-AGI-3 tasks at scale and train on them the way you could with ARC-AGI-1/2 because each interaction step depends on the previous one. The combinatorial space explodes. This is, intentionally, harder to saturate with data.

The paper identifies the limits of LRM intelligence precisely: Current reasoning models are knowledge-dependent. They can reason over domains they've been trained on, but show limited ability to transfer that reasoning to genuinely novel domains. ARC-AGI-3 is explicitly designed to test that transfer capability — and the results show it's close to zero.

Our Take

ARC-AGI-3 is the most honest intelligence benchmark released in years. Most benchmarks get saturated fast — either because models train on their test sets, or because the tasks are fundamentally about retrieval rather than reasoning. The interactive format of ARC-AGI-3 is genuinely adversarial to the dominant AI paradigm.

The below-1% result isn't a scandal. It's information. It tells us exactly where the ceiling is for current architectures when stripped of their knowledge scaffolding and asked to do something truly novel.

What's interesting is what this doesn't say. It doesn't say current AI is useless — coding agents, RAG systems, and LRM-powered tools are transforming real workflows right now. But it does say that "thinking like a human" — in the sense of efficiently navigating completely unknown territory — remains an unsolved problem.

For OpenClaw builders specifically: this is a useful compass. When your agent pipeline fails, it's almost certainly because of goal ambiguity, poor exploration strategy, or knowledge gaps — exactly the three things ARC-AGI-3 exposes. Building better agents means addressing those three things directly, not just upgrading the underlying model.

The ARC Prize competition will run on the private set. The grand prize threshold is 85%. Given the current <1% baseline, that target will not fall anytime soon. But the chase is exactly where the interesting AI research lives in 2026.

Citation

ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence
ARC Prize Foundation (Lead: François Chollet)
arXiv:2603.24621 [cs.AI] — March 24, 2026
Read the full paper →

Build More Capable Agents

The OpenClaw Field Guide covers agent architecture, goal management, exploration strategies, and production deployment — everything you need to build agents that go beyond text generation.

Get the Field Guide — $10 →