A Small AI Model Solved 76% of ARC-AGI 1 Tasks in Four Hours

A 75-million-parameter transformer trained from scratch on ARC data solved approximately 76% of tasks on the ARC-AGI 1 public evaluation set, reaching the performance range of much larger reasoning models using a single GPU.

Most progress in AI reasoning has followed the same basic formula: build a larger model, train it on more data and give it more time to reason. This approach has produced increasingly capable systems, but it has also made advanced reasoning expensive to train, costly to run, and difficult for most researchers to experiment with.

ARC-AGI has become one of the clearest tests of whether scale is the only path forward. The benchmark asks AI systems to infer a completely new transformation rule from only a few examples and apply it to transform an unfamiliar input grid into its correct output grid. Humans can often recognize these patterns quickly, while even advanced AI systems still struggle with them. The strongest results have generally come from large, broadly pretrained language models that use substantial computation to reason through each task.

The question was how far a much smaller, specialized system could go. The experiment originally focused on training a fitness function that could guide evolutionary search toward the correct output grid. That approach did not work as well as hoped. But testing it required a model that could generate possible solutions, and that supporting model became the more interesting result.

The small transformer model was redesigned around the structure of ARC-AGI. It preserved the two-dimensional layout of each grid, learned from augmented versions of the training examples, used a demonstration directly in its context, and predicted several upcoming tokens at once. Together, those choices helped a 75-million-parameter model solve approximately 76% of tasks on the ARC-AGI 1 public evaluation set, a score matching language models with billions, if not trillions, of parameters. Training the model from scratch and generating its answers took a little over four hours on a single NVIDIA H100 GPU.

That combination of performance, size, and speed is what makes the result important. It reached a level of performance associated with much larger reasoning models without broad internet-scale pretraining or an enormous compute budget. This does not mean specialized models can replace frontier LLMs across every task. It shows that scale is not the only source of capability. When a problem has a clear structure, representing it in a suitable way can make strong reasoning faster, cheaper, and far easier to improve through continued experimentation.

Key Findings

• Model size: 75 million parameters

• ARC-AGI 1 performance: ~76% pass@2 on the public evaluation set

• Training + evaluation: A little over four hours on one NVIDIA H100 GPU

• Faster configuration: ~44% in approximately 17 minutes

• Eight-model ensemble: ~80% on ARC-AGI 1

• ARC-AGI 2: ~17% with one model and ~22% with the ensemble

• Key techniques: 2D positional representation, data augmentation, multi-token prediction and candidate reranking

• Main takeaway: Specialized models designed around a problem's structure can achieve strong reasoning performance without frontier-model scale.

Why Is ARC-AGI Difficult for AI Reasoning Models? 

ARC-AGI looks simple at first. Each task is made up of small grids, no larger than 30 by 30 cells, using ten possible colors. A few input-output pairs demonstrate an unknown transformation rule. The rule might be to complete a pattern, move objects as if pulled by gravity, remove everything except the largest structure, or transform the grid in another way. The system then receives a new input and must generate the correct output under the same rule.

The difficulty comes from how little information is available. Each task typically provides only two to six demonstrations, and the rule changes from one task to the next. There is no large collection of nearly identical examples from which a model can slowly learn what to do. It must determine which objects and relationships matter, infer the transformation from a handful of clues, and apply it precisely to a case it has never seen.

This kind of rapid learning comes naturally to people. A person can look at a few examples, form a hypothesis about the rule, and test that hypothesis against a new grid. Current AI systems are usually built around the opposite advantage: they learn from enormous datasets and become powerful by absorbing patterns at scale. ARC-AGI is difficult because it asks them to adapt when that scale is not available for the task in front of them.

That makes the benchmark relevant despite its simple structure. Useful AI systems must be able to enter unfamiliar situations, identify the structure that matters, and learn from limited evidence. Progress on ARC-AGI therefore offers a way to study how AI might become more efficient and adaptable in the real world.

Animation 1. Given a demonstration pair showing an input and its correct output, followed by a new test input, the system generates the corresponding test output one cell at a time. It must infer the underlying transformation from the demonstration and apply it to the new grid.

How the 75-Million-Parameter Model Was Built for ARC-AGI 

The core system is a standard autoregressive transformer, the same general type of model used to predict the next token in language. The difference is that this transformer was built around the structure of ARC-AGI rather than the structure of written text.

Preserving the Grid's Two-Dimensional Structure

Its vocabulary contains only 14 tokens: the ten possible grid colors and four structural tokens that mark the beginning and end of a sequence, separate one grid from another, and indicate a new row. The model receives a demonstration pair, followed by a new input grid, and predicts the output one cell at a time.

The model was trained entirely on ARC-related data, including the demonstration pairs provided for the public evaluation tasks, but not the held-out test outputs used to calculate the final score.

Simply converting a grid into a sequence creates a problem. A transformer could see the cells as a one-dimensional string and lose the spatial relationships that define the task. To prevent that, each token also receives coordinates describing its row, column, and position among the demonstration and test grids. The model still generates a sequence, but it can understand where each cell is located within the two-dimensional puzzle as shown in the figure below.

Figure 1: Sequence encoding for demonstration and test grids. Each grid cell has an x- and y-coordinate that determines its row and column position, as well as a z-coordinate that signifies to which grid it belongs. The “↵” signifies a special newline token that additionally indicates the end of a row to the transformer model.

Using Multi-Token Prediction

A conventional autoregressive model learns to predict only the next token. This model learns to predict several upcoming tokens at once, encouraging it to produce outputs that remain coherent farther into the future. This change had a meaningful effect. Standard next-token prediction reached approximately 70% on ARC-AGI 1, while predicting four tokens ahead increased performance to 76%. Multi-token prediction also enabled faster generation during inference through self-speculative decoding.

Expanding Training Data Through Grid Augmentation

The model was trained on grids that had been recolored, rotated, reflected, and scaled while preserving the transformation connecting each input to its output. A red object might become blue, a pattern facing left might be rotated to face down, or each cell might be expanded into a larger block. These variations forced the model to look past surface details and learn rules that could still hold when a puzzle appeared in a different form.

Generating and Ranking Candidate Solutions

At inference time, the system applies the same idea from several directions. It transforms the demonstration and test grids in different ways, generates candidate outputs, reverses the transformations, and compares the results. Lightweight symbolic checks then remove candidates that violate relationships shared by every demonstration, such as a consistent ratio between the input and output sizes. The remaining candidates are ranked using augmented likelihood scoring, which measures how strongly the model supports each proposed output across several transformed views of the task.

How Did the Model Reach 76% on ARC-AGI 1 With One GPU? 

The final 75-million-parameter system reached approximately 76% pass@2 accuracy on the ARC-AGI 1 public evaluation set. In practical terms, one of its top two proposed outputs was correct for roughly three out of four tasks. Training the model from scratch and evaluating it took a little over four hours on a single H100 GPU.

A second version was optimized for throughput. That system completed training in approximately 12 minutes and evaluation in another five, bringing the full process to about 17 minutes. Even under that much smaller compute budget, it reached approximately 44% on ARC-AGI 1.

This is important because most leading systems report the cost of generating solutions with an already trained frontier model. The cost reported for this system includes both training the model from scratch and using it to generate the answers. Frontier models have broader capabilities and were not trained only for ARC-AGI, so this is not a direct comparison of everything those systems can do. It does show how much efficiency becomes possible when a model is designed for a particular problem.

An ensemble of eight independently trained models increased the ARC-AGI 1 result to approximately 80%. The same approach reached about 17% on the harder ARC-AGI 2 benchmark with one model and roughly 22% with the ensemble. The gap between those results makes clear that the approach has not solved abstract reasoning as a whole, but it also provides a fast, affordable environment in which to study what the next improvement requires.

These scores are estimates from ARC-AGI’s freely available public evaluation set. They have not been evaluated on the benchmark’s private leaderboard, so the chart below should not be read as an official leaderboard ranking. The vertical ranges reflect that uncertainty, and comparisons should account for differences in evaluation settings and how costs are reported.

Figure 2. Estimated performance of the system on the ARC-AGI 1 public evaluation set compared with reported results from leading models. Vertical ranges indicate that the result has not been evaluated on ARC-AGI's private leaderboard. Costs for this system include training from scratch and generating solutions, while reported costs for other models generally reflect inference only.

What This Experiment Suggests About Smaller, Specialized AI Models 

The most interesting part of this result is not where the system would appear on a leaderboard. It is what the experiment suggests about how AI systems should be built. A frontier language model is designed to do almost everything: it can write code, answer questions, analyze documents, interpret images, and move between thousands of domains. That breadth is enormously valuable, but it comes at a cost. Applying the same model indiscriminately to every problem can be inefficient when the task is narrow, structured, and repeated.

The model used in this work is many orders of magnitude smaller because it does not need to know everything. Its vocabulary, spatial representation, training objective, and augmentation strategy all reflect the structure of ARC-AGI. That specialization allows it to perform in a range that would otherwise appear to require a vastly larger reasoning model.

This points toward an alternative to building every AI system around one monolithic model. A more efficient architecture could combine a broad, general-purpose model with a network of smaller specialists, each designed for a particular kind of data or reasoning. The general model would provide flexibility and context, while the specialized systems could provide speed, precision, and lower cost where a problem has enough structure to take advantage of them.

The experiment also demonstrates why representation and expert knowledge matter. The transformer did not improve because it received longer written instructions or more time to reason. It improved because its representation preserved the grid’s spatial structure and aligned the learning process with the problem. Enterprise data presents the same design challenge. Transactions, images, policies, logs, and sensor readings all contain relationships that can disappear when everything is flattened into generic text. Exposing the right structure can reduce how much data and compute a system needs.

Finally, ARC-AGI makes the differences between human and machine reasoning unusually visible. The model can apply exact geometric transformations across hundreds of cells with a consistency that would be tedious for a person. Yet it can also select a shortcut rule that fits every demonstration but misses the more intuitive and general rule a person would infer. Tasks that are difficult for humans may be straightforward for the model, while rules that require an unconventional shift in perspective may remain difficult for the AI.

Those differences suggest a useful direction for human-AI collaboration. Machines can search for and execute complex procedures with speed and precision. People can contribute context, strong prior assumptions, and the ability to reframe a problem when the obvious interpretation fails. The goal is not to force both to reason in exactly the same way, but to design systems that use the strengths of each.

Candidate Selection Is the Next Opportunity for Improvement 

The model’s mistakes reveal a clear path forward. With a generation budget of 48 candidates per test input, the system produces the correct output somewhere in its candidate pool for approximately 83% of ARC-AGI 1 tasks. It ultimately places a correct answer in its top two for approximately 76%. For the ensemble, it’s approximately 91% to 80%.

That substantial gap means the model is already generating solutions that its current selection process cannot reliably recognize. A better method for ranking candidates could improve the final score without changing the underlying model. The system also has room to improve on tasks that require global planning, counting, or preserving the exact shape of an object as it moves across a grid.

Because the model can be trained and evaluated in hours, or minutes with the faster recipe, new representations, training objectives, and selection methods can be tested quickly. One possible next step is to place this fast experimental loop inside an automated research system that proposes changes, runs experiments, and uses the results to improve the next version.

That makes the system valuable as more than a single benchmark result. It creates a practical testbed for improving abstract reasoning without the time and infrastructure required to retrain a frontier model. The result is an early example of what becomes possible when models are designed around the structure of the problem rather than relying on scale alone.

Frequently Asked Questions