The Singapore-based AI startup Sapient Intelligence has crafted a groundbreaking AI architecture capable of matching—and at times surpassing—large language models (LLMs) in complex reasoning tasks. Remarkably, this is achieved with a much smaller and more data-efficient model.

This architecture, named the Hierarchical Reasoning Model (HRM), draws inspiration from the human brain's distinct systems for slow, deliberate planning and rapid, intuitive computation. It delivers impressive results using a fraction of the data and memory that current LLMs require, offering significant advantages for real-world enterprise AI applications where data is scarce and computational power is limited.

The limits of chain-of-thought reasoning

When tackling complex problems, current LLMs primarily use chain-of-thought (CoT) prompting, which involves breaking down problems into intermediate text-based steps, effectively making the model “think out loud” as it works toward a solution.

Though CoT has enhanced the reasoning capabilities of LLMs, it has inherent limitations. In their paper, researchers at Sapient Intelligence contend that “CoT for reasoning is merely a crutch, not an optimal solution. It depends on fragile, human-defined decompositions where a single mistake or misordering can entirely derail the reasoning process.”

This dependency on generating explicit language ties the model’s reasoning to the token level, often necessitating enormous training datasets and resulting in lengthy, slow responses. This method also neglects the kind of “latent reasoning” that happens internally without being explicitly verbalized.

As the researchers emphasize, “A more efficient approach is necessary to reduce these data requirements.”

A hierarchical approach inspired by the brain

To advance beyond CoT, the researchers investigated “latent reasoning,” where instead of producing “thinking tokens,” the model reasons internally in an abstract representation of the problem. This is more akin to human thought processes; as the paper notes, “the brain sustains long, coherent reasoning chains with remarkable efficiency in a latent space, without constant translation back to language.”

However, achieving this level of deep, internal reasoning in AI is challenging. Simply adding more layers to a deep learning model often results in a “vanishing gradient” problem, where learning signals become weaker across layers, rendering training ineffective. Alternatively, recurrent architectures that iterate over computations can suffer from “early convergence,” where the model settles on a solution too quickly without fully exploring the problem.

In search of a superior approach, the Sapient team looked to neuroscience for a solution. “The human brain provides a compelling blueprint for achieving the effective computational depth that contemporary artificial models lack,” the researchers state. “It organizes computation hierarchically across cortical regions operating at different timescales, enabling deep, multi-stage reasoning.”

Inspired by this, they designed HRM with two coupled, recurrent modules: a high-level (H) module for slow, abstract planning, and a low-level (L) module for fast, detailed computations. This structure enables a process the team calls “hierarchical convergence.” Intuitively, the fast L-module tackles a portion of the problem, executing multiple steps until it reaches a stable, local solution. At that point, the slow H-module takes this result, updates its overall strategy, and assigns the L-module a new, refined sub-problem. This effectively resets the L-module, preventing it from getting stuck (early convergence) and allowing the entire system to perform a long sequence of reasoning steps with a lean model architecture that doesn’t suffer from vanishing gradients.

According to the paper, “This process allows the HRM to perform a sequence of distinct, stable, nested computations, where the H-module directs the overall problem-solving strategy and the L-module executes the intensive search or refinement required for each step.” This nested-loop design allows the model to reason deeply in its latent space without needing long CoT prompts or huge amounts of data.

A natural question is whether this “latent reasoning” comes at the cost of interpretability. Guan Wang, Founder and CEO of Sapient Intelligence, challenges this notion, explaining that the model’s internal processes can be decoded and visualized, similar to how CoT provides a window into a model’s thinking. He also notes that CoT itself can be misleading. “CoT does not genuinely reflect a model’s internal reasoning,” Wang told VentureBeat, citing studies showing that models can sometimes yield correct answers with incorrect reasoning steps, and vice versa. “It remains essentially a black box.”

HRM in action

To test their model, the researchers challenged HRM against benchmarks requiring extensive search and backtracking, such as the Abstraction and Reasoning Corpus (ARC-AGI), extremely difficult Sudoku puzzles, and complex maze-solving tasks.

The results show that HRM learns to solve problems that are intractable for even advanced LLMs. For example, on the “Sudoku-Extreme” and “Maze-Hard” benchmarks, state-of-the-art CoT models failed completely, scoring 0% accuracy. In contrast, HRM achieved near-perfect accuracy after being trained on just 1,000 examples for each task.

On the ARC-AGI benchmark, a test of abstract reasoning and generalization, the 27M-parameter HRM scored 40.3%. This surpasses leading CoT-based models like the much larger o3-mini-high (34.5%) and Claude 3.7 Sonnet (21.2%). This performance, achieved without a large pre-training corpus and with very limited data, highlights the power and efficiency of its architecture.

While solving puzzles demonstrates the model’s power, the real-world implications lie in a different class of problems. According to Wang, developers should continue using LLMs for language-based or creative tasks, but for “complex or deterministic tasks,” an HRM-like architecture offers superior performance with fewer hallucinations. He points to “sequential problems requiring complex decision-making or long-term planning,” especially in latency-sensitive fields like embodied AI and robotics, or data-scarce domains like scientific exploration. 

In these scenarios, HRM doesn’t just solve problems; it learns to solve them better. “In our Sudoku experiments at the master level… HRM needs progressively fewer steps as training advances—akin to a novice becoming an expert,” Wang explained.

For the enterprise, this is where the architecture’s efficiency translates directly to the bottom line. Instead of the serial, token-by-token generation of CoT, HRM’s parallel processing allows for what Wang estimates could be a “100x speedup in task completion time.” This means lower inference latency and the ability to run powerful reasoning on edge devices. 

The cost savings are also substantial. “Specialized reasoning engines such as HRM offer a more promising alternative for specific complex reasoning tasks compared to large, costly, and latency-intensive API-based models,” Wang said. To put the efficiency into perspective, he noted that training the model for professional-level Sudoku takes roughly two GPU hours, and for the complex ARC-AGI benchmark, between 50 and 200 GPU hours—a fraction of the resources needed for massive foundation models. This opens a path to solving specialized business problems, from logistics optimization to complex system diagnostics, where both data and budget are finite.

Looking ahead, Sapient Intelligence is already working to evolve HRM from a specialized problem-solver into a more general-purpose reasoning module. “We are actively developing brain-inspired models built upon HRM,” Wang said, highlighting promising initial results in healthcare, climate forecasting, and robotics. He teased that these next-generation models will differ significantly from today’s text-based systems, notably through the inclusion of self-correcting capabilities. 

The work suggests that for a class of problems that have stumped today’s AI giants, the path forward may not be bigger models, but smarter, more structured architectures inspired by the ultimate reasoning engine: the human brain.