Thinking fast and slow in AI: The role of metacognition (2021)
A 2021 arXiv paper proposes a multi-agent AI architecture combining fast reactive agents and slow deliberative reasoning agents based on Kahneman's cognitive theory.
summary is enough While conceptually interesting as a precursor to modern test-time compute, the paper predates current LLM-based reasoning paradigms and offers high-level architecture rather than implementation details.
summary
Published in October 2021, this paper addresses narrow AI limitations by proposing a multi-agent framework inspired by Daniel Kahneman's theory of human cognition. It splits tasks between System 1 fast reactive agents using past experience and System 2 slow deliberative agents for complex search and reasoning. Both agent types rely on a shared world model and a model of self for metacognitive monitoring. Commenters note its historical interest given how modern models have since implemented reasoning capabilities through test-time compute and adaptive scaling.
key points
- Proposes dividing AI tasks between reactive System 1 agents and deliberate System 2 search agents.
- Incorporates a model of self and world knowledge to enable metacognition and better action selection.
- Addresses the limitations of narrow AI and pure data scaling by mimicking human dual-process theory.
who should read it
AI researchers and architects interested in cognitive architectures and the historical evolution of reasoning models.
discussion mixed
consensus
Commenters note that while this 2021 paper anticipated modern thinking fast and slow paradigms, current frontier models have implemented similar concepts via test-time compute and internal reasoning tokens rather than explicit multi-agent architectures.
pushback
Some users question whether Kahneman's dual-process theory is robust psychology or pop-science, arguing that applying cognitive boxes to LLMs is fundamentally flawed since they lack dynamic internal updating mechanisms.
Most humans have weak meta-cognition, a large percentage doesn't have verbal thoughts.
Meta-cognition makes sense in a dynamic and updatable and modular system, for example I can monitor thoughts coming from my amygdala with my prefrontal cortex and then adjust how I process these thoughts.
In LLMs it makes zero sense, even if you feed the output of one model into another, there is no way they can update the heuristics behind how those were computed.
It explains why applying metacognition to LLMs fails conceptually because the models lack a way to dynamically update the underlying heuristics of how outputs are computed.
Generated by an LLM from the article and thread. Check the sources before relying on it.