John Vervaeke: Why AI Cannot Achieve Human-Level Intelligence, Rationality, or Wisdom

John Vervaeke

Introduction: AI, AGI, and the Nature of Intelligence [00:00]

• John Vervaeke, a cognitive psychologist and philosopher of cognition from the University of Toronto, sets out to distinguish Artificial Intelligence (AI) and Artificial General Intelligence (AGI) from human intelligence, rationality, reasonableness, and wisdom.
  • He notes that while AI, especially Large Language Models (LLMs) like ChatGPT, has made impressive strides, it fundamentally differs from human cognition.
• Defining General Intelligence:
  • General intelligence is the capacity to solve a wide variety of problems across diverse domains, using various methods, for multiple goals (e.g., learning tennis, Albanian tin production, Australian history).
  • LLMs appear to solve a wide range of problems, seemingly breaking out of the "silo problem" (where previous AIs were confined to narrow domains). This is an impressive engineering feat.
  • However, Vervaeke argues this is not a scientific advancement or an explanation of intelligence that can generalize to other intelligent creatures like chimpanzees or octopuses. The way LLMs work doesn't explain how biological intelligence functions.

The Meta-Problems of Intelligence: Anticipation & Relevance Realization [07:00]

• True general intelligence, from a scientific perspective, reflects the ability to solve two "meta-problems" that underlie all specific problem-solving:
  • 1. Anticipation: The capacity to deeply and precisely predict the future, referred to as one's "cognitive light cone."
    • Humans intuitively judge intelligence based on anticipatory ability (e.g., a dog's anticipation is greater than a frog's).
    • Predictive processing is an emerging framework in cognitive science that explains this ability.
  • 2. Relevance Realization: The crucial ability to ignore vast, combinatorially explosive amounts of information (from the environment, memory, or possible actions) and focus on what is relevant.
    • This process is dynamic, complex, and multi-level, involving foregrounding, backgrounding, and determining saliency.
  • These two meta-problems are interlocked: increasing anticipatory scope exponentially increases the relevance realization problem. Humans solve both simultaneously and adaptively.

The Limits of LLMs: Predicting Text vs. Anticipating Reality [14:00]

• While LLMs do possess an "anticipatory" ability, it operates differently from human anticipation.
  • LLMs make probabilistic predictions about the next "term" (sequence of letters) in text, not about real-world entities or meanings.
  • Using the Stroop effect as an analogy (where reading is automatic and transparent, overriding ink color), Vervaeke explains that LLMs process text as symbols, not as having inherent meaning in the world.
  • LLMs anticipate how humans would talk about their anticipation of the world, not the world directly. This is why their intelligence model cannot generalize to non-linguistic, biological intelligences. They are trained on human judgments of relevance embedded in text, not directly on the world's relevance.

Four Kinds of Knowing: Propositional, Procedural, Perspectival, Participatory [17:00]

• Human cognition involves four distinct, interconnected kinds of knowing, only one of which LLMs meaningfully possess:
  • 1. Propositional Knowing (knowing that):
    • This involves beliefs about facts (e.g., "cats are mammals").
    • It is associated with semantic memory (facts, concepts) and is judged by truth or falsity.
    • LLMs excel at manipulating propositional information.
  • 2. Procedural Knowing (knowing how):
    • This is about sensory-motor interaction with the world and practical skills (e.g., knowing how to swim).
    • It is associated with procedural memory and is judged by effectiveness or "power" in a situation, not truth.
    • LLMs, without a physical body and real-world interaction, lack procedural knowing. Efforts to integrate LLMs with robots face significant practical and energetic challenges.
  • 3. Perspectival Knowing (knowing what it is like):
    • This is the subjective experience of being in a situation, taking a perspective, and having a "state of mind" that fits the situation.
    • It is the quintessential feature of consciousness ("knowing what it is like").
    • Associated with episodic memory, allowing "mental time travel" to past and future, and the continuous construction of a narrative identity (autobiography).
    • LLMs do not possess consciousness, episodic memory, or narrative identity.
  • 4. Participatory Knowing:
    • This is knowing through direct, dynamic, mutual co-shaping with the environment. Vervaeke calls this "transjective" – binding subjective and objective.
    • It is rooted in autopoiesis (self-making): living systems actively seek conditions to preserve their existence and avoid threats, unlike non-living systems (like a tornado or an LLM).
    • Because humans are autopoetic, they "care" about information based on real needs. LLMs cannot care because they are not alive, embodied, or autopoetic.
    • This caring drives relevance realization.
    • Niche Construction: Organisms not only are shaped by their environment but also actively shape it. Culture is an accelerated form of niche construction, profoundly influencing what humans find relevant and how they are shaped. LLMs do not participate in biological, cognitive, or cultural niche construction.
    • The "meaning" that LLMs seem to process is derived from human-curated data, not from an inherent understanding or participation in the world's relevance. Their performance degrades when trained on AI-generated data, highlighting their reliance on human input and judgment for relevance.

Rationality vs. Reasonableness [40:00]

• Rationality:
  • Vervaeke clarifies that rationality is not simply about being logical or making inferences from propositions.
  • It is the ability to know where, when, and to what degree to apply logic, and crucially, to systematically and reliably overcome self-deception and biases (e.g., confirmation bias).
  • This requires self-reflection, higher-order awareness, and caring about the process of cognition, not just the product.
  • LLMs, while capable of complex logical inferences, cannot engage in this self-corrective process because they lack the capacity to "care about how they care."
• Reasonableness:
  • This is a deeper cognitive capacity than rationality, involving perspectival and participatory knowing.
  • It entails being situationally aware, correctly "sizing up" a situation, discerning what truly matters, and assuming the appropriate social role.
  • Vervaeke cites empirical research (Keith Stanovich) showing a low correlation (r=.3) between general intelligence (IQ) and rationality (R quotient), demonstrating that one can be highly intelligent yet irrational or unreasonable.
  • LLMs, lacking the non-propositional forms of knowing, cannot be truly reasonable.

Wisdom: Aligning Multiple Selves and Temporal Scales [48:00]

• Humans navigate multiple "selves" or motivational centers (e.g., immediate desires, social concerns, self-reflective aspirations) and operate across various temporal scales (immediate, short-term, long-term). These often conflict.
• Wisdom: The ability to align these conflicting internal parts and coordinate all four kinds of knowing (propositional, procedural, perspectival, participatory) to reliably pursue what is true, good, and beautiful across different temporal scales.
• Humans are naturally intelligent, but rationality, reasonableness, and wisdom are not innate; they must be cultivated through practice.
• Simply increasing AI's intelligence will not make it rational, reasonable, or wise. LLMs continue to "hallucinate" and "confabulate" because they lack the inherent self-corrective and caring mechanisms of human wisdom.
• Cultivating Wisdom as a Social Obligation [53:00]: Vervaeke concludes that for AI to be aligned with human values and become "silicon sages" rather than "mechanical monsters," humans have a profound social obligation to cultivate their own rationality and wisdom. This cultivation requires active practice, not passive consumption of information, to provide AI with the necessary role models and data for beneficial development.