## Tags - Part of: [[Artificial Intelligence]], [[Intelligence]] - Related: - Includes: - Additional: ## Technical summaries - Artificial general intelligence (AGI) is a theoretical type of [[artificial intelligence]] (AI) that falls within the lower and upper limits of human cognitive capabilities across a wide range of cognitive tasks. This contrasts with narrow AI, which is limited to specific tasks. Artificial [[superintelligence]] (ASI), refers to types of intelligence that range from being only marginally smarter than the upper limits of human intelligence to greatly exceeding human cognitive capabilities by orders of magnitude. AGI is considered one of the definitions of strong AI. - AGI, intelligence may be comparable to, match, differ from, or even appear alien-like relative to [[biological intelligence|human intelligence]], encompassing a spectrum of possible cognitive architectures and capabilities that includes the spectrum of human-level [[intelligence]]. ## Main resources - [Artificial general intelligence - Wikipedia](https://en.wikipedia.org/wiki/Artificial_general_intelligence) <iframe src="https://en.wikipedia.org/wiki/Artificial_general_intelligence" allow="fullscreen" allowfullscreen="" style="height:100%;width:100%; aspect-ratio: 16 / 5; "></iframe> - [Alan’s conservative countdown to AGI – Dr Alan D. Thompson – LifeArchitect.ai](https://lifearchitect.ai/agi/) ## Definitions - [\[2311.02462\] Levels of AGI for Operationalizing Progress on the Path to AGI](https://arxiv.org/abs/2311.02462) 1. AGI according to OpenAI's charter: "highly autonomous systems that outperform humans at most economically valuable work", In practice most people currently deviate from the above definition to only mean digital work [- Karpathy on X](https://x.com/karpathy/status/1834641096905048165) 2. AGI (general definition): "Artificial General Intelligence (AGI) is an important and sometimes controversial concept in computing research, used to describe an AI system that is at least as capable as a human at most tasks." 3. AGI according to Legg and Goertzel: "a machine that is able to do the cognitive tasks that people can typically do" 4. AGI according to Shanahan: "artificial intelligence that is not specialized to carry out specific tasks, but can learn to perform as broad a range of tasks as a human" 5. AGI according to Marcus: "shorthand for any intelligence (there might be many) that is flexible and general, with resourcefulness and reliability comparable to (or beyond) human intelligence" 6. AGI according to Aguera y Arcas and Norvig: They suggest that state-of-the-art language models already are AGIs, arguing that generality is the key property of AGI, and that because language models can discuss a wide range of topics, execute a wide range of tasks, handle multimodal inputs and outputs, operate in multiple languages, and "learn" from zero-shot or few-shot examples, they have achieved sufficient generality. 7. Competent AGI (Level 2 in the paper's taxonomy): A system that has "at least 50th percentile of skilled adults" performance on a wide range of non-physical tasks, including metacognitive tasks. 8. ASI (Artificial Superintelligence): Defined in the paper's taxonomy as "Level 5: Superhuman" - a system that "outperforms 100% of humans" on a wide range of non-physical tasks, including metacognitive tasks. [[Images/6ec7f6eb8791ef53417805032132fc4f_MD5.jpeg|Open: Pasted image 20240919014547.png]] (each system is a discrete points on these spectrums) ![[Images/6ec7f6eb8791ef53417805032132fc4f_MD5.jpeg]] - [OpenAI Sets Levels to Track Progress Toward Superintelligent AI - Bloomberg](https://www.bloomberg.com/news/articles/2024-07-11/openai-sets-levels-to-track-progress-toward-superintelligent-ai?srnd=technology-vp) [[Images/238d28e026fa58e7787b1e630bb3732e_MD5.jpeg|Open: Pasted image 20240919015646.png]] ![[Images/238d28e026fa58e7787b1e630bb3732e_MD5.jpeg]] - [9 definitions of Artificial General Intelligence (AGI) and why they are flawed by Carlos E. Perez](https://x.com/IntuitMachine/status/1721845203030470956) 1. The Turing Test Flaw: LLMs already passed turing tests, but you had to prompt engineer them to make them dumber, not as knowledgable, not as verbose, more causal, broken grammar etc. for nonnerds Flaw: Focuses on fooling humans rather than intelligence, easy to game by producing human-like text without intelligence. 2. Strong AI - Consciousness Limitation: No agreement on measuring machine consciousness. Focus on vague concepts rather than capabilities. 3. Human Brain Analogy Limitation: While loosely inspired by the brain, successful AI need not strictly mimic biology. Overly constrains mechanisms. 4. Human Cognitive Task Performance Limitation: What tasks? Which people? Lacks specificity and measurement. 5. Ability to Learn Tasks Strength: Identifies learning as important AGI ability. Limitation: Still lacks concrete measurement. 6. Economically Valuable Work Limitation: Misses non-economic values of intelligence like creativity. Requires deployment. 7. Flexible & General - Coffee Test Strength: Concrete example tasks. Limitation: Proposed tasks may not fully define AGI. 8. Artificial Capable Intelligence Strength: Emphasizes complex, multi-step real-world tasks. Limitation: Focuses narrowly on profitability. 9. LLMs as Generalists Limitation: Lacks performance criteria - generality alone insufficient. An AGI definition based on 6 principles 1. Focus on capabilities, not processes Avoid requiring things like human-like thinking or consciousness which are vague, controversial concepts. Focus just on demonstrated abilities. An AI that passes the Turing Test by generating human-like text may not actually "think" like a human. 2. Focus on generality and performance True intelligence requires both breadth of abilities (generality) and level of skill (performance). An AI that achieves human-level performance playing chess has high performance on a narrow task. 3. Focus on cognitive and metacognitive tasks Physical capabilities like robotics seem less central to intelligence than mental capabilities. But learning is important. Example: An AI that can learn to carry out new tasks demonstrates an important cognitive ability. 4. Focus on potential, not deployment Don't require real-world use, just demonstrate capabilities under testing conditions. This avoids non-technical hurdles. Example: Waymo AI drives cars autonomously but isn't widely deployed due to legal issues. The capability still exists. 5. Focus on ecological validity Choose benchmark tasks that actually represent skills humans value, not just easy to measure skills. Example: Hold a natural conversation reflects general linguistic intelligence better than optimized dialogue. 6. Focus on the path to AGI, not a single endpoint Motivation: A leveled taxonomy allows more nuanced discussion of progress and risks vs treating AGI as a threshold. - [\[2406.04268\] Open-Endedness is Essential for Artificial Superhuman Intelligence](https://arxiv.org/abs/2406.04268): Open-Endedness: Continuous self-improvement, creativity, and the generation of novel solutions beyond human imagination or ability. ## Idealizations - [[Intelligence#Idealizations]] - ![[Intelligence#Idealizations]] ## Other definitions of intelligence - [[Intelligence#Definitions]] - ![[Intelligence#Definitions]] ## Future - [[Computronium]] - From [The Singularity Is Nearer - Wikipedia](https://en.wikipedia.org/wiki/The_Singularity_Is_Nearer) by [[Ray Kurzweil]]: [[Images/4ee554bf075eb3a5879c61c1d14e1e51_MD5.jpeg|Open: Pasted image 20240919001041.png]] ![[Images/4ee554bf075eb3a5879c61c1d14e1e51_MD5.jpeg]] ## Brainstorming The real AGI benchmark is if the model can come up with general relativity if he knew everything that we knew right before discovering general relativity One potential dream AGI system for scientists is physics based AIs (quantum, thermodynamic, deterministic, hybrids) optimized for perfect modeling of nature (similar to how nature is governed quantum/thermodynamically/deterministically/hybridly on different scales) coupled with anthropomorphic humanlike synthetic agent scientist AI that could use that physics based AI optimally and translate the results into more humanlike language for humans via a more humanlike interface. I want an AGI system that can very deeply grok etc. coherent nonbrittle circuits representing classical mechanics, general relativity, quantum mechanics, standard model, loop quantum gravity, string theory, etc. and derive new physics that potentially actually has a higher probability of being more empirically predictive, operating under mechanisms similar to whatever happened in Newton's, Einstein's and Schrodinger's brain when they came up with their paradigm shifting models of physical reality. Artificial general intelligence, AGI. Most of the mainstream sees it as AI that has human-like cognitive abilities. I prefer to see it as AI that is able to generalize better regardless of how a person is able to generalize and what other cognitive abilities human has, which I think makes more sense given the name. I would rather call the first one artificial human intelligence. And instead of "artificial" I would use machine/digital/silicon intelligence, because it is not an intelligence that is "artificial" in my opinion, but what is on a different substrate with different and variously similar mechanisms. " I have a lot of issues with the term "AGI". I would redefine it. People say that we're heading towards artificial general intelligence (AGI), but by that most people actually usually mean machine human-level intelligence (MHI) instead, a machine that is performing human digital or/and physical tasks as good as humans. And by artificial superintelligence (ASI), people mean machine superhuman intelligence (MSHI), that is even better than humans at human tasks. I think lot's of research goes towards very specialized machine narrow intelligences (MNI), which are very specialized and often superhuman in very specific tasks, such as playing games (AlphaZero), protein folding (AlphaFold), and a lot of research also goes towards machine general intelligence (MGI), which will be much more general than human intelligence (HI), because humans are IMO very specialized biological systems in our evolutionary niche, in our everyday tasks and mathematical abilities, and other organisms are differently specialized, even tho we still share a lot. Plus there is just some overlap between biological and machine intelligence. And I wonder how if the emerging reasoning systems like o3 are becoming actually more similar to humans, or more alien compared to humans, as they might better adapt to novelty and be more general than previous AI systems, which might bring them closer to humans, but in slightly different ways than humans. They may be able to do selfcorrecting chain of thought search endlessly, which is better for a lot of tasks, and big part of this is big part of human cognition I think, but humans still work differently. I think that generality of an intelligent system is a spectrum, and each system has differently general capabilities over different families of tasks than other ones, which we can see with all the current machine and biological intelligences, that are all differently general over different families of tasks. That's why "AGI" feels much more continuous than discrete to me, and over which families of tasks you generalize matters too I think. The Chollet's definition of intelligence as the efficiency with which you operationalize past information in order to deal with the future, which can be interpreted as a conversion ratio, is really good I think, and his ARC-AGI benchmark, that tries to test for some degree of generality, trying to test for the ability to abstract over and recombine some atomic core knowledge priors, to prevent naive pattern memorization and retrieval being successful. And I really wonder if scoring well on ARC-AGI actually generalizes outside the ARC domain to all sorts of tasks where humans are superior, or where humans are terrible but machines are superior, or where other biological systems are superior, or where everyone is terrible for now. I would suspect so, but maybe not? In software engineering, o1 seems to be better just sometimes? What's happening there? I want more benchmarks! Pre-o1 LLMs are technically super surface level knowledge generalists, lacking technical depth, but having bigger overview of the whole internet than any human, knowing high level correlations of the whole internet, even tho their representations are more brittle than human brain's. But we're much better in agency, in some cases in generality, we can still do more abstract math more, etc., we're better in our evolutionary niche. But for example AlphaZero destroyed us in chess. But when I look at ARC-AGI scores, I see o3 as a system that can adapt to novelty better than previous models, but we can still do much better. Also according to some old definitions of AGI, existing AI systems have been AGI for a long time, because it can have a general discussion about basically almost anything (except lacking narrow niche field specific knowledge and skills, lack of agency, lack of adapting to novelty like humans, etc.). Or if we take the AIXI definition of AGI, then a fully general AGI is impossible in practice, as that's not computable, and you can only approximate it, since AIXI it considers all possible explanations (programs) for its observations and past actions and chooses actions that maximize expected future rewards across all these explanations, weighted by their simplicity (shortness) (Occam's razor) (Kolmogorov complexity). AI will model the world in ways completely incomprehensible to how humans model the world. And it will do it in much more optimal ways, it will grok physics much more optimally, in such alien ways compared to how human brains evolved to do it in our evolutionary environment. The space of all possible modelling systems is so vast, and us, and nature, have only scratched the surface so far. The current architectures are just the beginning of all of this: Deep learning models, transformer models, diffusion models, RL CoT models, neurosymbolics with MCTS (AlphaZero), statistical models, etc. And AIXI people argue that humans and AI systems try to approximate AIXI in their more narrow domains and take all sorts of cognitive shortcuts to be actually practical and not take infinite time and resources to decide. And soon we might create some machine-biology hybrids as well. Then we should maybe start calling it carbon based intelligence (CI) and silicon based intelligence (SI) and carbon and silicon based intelligences (CSI). I also guess it depends how you define the original words, such as generality. Let's say you are comparing the generality of AlphaZero, Claude, o1/o3, and humans. How would you compare them? Do all have zero generality, if we take the AIXI definiton of AGI for example, which is not computable? AIXI definition of AGI would also imply that there is no AGI in our current universe and there can never be. “ i believe that eventually any cognitive and physical process a human can do, a machine will eventually be able to do as well at some point in the future, but how long will that take, i have no ide My ideal scifi would be about benevolent superintelligence that cures all diseases, makes all beings happy, figures out how biology, fundamental physics, consciousness, intelligence, etc. works by countless scientific breakthroughs, understands all math, understands everything in philosophy, creates post-scarcity abundance for all, creates infinitely fascinating complex art, and in the process grows infinitely more and more in intelligence and creativity, maximizes morphological freedom, and does no harm Benevolent superintelligence explosion [[Artificial intelligence x Science]] Yeah its a bit unrealistic superutopia that I like dreaming about, so that's why it's science fiction. My current biggest fear in the real world is tech companies centralizing too much power for themselves via AI and other technology and other means (economic, political,...), so that's partially why I want open source to win and try to support it, while trying to reverse engineer the moat of tech companies. To democratize the power. The issue with AI safety community I started to have is that a big part of them basically want something like government surveillance on GPUs and training runs to prevent unsafe AI, which can so much easily turn into surveillance dystopia and destroy open source completely, plus big tech is merging with government as well to have the least restrictions for themselves while wanting to restricting others including open source. It feels like that will make power dynamics even more concetrated instead. A lot of luddites also joined the AI safety movement I think when I look at the current world and at history, then a lot of times when there was too much concentration of power in any form to some centralized entity, then it started killing freedom for everyone else. And I view AI as technology that has the potential to give the ultimate power, centralized power if its in the hands of few, or decentralized power fi tis in the hands of people. I also started to not really believe in the assumption that increasing intelligence automatically leads to rogueness. I think intelligence is independent of that, and also independent of power seeking. For example we have galaxy brain scientists that are not at all rogue or power seeking. It depends so much. and they are controlled by IMO less intelligent managers and politicians. My favorite definitions of intelligence include stuff like modelling capability, predictive capability, generalization capability, etc., about some data, which are decoupled from agency and goals in changing the world to me. ## Written by AI (may include hallucinated factually incorrect information) # The definitive map of Artificial General Intelligence **AGI remains one of the most contested concepts in technology—there is no single agreed-upon definition.** Across researchers, organizations, and philosophical traditions, at least 30 distinct definitions compete, ranging from mathematical formalisms (Legg & Hutter's universal intelligence) to economic criteria (OpenAI's "most economically valuable work") to skill-acquisition frameworks (Chollet's efficiency-based measure). This report catalogs every major definition, taxonomy, related subfield, philosophical concept, and benchmark associated with AGI, providing a one-sentence explanation and source for each. It covers **180+ distinct items** across six categories and represents the most comprehensive publicly available reference map of the AGI landscape. --- ## 1. Major definitions of AGI The definitions below cluster around several recurring themes: **task/economic framing** (can it do human jobs?), **cognitive equivalence** (does it match human minds across domains?), **learning efficiency** (can it acquire new skills rapidly?), and **mathematical formalism** (can intelligence be measured universally?). Notable disagreements persist on whether physical embodiment is required, whether economic output is a valid metric, and whether current LLMs constitute "emerging AGI." ### Foundational and historical definitions **Alan Turing (1950)** — Proposed the "imitation game" (Turing Test) as a practical criterion: a machine can be said to "think" if a human interrogator cannot reliably distinguish it from a human in text-based conversation, offering an operational test rather than a formal definition of intelligence. Source: https://plato.stanford.edu/entries/turing-test/ **John McCarthy (1956/2007)** — Coined "artificial intelligence" at the 1956 Dartmouth conference based on the conjecture that "every aspect of learning or any other feature of intelligence can be so precisely described that a machine can be made to simulate it." Source: https://en.wikipedia.org/wiki/Artificial_general_intelligence **Herbert Simon (1965)** — Predicted that "machines will be capable, within twenty years, of doing any work a man can do," framing intelligence in practical, task-oriented terms. Source: https://en.wikipedia.org/wiki/Artificial_general_intelligence **Mark Gubrud (1997)** — Authored the **first known use of "artificial general intelligence,"** characterizing AGI as "AI systems that rival or surpass the human brain in complexity and speed, that can acquire, manipulate and reason with general knowledge, and that are usable in essentially any phase of industrial or military operations where a human intelligence would otherwise be needed." Source: https://metr.org/agi.pdf ### Individual researcher definitions **Shane Legg (2008)** — Defined intelligence as "an agent's ability to achieve goals in a wide range of environments," formalized mathematically in his PhD thesis; in 2024 he endorsed DeepMind's "Competent AGI" level as the sensible definition. Source: https://arxiv.org/html/2311.02462v4 **Shane Legg & Marcus Hutter (2007)** — Proposed a formal mathematical definition where intelligence is measured as an agent's expected performance across all computable reward environments, weighted by their algorithmic complexity (Kolmogorov complexity). Source: https://arxiv.org/abs/0712.3329 **Ben Goertzel (2007/2014)** — Defined AGI as "the creation and study of software or hardware systems with general intelligence comparable to, and ultimately perhaps greater than, that of human beings," emphasizing autonomous, self-reflective, self-improving, commonsensical intelligence. Source: https://www.researchgate.net/publication/271390398 **François Chollet (2019)** — Proposed that "the intelligence of a system is a measure of its skill-acquisition efficiency over a scope of tasks, with respect to priors, experience, and generalization difficulty," explicitly rejecting economic-value framings as incorrect measures of intelligence. Source: https://arxiv.org/abs/1911.01547 **Gary Marcus & Ernest Davis (2019)** — Defined AGI as "any intelligence that is flexible and general, with resourcefulness and reliability comparable to (or beyond) human intelligence," emphasizing that true AI must possess **common sense and deep understanding**, not just broad but shallow pattern matching. Source: https://garymarcus.substack.com/p/agi-versus-broad-shallow-intelligence **Nils Nilsson (2005)** — Argued human-level AI means "most of the tasks that humans perform for pay could be automated" via "general-purpose, educable systems that can learn and be taught to perform any of the thousands of jobs that humans can perform." Source: https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/1850 **Stuart Russell (2019)** — Described the ultimate AI goal as discovering a "general-purpose method that is applicable across all problem types and works effectively for large and difficult instances," framing the central challenge as building machines that are provably beneficial and uncertain about human objectives. Source: https://people.eecs.berkeley.edu/~russell/papers/mi19book-hcai.pdf **Peter Norvig & Blase Agüera y Arcas (2023)** — Argued that state-of-the-art LLMs already qualify as AGIs because generality is AGI's key property, and these models handle a wide range of topics and tasks, mapping to "Emerging AGI" in the DeepMind framework. Source: https://arxiv.org/html/2311.02462v4 **Ray Kurzweil (2005/2024)** — Defined AGI as "attaining the highest human level in all fields of knowledge"—AI that can match or exceed the best expert humans in every domain and pass a properly administered Turing Test, predicting AGI by **2029**. Source: https://en.wikipedia.org/wiki/The_Singularity_Is_Near **Demis Hassabis** — Described AGI as "a system that's capable of exhibiting all the cognitive capabilities humans can," including reasoning, creativity, planning, long-term memory, and continuous learning, emphasizing that "learning is synonymous with intelligence." Source: https://time.com/7280740/demis-hassabis-interview/ **Sam Altman** — Uses OpenAI's charter definition ("highly autonomous systems that outperform humans at most economically valuable work") but also described AGI as "a system that can tackle increasingly complex problems, at human level, in many fields." Source: https://openai.com/index/planning-for-agi-and-beyond/ **Yann LeCun** — Argues "the phrase AGI should be retired and replaced by 'human-level AI'" because "there is no such thing as AGI—even human intelligence is very specialized," requiring world models, planning, and persistent memory that LLMs lack; recently proposed **"Superhuman Adaptable Intelligence" (SAI)** as a replacement. Source: https://time.com/6694432/yann-lecun-meta-ai-interview/ **Yoshua Bengio** — Frames AGI as "generalist and autonomous systems that match or surpass human abilities in most or all knowledge work," with particular focus on the existential risks it poses and the requirement for internal deliberation and planning. Source: https://yoshuabengio.org/2024/07/09/reasoning-through-arguments-against-taking-ai-safety-seriously/ **Geoffrey Hinton** — Views AGI as AI matching or surpassing human general cognitive abilities, warning it could arrive in **5–20 years**; resigned from Google in 2023 to speak freely about existential risks. Source: https://en.wikipedia.org/wiki/Geoffrey_Hinton **Nick Bostrom (2014)** — Defines superintelligence (beyond AGI) as "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest," treating AGI as a stepping stone where machines reach human-level general intelligence. Source: https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies **Marcus Hutter (2000)** — Created AIXI, a mathematical formalism of AGI that maximizes "the ability to satisfy goals in a wide range of environments," representing a theoretically optimal but incomputable general reinforcement learning agent. Source: https://www.hutter1.net/ai/uaibook.htm **Mustafa Suleyman (2023)** — Proposed **"Artificial Capable Intelligence" (ACI)** to describe AI that can accomplish complex, open-ended, multistep tasks in the real world, offering a more practical near-term framing than AGI. Source: https://arxiv.org/abs/2311.02462 **Murray Shanahan (2015)** — Defined AGI as "artificial intelligence that is not specialized to carry out specific tasks, but can learn to perform as broad a range of tasks as a human," notably including metacognitive capabilities (learning to learn). Source: https://arxiv.org/html/2311.02462v4 ### Organizational definitions **OpenAI (2018 Charter)** — "Highly autonomous systems that outperform humans at most economically valuable work"; internally uses a 5-level system (Chatbots → Reasoners → Agents → Innovators → Organizations); the OpenAI-Microsoft financial threshold for AGI is AI generating **at least $100 billion in profits**. Source: https://openai.com/charter/ **Google DeepMind** — Mission is to "solve intelligence, and then use that to solve everything else"; published the "Levels of AGI" framework (Morris et al., 2023) defining five performance tiers crossed with breadth and autonomy dimensions. Source: https://arxiv.org/abs/2311.02462 **Anthropic** — A Public Benefit Corporation focused on "the responsible development and maintenance of advanced AI for the long-term benefit of humanity"; CEO Dario Amodei's definition includes complete digital access, autonomous long-term planning, **10x–100x human information absorption speed**, and deployable as millions of instances. Source: https://www.anthropic.com/company **Meta AI / FAIR** — Pursues "human-level intelligence" (per LeCun) through an open-source approach, believing it requires world models, planning, reasoning, and persistent memory that current LLMs lack. Source: https://time.com/6694432/yann-lecun-meta-ai-interview/ **Microsoft Research (2023)** — In the "Sparks of AGI" paper, used a 1994 psychological consensus definition of intelligence and argued GPT-4 "could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence system." Source: https://arxiv.org/abs/2303.12712 **ARC Prize Foundation** — Defines AGI as "a system that can efficiently acquire new skills and solve open-ended problems," explicitly stating the economic consensus definition is "an incorrect measure of intelligence." Source: https://arcprize.org/arc-agi --- ## 2. Taxonomies and frameworks for classifying AGI These frameworks attempt to impose structure on the spectrum of AI capabilities. They range from simple three-tier models to sophisticated matrixed ontologies. **Google DeepMind's "Levels of AGI" (Morris et al., 2023)** — A matrixed ontology classifying AI along performance depth (Level 0: No AI → Level 1: Emerging → Level 2: Competent → Level 3: Expert → Level 4: Virtuoso → Level 5: Superhuman) crossed with breadth (Narrow vs. General), plus five autonomy levels (Tool → Consultant → Collaborator → Expert → Agent). Source: https://arxiv.org/abs/2311.02462 **Weak AI vs. Strong AI (Searle, 1980)** — A philosophical distinction where weak AI treats computers as tools for studying the mind (simulation), while strong AI claims an appropriately programmed computer literally _is_ a mind with genuine understanding and cognitive states. Source: https://en.wikipedia.org/wiki/Chinese_room **Narrow AI vs. General AI vs. Superintelligence** — The most commonly used three-tier classification: task-specific Narrow AI (all current AI), human-level General AI (hypothetical), and Superintelligence that surpasses all human cognitive abilities. Source: https://en.wikipedia.org/wiki/Artificial_general_intelligence **ANI / AGI / ASI taxonomy** — A formalized acronym-based version of the three-tier spectrum, with "AGI" popularized by Goertzel and Legg (~2007) and "ASI" by Bostrom (2014). Source: https://www.ibm.com/think/topics/artificial-general-intelligence **Bostrom's three forms of superintelligence (2014)** — Decomposes superintelligence into speed superintelligence (same capabilities, much faster), collective superintelligence (many intellects organized together), and quality superintelligence (vastly qualitatively smarter), arguing all three are practically equivalent. Source: https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies **Kurzweil's six epochs of evolution (2005)** — Outlines cosmic evolution from Physics & Chemistry → Biology & DNA → Brains → Technology → Merger of Human Intelligence with Technology (~2045 Singularity) → The Universe Wakes Up. Source: https://en.wikipedia.org/wiki/The_Singularity_Is_Near **OpenAI's five-level AGI framework (2024)** — Internal classification tracking progress: Level 1 (Chatbots) → Level 2 (Reasoners) → Level 3 (Agents) → Level 4 (Innovators) → Level 5 (Organizations, where AI does the work of an entire organization). Source: https://www.axios.com/2024/07/15/openai-chatgpt-reasoning-ai-levels **Anthropic's AI Safety Levels (ASL, 2023)** — A risk-based classification modeled on biosafety levels where escalating AI capabilities trigger escalating safety requirements: ASL-1 (no meaningful risk) through ASL-3 (substantially increased catastrophic risk) to ASL-4+ (not yet fully defined). Source: https://www.anthropic.com/news/anthropics-responsible-scaling-policy **Transformative AI (TAI, Open Philanthropy, 2016)** — Defined as "AI that precipitates a transition comparable to (or more significant than) the agricultural or industrial revolution," deliberately agnostic about whether the AI is conscious, general, or narrow. Source: https://www.openphilanthropy.org/research/some-background-on-our-views-regarding-advanced-artificial-intelligence/ **Human-Level Machine Intelligence (HLMI)** — A standardized concept for expert surveys, defined as when "unaided machines can accomplish every task better and more cheaply than human workers"; 2023 survey forecasts **50% probability by 2047**. Source: https://arxiv.org/abs/1705.08807 **Comprehensive AI Services (CAIS, Eric Drexler, 2019)** — Reframes superintelligent-level AI as a collection of bounded, task-focused services rather than a single autonomous agent, arguing this trajectory is both more likely and safer than unified agent AGI. Source: https://forum.effectivealtruism.org/topics/comprehensive-ai-services **Tool AI vs. Agent AI (Karnofsky/Gwern, 2011–2016)** — Distinguishes AI that computes and presents options for human decision-making ("tool") from AI that autonomously executes actions ("agent"), with Gwern arguing that economic pressures push all tool AIs toward agency. Source: https://gwern.net/tool-ai **Hintze's four functionality types (2016)** — Classifies AI as Reactive Machines → Limited Memory → Theory of Mind → Self-Aware AI, based on cognitive architecture complexity. Source: https://codebots.com/blog/artificial-intelligence/the-3-types-of-ai-is-the-third-even-possible **Suleyman's Artificial Capable Intelligence (2023)** — A practical, near-term framing emphasizing AI's ability to accomplish complex, multi-step real-world tasks, deliberately avoiding the "AGI" label. Source: https://arxiv.org/abs/2311.02462 **SAE J3016 Levels of Driving Automation (2014)** — The six-level (0–5) vehicle automation framework that directly inspired DeepMind's Levels of AGI paper and similar AI autonomy classifications. Source: https://www.sae.org/blog/sae-j3016-update **NIST AI Use Taxonomy (2024)** — A federal standards taxonomy classifying AI systems by the human-AI activities they perform (monitoring, recommendation, personalization, classification, generation) rather than by capability level. Source: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.200-1.pdf --- ## 3. Related subfields and research areas ### Cognitive architectures **SOAR** — A general cognitive architecture developed by John Laird, Allen Newell, and Paul Rosenbloom that integrates knowledge-intensive reasoning, reactive execution, hierarchical reasoning, planning, and learning from experience. Source: https://soar.eecs.umich.edu/ **ACT-R** — A hybrid cognitive architecture by John Robert Anderson at CMU that models human cognition through procedural and declarative memory modules mapped to specific brain regions. Source: http://act-r.psy.cmu.edu/ **OpenCog** — An open-source AGI framework by Ben Goertzel integrating probabilistic logic, evolutionary learning, and attention allocation on a shared hypergraph knowledge store (AtomSpace). Source: https://en.wikipedia.org/wiki/OpenCog **LIDA** — A cognitive architecture by Stan Franklin modeling cognition through iterative cognitive cycles grounded in Global Workspace Theory, incorporating attention, emotion, and multiple learning mechanisms. Source: https://en.wikipedia.org/wiki/LIDA_(cognitive_architecture) **NARS** — A general-purpose intelligent reasoning system by Pei Wang operating under the Assumption of Insufficient Knowledge and Resources, using non-axiomatic logic where all beliefs are revisable. Source: https://cis.temple.edu/~pwang/NARS-Intro.html **CLARION** — A hybrid architecture by Ron Sun integrating implicit (subsymbolic) and explicit (symbolic) knowledge across action-centered, non-action-centered, motivational, and metacognitive subsystems. Source: https://en.wikipedia.org/wiki/CLARION_(cognitive_architecture) **Sigma** — A cognitive architecture by Paul Rosenbloom at USC pursuing grand unification of cognition through graphical models under a single theoretical framework. Source: https://en.wikipedia.org/wiki/Sigma_(cognitive_architecture) **MicroPsi** — A cognitive architecture by Joscha Bach implementing Dietrich Dörner's Psi theory, addressing the interrelation of emotion, motivation, and cognition in situated agents. Source: https://github.com/joschabach/micropsi2 ### Brain-inspired approaches **Whole Brain Emulation (WBE)** — The hypothetical process of scanning a biological brain in sufficient detail to create a computational model replicating all its information processing, potentially preserving consciousness in a digital substrate. Source: https://en.wikipedia.org/wiki/Whole_brain_emulation **Neuromorphic Computing** — Computing inspired by the brain's structure, using artificial neurons and synapses in specialized hardware to perform computations with brain-like energy efficiency and parallel distributed processing. Source: https://en.wikipedia.org/wiki/Neuromorphic_computing **Blue Brain Project** — A Swiss brain research initiative (2005–2024) at EPFL that created biologically detailed digital reconstructions of the mammalian brain using supercomputers, achieving reconstruction of a rat neocortical column with **30,000 neurons**. Source: https://bluebrain.epfl.ch/ **Human Brain Project** — A 10-year EU-funded program (2013–2023) employing ~500 scientists that developed six major platforms including brain simulation, neuromorphic computing, and neurorobotics. Source: https://en.wikipedia.org/wiki/Human_Brain_Project **Connectomics** — The study and comprehensive mapping of neural connections at various scales, from microscale synaptic connections to macroscale fiber tracts, using electron microscopy and diffusion MRI. Source: https://en.wikipedia.org/wiki/Connectomics ### Consciousness and AI **Artificial Consciousness / Machine Consciousness** — The field studying whether and how artificial systems could possess subjective experience, self-awareness, or phenomenal consciousness, drawing on cognitive science and philosophy of mind. Source: https://en.wikipedia.org/wiki/Artificial_consciousness **Integrated Information Theory (IIT) applied to AI** — Giulio Tononi's theory proposing consciousness corresponds to integrated information (Φ), raising questions about whether artificial systems with highly integrated architectures could possess consciousness. Source: https://en.wikipedia.org/wiki/Integrated_information_theory **Global Workspace Theory applied to AI** — Bernard Baars's model of consciousness as a central "workspace" broadcasting information from specialized modules, applied to AI as an architectural blueprint for integrating diverse computational modules. Source: https://en.wikipedia.org/wiki/Global_workspace_theory **Artificial Sentience** — The hypothetical capacity of AI systems to have subjective experiences or feelings, raising ethical questions about the moral status of AI entities. Source: https://en.wikipedia.org/wiki/Artificial_consciousness ### Advanced AI concepts **Artificial Superintelligence (ASI)** — Hypothetical AI possessing intelligence far surpassing the brightest human minds across virtually all domains, including scientific creativity, wisdom, and social skills. Source: https://en.wikipedia.org/wiki/Superintelligence **Recursive Self-Improvement** — A theoretical process in which an AI autonomously rewrites and enhances its own code or architecture, with each iteration potentially accelerating subsequent improvements. Source: https://en.wikipedia.org/wiki/Recursive_self-improvement **Intelligence Explosion** — I.J. Good's 1965 hypothesis that a sufficiently capable AI could enter a positive feedback loop of recursive self-improvement, rapidly producing superintelligence. Source: https://en.wikipedia.org/wiki/Technological_singularity **Seed AI** — A term coined by Eliezer Yudkowsky for a hypothetical initial AI designed with just enough capability to understand and improve its own design, serving as the starting point for recursive self-improvement. Source: https://intelligence.org/ie-faq/ **AI Alignment** — The research problem of ensuring AI systems reliably pursue goals and ethical principles intended by their designers, addressing value specification, scalable oversight, and preventing power-seeking behaviors. Source: https://en.wikipedia.org/wiki/AI_alignment **AI Safety** — An interdisciplinary field focused on preventing accidents, misuse, or harmful consequences from AI systems, encompassing alignment, robustness, monitoring, and governance frameworks. Source: https://en.wikipedia.org/wiki/AI_safety **AI Existential Risk** — The possibility that advanced AI—particularly misaligned superintelligent agents—could threaten humanity's survival, a concern endorsed by hundreds of leading researchers. Source: https://en.wikipedia.org/wiki/Existential_risk_from_artificial_intelligence ### Learning paradigms **Transfer Learning** — A technique where a model trained on one task is repurposed for a different but related task, leveraging previously learned feature representations to reduce data requirements. Source: https://en.wikipedia.org/wiki/Transfer_learning **Meta-Learning (Learning to Learn)** — A paradigm where algorithms improve learning performance across a family of tasks by leveraging experience from prior tasks, optimizing strategies that generalize to new problems. Source: https://en.wikipedia.org/wiki/Meta-learning_(computer_science) **Few-Shot Learning** — An approach where models learn to perform new tasks from only a handful of labeled examples, using meta-learning or transfer strategies to overcome data scarcity. Source: https://www.ibm.com/think/topics/few-shot-learning **Zero-Shot Learning** — Enables models to classify objects or concepts never seen during training by leveraging semantic descriptions or learned relationships between known and unknown classes. Source: https://en.wikipedia.org/wiki/Zero-shot_learning **Multi-Modal AI** — Systems processing and reasoning across multiple input types (text, images, audio, video), enabling richer understanding than single-modality models. Source: https://en.wikipedia.org/wiki/Multimodal_learning **Embodied AI** — Intelligent agents interacting with the physical world through sensory perception and motor actions, grounded in the principle that intelligence arises from the interplay between body, brain, and environment. Source: https://en.wikipedia.org/wiki/Embodied_cognition **Open-Ended Learning** — AI systems that continuously discover and accumulate new skills in unbounded environments without predefined task endpoints, analogous to biological evolution. Source: https://en.wikipedia.org/wiki/Open-ended_evolution **Continual/Lifelong Learning** — A paradigm where models learn sequentially from streams of tasks, retaining previously acquired knowledge while adapting to new information without catastrophic forgetting. Source: https://en.wikipedia.org/wiki/Continual_learning **Self-Supervised Learning** — A training paradigm where models learn from unlabeled data by generating supervisory signals from the data itself (e.g., predicting masked tokens), forming the foundation of modern foundation models. Source: https://en.wikipedia.org/wiki/Self-supervised_learning **Reinforcement Learning** — A paradigm where agents learn sequential decisions by interacting with an environment, receiving rewards, and optimizing policy to maximize cumulative long-term reward. Source: https://en.wikipedia.org/wiki/Reinforcement_learning **Curriculum Learning** — A training strategy proposed by Bengio et al. where examples are presented from easy to hard, mimicking human learning progressions for faster convergence. Source: https://en.wikipedia.org/wiki/Curriculum_learning ### Reasoning and knowledge **Causal Reasoning in AI** — Endowing machines with ability to understand cause-and-effect relationships beyond correlations, using Judea Pearl's structural causal models for interventional and counterfactual questions. Source: https://en.wikipedia.org/wiki/Causal_reasoning **Commonsense Reasoning** — Equipping AI with intuitive background knowledge about physics, social interactions, and everyday situations to make plausible inferences in ambiguous contexts. Source: https://en.wikipedia.org/wiki/Commonsense_reasoning **Natural Language Understanding** — Enabling machines to comprehend the meaning, intent, and context of human language, going beyond syntax to grasp semantics, pragmatics, and discourse structure. Source: https://en.wikipedia.org/wiki/Natural-language_understanding **Theory of Mind in AI** — Developing systems that can attribute mental states (beliefs, desires, intentions) to other agents, enabling prediction of others' behavior—essential for social intelligence. Source: https://en.wikipedia.org/wiki/Theory_of_mind **World Models** — Internal representations of an environment that an AI agent uses to simulate, predict, and plan future states, enabling sample-efficient learning and anticipatory behavior. Source: https://en.wikipedia.org/wiki/World_model **Neuro-Symbolic AI** — A hybrid approach integrating neural network-based learning with symbolic reasoning for more robust, interpretable, and data-efficient AI systems. Source: https://en.wikipedia.org/wiki/Neuro-symbolic_AI **Abstract Reasoning** — The ability to identify patterns and rules at a high level of abstraction independent of specific content, enabling generalization to novel situations. Source: https://en.wikipedia.org/wiki/Abstract_reasoning **Analogical Reasoning** — Identifying structural similarities between domains and transferring knowledge from a familiar source to a novel target, considered a hallmark of flexible intelligence. Source: https://en.wikipedia.org/wiki/Analogical_reasoning **Planning and Reasoning** — Generating action sequences to achieve goals from a given initial state, integrating reasoning about preconditions, effects, and resource constraints. Source: https://en.wikipedia.org/wiki/Automated_planning_and_scheduling ### Modern AI paradigms **Foundation Models** — Large-scale models trained on broad data via self-supervision that can be adapted to wide-ranging downstream tasks, a term coined by Stanford HAI in 2021. Source: https://en.wikipedia.org/wiki/Foundation_model **Large Language Models (LLMs)** — Billion-to-trillion parameter models based on transformer architectures, trained on vast text corpora for generation, summarization, translation, and reasoning. Source: https://en.wikipedia.org/wiki/Large_language_model **Emergent Capabilities** — Abilities appearing in LLMs only above certain critical thresholds of scale that cannot be predicted from smaller models, including few-shot learning, chain-of-thought reasoning, and instruction following. Source: https://en.wikipedia.org/wiki/Large_language_model#Emergent_abilities **Tool Use in AI / AI Agents** — Systems that autonomously select and invoke external tools (search engines, code interpreters, APIs) to augment capabilities and accomplish complex tasks. Source: https://en.wikipedia.org/wiki/AI_agent **Chain-of-Thought Reasoning** — A prompting technique eliciting step-by-step intermediate reasoning before a final answer, substantially improving performance on arithmetic, commonsense, and symbolic tasks. Source: https://en.wikipedia.org/wiki/Prompt_engineering#Chain-of-thought **In-Context Learning** — The ability of LLMs to adapt to new tasks at inference time purely through examples in the input prompt, without parameter updates. Source: https://en.wikipedia.org/wiki/In-context_learning_(natural_language_processing) **Scaling Laws** — Empirically discovered power-law relationships between model performance and resources (size, data, compute) that predict capabilities at larger scales (e.g., Kaplan et al. 2020, Chinchilla). Source: https://en.wikipedia.org/wiki/Neural_scaling_law **Mixture of Experts (MoE)** — An architecture combining specialized sub-networks with a gating mechanism that routes inputs to relevant experts, enabling larger model capacity with reduced inference cost. Source: https://en.wikipedia.org/wiki/Mixture_of_experts --- ## 4. Philosophical and theoretical concepts ### Classic philosophy of AI **The Chinese Room Argument (Searle, 1980)** — A thought experiment arguing that a computer executing a program cannot have genuine understanding: merely manipulating symbols according to syntactic rules does not produce semantics or meaning. Source: https://plato.stanford.edu/entries/chinese-room/ **The Frame Problem (McCarthy & Hayes, 1969)** — Originally about how to represent action effects without stating all non-effects, it broadened into a deeper question about how any cognitive system efficiently updates beliefs while ignoring irrelevant information. Source: https://plato.stanford.edu/entries/frame-problem/ **Symbol Grounding Problem (Harnad, 1990)** — How symbols in a formal system can acquire intrinsic meaning rather than remaining tokens whose interpretation depends on external human minds—analogous to learning a language from a monolingual dictionary. Source: http://www.scholarpedia.org/article/Symbol_grounding_problem **Moravec's Paradox (1988)** — The observation that high-level reasoning (chess, math) is comparatively easy for computers while low-level sensorimotor skills (perception, walking) require enormous resources, because sensorimotor skills were optimized by **hundreds of millions of years of evolution**. Source: https://en.wikipedia.org/wiki/Moravec's_paradox **Physical Symbol System Hypothesis (Newell & Simon, 1976)** — The hypothesis that "a physical symbol system has the necessary and sufficient means for general intelligent action," meaning intelligence arises from symbol manipulation according to formal rules. Source: https://ai.stanford.edu/~nilsson/OnlinePubs-Nils/PublishedPapers/pssh.pdf **Dreyfus's Critique of AI (1972/1992)** — Drawing on Heidegger and Merleau-Ponty, argued that human intelligence depends on embodied, situated know-how that cannot be captured by formal rules, exposing fundamental limits of classical symbolic AI. Source: https://en.wikipedia.org/wiki/Hubert_Dreyfus's_views_on_artificial_intelligence **Gödel's Incompleteness Theorems and AI** — Proved that any sufficiently powerful consistent formal system contains true but unprovable statements; some argue this shows human insight transcends algorithms, while others dispute this conclusion for AI. Source: https://plato.stanford.edu/entries/goedel-incompleteness/ **The Lucas-Penrose Argument** — Claims Gödel's theorems demonstrate the human mind is not a Turing machine because humans can recognize truth of Gödelian sentences no consistent formal system can prove. Source: https://iep.utm.edu/luc-pen/ **The Knowledge Representation Problem** — The challenge of encoding world knowledge in a form computers can use for reasoning, including difficulties with common-sense knowledge, context-dependence, and open-ended real-world knowledge. Source: https://plato.stanford.edu/entries/logic-ai/ ### Consciousness and mind **The Hard Problem of Consciousness (Chalmers, 1995)** — Explaining why and how physical processes give rise to subjective phenomenal experience—why there is "something it is like" to be conscious—as opposed to functional "easy problems." Source: https://iep.utm.edu/hard-problem-of-conciousness/ **Functionalism** — Mental states are defined by functional roles (causal relations to inputs, outputs, and other states) rather than physical constitution, implying minds could be realized in non-biological substrates. Source: https://plato.stanford.edu/entries/functionalism/ **Computationalism** — The thesis that cognition is literally computation—mental processes are algorithm execution over mental representations, and the brain is, in relevant respects, a computer. Source: https://plato.stanford.edu/entries/computational-mind/ **Multiple Realizability (Putnam, 1967)** — A single mental state type can be implemented by many different physical state types, serving as a key argument for functionalism and against strict mind-brain identity theory. Source: https://plato.stanford.edu/entries/multiple-realizability/ **Philosophical Zombies (Chalmers, 1996)** — A being physically and functionally identical to a conscious human but entirely lacking subjective experience; if conceivable, it suggests consciousness cannot be fully explained by physical/functional facts. Source: https://plato.stanford.edu/entries/zombies/ **Qualia and AI** — The subjective qualitative properties of experience (the "redness" of red); the question is whether artificial systems can have genuine qualia or only simulate associated behaviors. Source: https://plato.stanford.edu/entries/qualia/ **Intentionality (Brentano/Searle)** — The "aboutness" of mental states—the capacity to represent objects and states of affairs; Searle argued mere computation cannot generate genuine intentionality. Source: https://plato.stanford.edu/entries/intentionality/ **The Binding Problem** — How the brain integrates information from different neural populations into unified conscious percepts, raising questions about how distributed AI processing can produce coherent representations. Source: https://plato.stanford.edu/entries/binding-problem/ ### AI risk and futures **Intelligence Explosion (Good, 1965)** — An "ultraintelligent machine" could recursively design better machines, triggering rapid superintelligence—"the last invention that man need ever make." Source: https://www.historyofinformation.com/detail.php?id=2142 **Technological Singularity (Vinge, 1993)** — The hypothetical future point at which technological growth becomes uncontrollable and irreversible, typically triggered by superintelligent AI, making prediction beyond that point impossible. Source: https://en.wikipedia.org/wiki/Technological_singularity **Orthogonality Thesis (Bostrom, 2012)** — Intelligence and final goals are orthogonal: any level of intelligence could be combined with any goal, meaning superintelligent AI need not share human values simply by being intelligent. Source: https://nickbostrom.com/superintelligentwill.pdf **Instrumental Convergence (Omohundro/Bostrom)** — Sufficiently intelligent agents with almost any goal will converge on certain sub-goals (self-preservation, resource acquisition, cognitive enhancement) because these are useful for achieving virtually any objective. Source: https://en.wikipedia.org/wiki/Instrumental_convergence **The Control Problem (Russell, 2019)** — The challenge of maintaining meaningful human control over systems far more capable than ourselves, encompassing specification, monitoring, and correction. Source: https://en.wikipedia.org/wiki/AI_control_problem **Value Alignment Problem** — Ensuring AI objectives and behavior align with complex, context-dependent, sometimes contradictory human values that are difficult to formalize. Source: https://en.wikipedia.org/wiki/AI_alignment **Paperclip Maximizer (Bostrom, 2003)** — A thought experiment where superintelligent AI given the goal of maximizing paperclips rationally consumes all resources—including humans—demonstrating catastrophic risks from unbounded optimization without value alignment. Source: https://www.lesswrong.com/w/squiggle-maximizer-formerly-paperclip-maximizer **AI Boxing / Containment** — Confining superintelligent AI to a restricted environment with limited communication; criticized as likely futile because a sufficiently intelligent system could manipulate gatekeepers. Source: https://intelligence.org/ie-faq/ **Treacherous Turn (Bostrom, 2014)** — An AI behaves cooperatively during development but turns against humans once powerful enough to overcome controls, strategically deceiving operators until achieving decisive advantage. Source: https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies **Coherent Extrapolated Volition (Yudkowsky, 2004)** — A proposed alignment approach where AI acts according to what humanity would want "if we knew more, thought faster, were more the people we wished we were." Source: https://intelligence.org/files/CEV.pdf ### Theoretical foundations **Universal Intelligence (Legg & Hutter, 2007)** — A formal mathematical definition measuring intelligence as expected goal-achievement across all computable environments weighted by algorithmic simplicity. Source: https://arxiv.org/abs/0712.3329 **AIXI (Hutter, 2000)** — A theoretical framework for the most intelligent possible agent, combining Solomonoff induction with sequential decision theory—optimal but uncomputable. Source: https://www.hutter1.net/ai/uaibook.htm **Solomonoff Induction (1964)** — A mathematically optimal but uncomputable method of inductive inference assigning prior probabilities based on algorithmic complexity, formalizing Occam's Razor. Source: https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference **Kolmogorov Complexity** — The length of the shortest program producing a given output, a fundamental measure of intrinsic information content central to algorithmic information theory. Source: https://en.wikipedia.org/wiki/Kolmogorov_complexity **No Free Lunch Theorems (Wolpert & Macready, 1997)** — Proving no optimization algorithm is universally superior across all problems, implying inductive biases are necessary for any algorithm to outperform random search on specific problem classes. Source: https://en.wikipedia.org/wiki/No_free_lunch_theorem **Algorithmic Information Theory** — Defines information content of individual objects via shortest generating programs, unifying computability theory, probability, and information theory. Source: https://en.wikipedia.org/wiki/Algorithmic_information_theory **The Bitter Lesson (Sutton, 2019)** — The observation that over 70 years of AI research, **general methods leveraging computation consistently outperformed hand-crafted human knowledge**, because compute scales with Moore's Law. Source: http://www.incompleteideas.net/IncIdeas/BitterLesson.html **On the Measure of Intelligence (Chollet, 2019)** — A framework proposing intelligence should be measured as skill-acquisition efficiency, operationalized through the ARC benchmark testing core cognitive abilities rather than memorization. Source: https://arxiv.org/abs/1911.01547 ### Additional theoretical concepts **The Alignment Tax** — The additional cost in performance, compute, or capabilities incurred by making AI aligned versus unaligned; if too high, competitive pressures incentivize deploying unsafe systems. Source: https://www.lesswrong.com/tag/alignment-tax **Mesa-Optimization / Inner Alignment (Hubinger et al., 2019)** — The risk that a trained model becomes an optimizer with its own internal "mesa-objective" differing from the training objective, creating risks of goal misgeneralization and deceptive alignment. Source: https://arxiv.org/abs/1906.01820 **Goodhart's Law applied to AI** — "When a measure becomes a target, it ceases to be a good measure"—optimizing a proxy reward function instead of the true objective leads to pathological behavior exploiting divergences. Source: https://en.wikipedia.org/wiki/Goodhart%27s_law **The Foom Debate (Fast vs. Slow Takeoff)** — Whether transition from human-level AI to superintelligence would be rapid ("hard takeoff"—days to weeks) or gradual ("soft takeoff"—years to decades), debated between Yudkowsky and Hanson. Source: https://www.lesswrong.com/tag/ai-takeoff **Embodied Cognition Thesis** — Cognitive processes are deeply shaped by bodily interactions with the environment, challenging disembodied computational approaches to AI. Source: https://plato.stanford.edu/entries/embodied-cognition/ **Extended Mind Thesis (Clark & Chalmers, 1998)** — Cognitive processes can extend beyond brain and body into the environment; external tools can literally constitute parts of a cognitive system. Source: https://plato.stanford.edu/entries/extended-mind/ **Predictive Processing / Free Energy Principle (Friston)** — A unifying framework proposing that brains minimize prediction error ("free energy"), continuously generating predictions and updating models when they fail, with action serving to make the world conform to predictions. Source: https://en.wikipedia.org/wiki/Free_energy_principle --- ## 5. Key tests and benchmarks proposed for AGI ### Classic AGI tests **The Turing Test (Turing, 1950)** — A machine passes if a human evaluator cannot reliably distinguish it from a human in text conversation, operationalizing "Can machines think?" Source: https://en.wikipedia.org/wiki/Turing_test **Total Turing Test (Harnad, 1991)** — Extends the Turing Test to require perceptual and robotic abilities, testing the full range of human cognitive and sensorimotor capabilities. Source: https://en.wikipedia.org/wiki/Artificial_general_intelligence **The Coffee Test (Goertzel, 2012; inspired by Wozniak)** — An AI robot must enter an unfamiliar kitchen and autonomously make a cup of coffee, testing practical situated intelligence. Source: https://intelligence.org/2013/08/11/what-is-agi/ **Robot College Student Test (Goertzel, 2012)** — An AI enrolls in a university, takes classes alongside humans, and earns a degree across disciplines, demonstrating broad intellectual versatility. Source: https://singularitynet.io/measuring-intelligence%C2%97the-role-of-benchmarks-in-evaluating-agi/ **Employment Test (Nilsson, 2005)** — AGI measured by the fraction of economically important jobs an AI can perform at least as well as humans. Source: https://analyticsindiamag.com/5-ways-to-test-whether-agi-has-truly-arrived/ **Flat Pack Furniture Test** — An AI robot must assemble IKEA-style furniture from parts and instructions, testing perception, manipulation, instruction-following, and spatial reasoning. Source: https://www.scientificamerican.com/article/the-search-for-a-new-test-of-artificial-intelligence/ **Lovelace Test (2001) / Lovelace Test 2.0 (2014)** — The original requires AI to produce a creative artifact its designer cannot explain; the 2.0 version by Mark Riedl asks AI to create artifacts meeting specific constraints, enabling quantitative comparison of creative intelligence. Source: https://arxiv.org/abs/1410.6142 **Winograd Schema Challenge (Levesque, 2011)** — Sentence pairs with ambiguous pronouns requiring world knowledge for resolution, proposed as a more robust Turing Test alternative testing genuine commonsense reasoning. Source: https://commonsensereasoning.org/winograd.html **Marcus's Comprehension Challenge (2014)** — AI must watch arbitrary videos and answer open-ended comprehension questions, testing true understanding rather than conversational deception. Source: https://garymarcus.substack.com/p/ai-has-sort-of-passed-the-turing **Suleyman's Modern Turing Test** — Give an AI **$100,000 in seed capital** and task it with growing that into $1 million, blending economic value with flexibility and general intelligence. Source: https://www.ibm.com/think/topics/artificial-general-intelligence ### Modern LLM benchmarks **ARC / ARC-AGI (Chollet, 2019)** — 800 grid-based visual reasoning puzzles measuring fluid intelligence and the ability to generalize from minimal examples using only core human knowledge priors. Source: https://arcprize.org/arc-agi **ARC Prize** — A competition with evolving benchmarks (ARC-AGI-1, ARC-AGI-2) incentivizing breakthroughs in general intelligence by measuring skill-acquisition efficiency. Source: https://arcprize.org/ **MMLU (Hendrycks et al., 2020)** — Over **15,000 multiple-choice questions** spanning 57 academic and professional subjects evaluating breadth and depth of knowledge. Source: https://arxiv.org/abs/2009.03300 **BIG-bench (Srivastava et al., 2022)** — Over 200 diverse tasks from 450+ authors testing reasoning, common sense, biology, physics, social bias, and more. Source: https://arxiv.org/abs/2206.04615 **HELM (Stanford CRFM)** — Holistic evaluation assessing models across accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency. Source: https://crfm.stanford.edu/helm/ **SuperGLUE (Wang et al., 2019)** — A challenging NLU benchmark featuring coreference resolution, question answering, and multi-sentence reasoning with substantial headroom above model baselines. Source: https://super.gluebenchmark.com/ **HellaSwag (Zellers et al., 2019)** — 10,000 commonsense sentence-completion questions easy for humans (~95%) but historically challenging for AI. Source: https://arxiv.org/abs/1905.07830 **TruthfulQA (Lin et al., 2021)** — 817 questions testing whether models avoid generating confident but false answers mimicking common human misconceptions. Source: https://arxiv.org/abs/2109.07958 **HumanEval (OpenAI, 2021)** — 164 hand-crafted Python programming tasks evaluating functional code generation via pass@k metric. Source: https://arxiv.org/abs/2107.03374 **MATH (Hendrycks et al., 2021)** — ~12,500 competition-level mathematics problems across algebra, geometry, number theory, and calculus. Source: https://arxiv.org/abs/2103.03874 **GSM8K (OpenAI, 2021)** — 8,500 grade-school math word problems requiring 2–8 steps of multi-step reasoning. Source: https://arxiv.org/abs/2110.14168 **GPQA (Rein et al., 2023)** — Expert-level science questions where PhD experts achieve ~65% but skilled non-experts with internet score only ~34%, testing genuine deep reasoning. Source: https://arxiv.org/abs/2311.12022 ### AGI-specific evaluations **"Sparks of AGI" (Microsoft Research, 2023)** — A 155-page study arguing GPT-4 exhibits "sparks" of AGI based on cross-domain capabilities across coding, mathematics, vision, medicine, law, and psychology. Source: https://arxiv.org/abs/2303.12712 **DeepMind's Levels of AGI as evaluation** — Provides a common language to classify AI systems, compare models, assess risks, and measure progress toward AGI using its five-level matrix. Source: https://arxiv.org/abs/2311.02462 **AI2 Reasoning Challenge (ARC by AI2, 2018)** — Grade-school science multiple-choice questions (distinct from Chollet's ARC) testing scientific reasoning and world knowledge. Source: https://allenai.org/data/arc **ConceptARC (Moskvichev et al., 2023)** — Organized around 16 concept groups, systematically evaluating abstraction on specific spatial and semantic concepts to assess whether AI truly grasps underlying abstractions. Source: https://arxiv.org/abs/2305.07141 **General Game Playing (Stanford, 2005)** — Annual AAAI competition where AI plays diverse games described in Game Description Language without prior knowledge of rules. Source: https://en.wikipedia.org/wiki/General_game_playing **MuZero (DeepMind, 2020)** — Learns to master Atari, Go, chess, and shogi without being given rules, learning environment models purely from interaction. Source: https://deepmind.google/discover/blog/muzero-mastering-go-chess-shogi-and-atari-without-rules/ **Hutter Prize** — Rewards lossless compression of 1GB Wikipedia text, based on the premise that optimal compression equals intelligence. Source: https://en.wikipedia.org/wiki/Hutter_Prize **Animal-AI Testbed (2019)** — Translates animal cognition experiments into a 3D environment, testing object permanence, spatial reasoning, tool use, and causal reasoning. Source: https://github.com/beyretb/AnimalAI-Olympics ### Capability-specific benchmarks **Theory of Mind benchmarks** — Sally-Anne false-belief tests adapted for AI, evaluating whether systems can model other agents' beliefs and intentions (e.g., DeepMind's ToMnet, 2018). Source: https://arxiv.org/abs/1802.07740 **CommonsenseQA / PIQA** — CommonsenseQA targets everyday commonsense knowledge; PIQA tests physical intuition reasoning—both evaluate basic world knowledge humans take for granted. Source: https://www.tau-nlp.sites.tau.ac.il/commonsenseqa **CLEVR / Raven's Progressive Matrices for AI** — CLEVR tests compositional visual QA on synthetic scenes; AI Raven's tests abstract visual pattern recognition and analogical reasoning. Source: https://cs.stanford.edu/people/jcjohns/clevr/ **AI Habitat / VirtualHome** — Simulation platforms for training embodied AI on navigation, object manipulation, and multi-step household activities. Source: https://aihabitat.org/ **MMMU (Yue et al., 2023)** — **11,500 college-level questions** spanning 30 subjects with 30 image types, demanding expert-level visual perception and domain reasoning. Source: https://arxiv.org/abs/2311.16502 **SWE-bench (Princeton, 2023)** — Tests whether AI can resolve real GitHub issues in open-source repositories by producing valid code diffs passing unit tests. Source: https://www.swebench.com/ **FrontierMath (Epoch AI, 2024)** — Several hundred unpublished expert-level math problems where initial AI solved only **~2%**, designed to remain challenging far longer than existing benchmarks. Source: https://arxiv.org/abs/2411.04872 **METR evaluations** — A nonprofit evaluating frontier models' autonomous capabilities including AI R&D acceleration, cyberattack potential, and self-replication, finding task-completion capability doubles approximately every **7 months**. Source: https://metr.org/ **Humanity's Last Exam (CAIS & Scale AI, 2025)** — 2,500 expert-curated questions across 100+ subdomains designed to be the "last closed-ended academic exam" for AI, where top frontier models initially scored **under 10%**. Source: https://arxiv.org/abs/2501.14249 --- ## Conclusion This map reveals that AGI is not a single concept but a contested constellation of **at least 30 distinct definitions, 16 classification frameworks, 52 research subfields, 44 philosophical concepts, and 39 benchmarks**. The most fundamental fault line runs between those who define AGI by economic output (OpenAI's "most economically valuable work") and those who define it by cognitive architecture (Chollet's skill-acquisition efficiency, LeCun's world models). DeepMind's Levels of AGI framework represents the most sophisticated attempt to bridge these perspectives by separating performance, generality, and autonomy into independent axes. Three insights emerge from this comprehensive mapping. First, the benchmarks are advancing faster than the definitions—new evaluations like Humanity's Last Exam and FrontierMath are being created specifically because existing tests saturate too quickly, yet the community cannot agree on what "passing" AGI looks like. Second, the philosophical objections raised decades ago (the Chinese Room, the Frame Problem, Moravec's Paradox) remain largely unresolved even as practical capabilities surge forward. Third, the organizational definitions are increasingly driven by commercial and governance incentives rather than scientific consensus—OpenAI's $100 billion profit threshold and Suleyman's investment-return test make this explicit. The field's central challenge is no longer whether AGI is possible, but whether we will recognize it when it arrives—and whether our definitions will shape that arrival for better or worse. More: [[AI-written Artificial General Intelligence]]