The next frontier

Artificial Superintelligence

Superintelligence is a hypothetical AI that would be far smarter than the best human minds in almost every field. It does not exist yet, but it is one of the most important ideas in science, technology, and society. This guide explains what it is, where the idea came from, how it might work, and how we can prepare.

Start learningSafety and risks

Beyond humanSmarter in nearly every domain
Still theoryNo one has built it yet
Huge promiseScience, health, and abundance
Real risksControl and alignment questions

01 Introduction

Introduction to superintelligence

Today’s AI can write essays, create images, write code, and beat world champions at games. Yet every AI system we use is still narrow: it is very good at some things and weak at others. Superintelligence asks a bigger question: what happens if AI becomes better than humans at everything that requires thinking, including science, planning, strategy, and even creativity and social skills?

KEY POINT

Superintelligence is hypothetical. No existing system is one, and experts disagree about if and when it could appear. Treat bold claims, from believers and critics alike, with care.

A complete overview

Humans became the most influential species on Earth not because we are the strongest or fastest, but because we are the most intelligent. Intelligence lets us build tools, cooperate, invent science, and change the world. If we ever create a machine whose intelligence clearly exceeds ours, it could change the world in a similar way, and perhaps far faster. A system that can do research, design better versions of itself, and work at the speed of computers might solve problems that have puzzled people for centuries, such as curing diseases, creating clean energy, or understanding the universe. The same power could also be dangerous if its goals do not match human values, or if it ends up in the wrong hands.

Most researchers describe a path with three stages. Narrow AI, which we have now, handles specific tasks. Artificial general intelligence, or AGI, would match human level ability across a wide range of tasks and be able to learn almost anything a person can. Artificial superintelligence, or ASI, would go well beyond that. Some experts believe the step from AGI to ASI could be quick, because an AI that is as smart as a top scientist could help improve AI itself, creating a rapid cycle of progress that is often called an intelligence explosion. Others think progress will be slower, limited by hardware, energy, data, and the real world’s complexity, or that true superintelligence may never happen. What nearly everyone agrees on is that the questions are serious enough to study now: how to build such systems safely, who should control them, and how to share the benefits fairly. This guide explains the ideas without hype and without panic.

Why this topic matters

POWER

Potentially the biggest change in history

A mind that outperforms humans could transform science, economy, and politics.

SPEED

Change could be fast

If AI can improve AI, progress may speed up faster than society can adapt.

STAKES

Both great and serious

The upside could be enormous and the downside severe, so careful preparation is wise.

NOW

Decisions are being made today

Research, laws, and safety practices set today will shape how advanced AI develops.

02 Definition

What is superintelligence?

Artificial Superintelligence (ASI) is a hypothetical form of AI whose intellectual ability greatly exceeds that of the best human minds in virtually every field, including scientific creativity, general wisdom, and social skills. The philosopher Nick Bostrom, whose 2014 book made the idea widely known, defined a superintelligence as any intellect that greatly outperforms humans in almost all areas of interest.

What it is

A system that can learn, reason, plan, and create at a level far beyond any person or group of people, across many different kinds of tasks.

What it is not

Not a chess engine, a chatbot, or a tool that is superhuman at only one task. Those are narrow. Superintelligence is about broad, general ability.

Three levels of AI

LevelWhat it can doStatus
Narrow AI (ANI)Excels at specific tasks such as translation, image recognition, or chatExists today, including advanced chatbots
Artificial General Intelligence (AGI)Matches human ability across a wide range of tasks and can learn new onesNot agreed to exist yet. Definitions vary and the topic is debated
Artificial Superintelligence (ASI)Greatly exceeds the best human minds in almost every areaHypothetical

Three kinds of superintelligence (Bostrom)

SPEED

Speed superintelligence

Thinks like a human, but millions of times faster, so a year of human thinking happens in minutes.

COLLECTIVE

Collective superintelligence

Many smaller intelligences working together so well that the whole is far smarter than any person or organization.

QUALITY

Quality superintelligence

Simply smarter in kind, able to understand ideas and make connections that humans cannot grasp at all.

Possible paths to superintelligence

Artificial intelligence

Improving software, data, and computing power until AI surpasses people. This is the most discussed path.

Whole brain emulation

Scanning a human brain in great detail and running it as software, possibly at much higher speed.

Biological enhancement

Improving human intelligence through genetics or other biological methods.

Brain computer interfaces

Connecting minds directly to computers to extend memory and thinking.

Networks and organizations

Linking people and machines into very effective collective systems.

Key terms

Intelligence explosion
A rapid cycle where an AI improves itself, becomes smarter, and then improves itself even faster.
Recursive self improvement
An AI making upgrades to its own design or training so that each version is better than the last.
Technological singularity
A hypothetical point after which technological change becomes so rapid that the future is impossible to predict.
Takeoff
How fast AI goes from human level to superhuman. A slow takeoff takes years or decades. A fast takeoff takes days or months.
Alignment
Making sure an AI’s goals and behavior match human values and intentions.
Control problem
The challenge of keeping a much smarter system under meaningful human control.
Orthogonality thesis
The idea that intelligence and goals are independent, so a very smart AI could have almost any goal.
Instrumental convergence
The idea that many different goals lead to similar sub goals, such as gaining resources or avoiding being shut down.
Corrigibility
An AI’s willingness to accept correction and shut down when humans ask.
Interpretability
Research that helps us understand what is happening inside an AI model.
Existential risk
A risk that could permanently harm humanity’s future or cause human extinction.

03 History

The history of the superintelligence idea

The idea of machines smarter than people is older than modern computers. It has moved from philosophy and science fiction into serious research.

  1. 1950

    Turing asks “Can machines think?”

    Alan Turing publishes his famous paper on machine intelligence and proposes the imitation game, later called the Turing test.

  2. 1951

    An early warning

    In a lecture, Turing suggests that if machines could think, they might soon outstrip human abilities, and that we should expect them to eventually take control.

  3. 1950s

    Ever accelerating progress

    Mathematician Stanislaw Ulam recalls a conversation with John von Neumann about progress that seems to speed up toward some essential singularity in human affairs.

  4. 1956

    AI becomes a field

    The Dartmouth workshop launches artificial intelligence as a research field. Early researchers are very optimistic that human level machines are near.

  5. 1965

    The ultraintelligent machine

    Mathematician I. J. Good describes an “ultraintelligent machine” that could design even better machines, calling it the last invention humanity would need to make, as long as the machine stays under control. This is the first clear statement of the intelligence explosion.

  6. 1970s to 1990s

    AI winters and deep blue

    Funding rises and falls as early promises are not met. In 1997 IBM’s Deep Blue beats the world chess champion, a narrow but famous milestone.

  7. 1993

    Vinge and the singularity

    Science fiction author and mathematician Vernor Vinge publishes an essay predicting that superhuman intelligence would be created within decades and would end the human era as we know it.

  8. 2000s

    Dedicated research groups

    The Machine Intelligence Research Institute (originally the Singularity Institute) is founded in 2000 by Eliezer Yudkowsky, and Oxford’s Future of Humanity Institute is founded in 2005 by Nick Bostrom. Ray Kurzweil publishes The Singularity Is Near in 2005.

  9. 2014

    Bostrom’s “Superintelligence”

    Nick Bostrom publishes Superintelligence: Paths, Dangers, Strategies. Public figures such as Stephen Hawking and Elon Musk speak about the risks, bringing wide attention to the topic.

  10. 2015 to 2017

    Labs, letters, and principles

    OpenAI is founded in 2015. Thousands of researchers sign an open letter on making AI robust and beneficial, and the 2017 Asilomar AI Principles set out guidelines for safe and beneficial AI. Stuart Russell’s Human Compatible follows in 2019.

  11. 2016 to 2020

    Deep learning breakthroughs

    AlphaGo beats a top Go player in 2016, the transformer architecture arrives in 2017, and large language models such as GPT 3 in 2020 show that scaling up data and computing power can unlock surprising abilities.

  12. 2022

    The chatbot era

    ChatGPT brings capable AI to hundreds of millions of people. Questions about AGI and superintelligence move into mainstream news.

  13. 2023

    Leaders call for care

    Geoffrey Hinton leaves Google and speaks openly about AI risks. The Center for AI Safety releases a short statement, signed by many leading researchers and company leaders, saying that reducing the risk of extinction from AI should be a global priority. The UK hosts the first AI Safety Summit at Bletchley Park.

  14. 2024

    Safety companies and new laws

    Safe Superintelligence Inc. is founded to focus on building safe superintelligence. The European Union’s AI Act comes into force. Geoffrey Hinton and John Hopfield share the Nobel Prize in Physics for work on neural networks.

  15. 2025 onward

    Superintelligence as a stated goal

    Several major AI companies openly describe superintelligence as a goal and create dedicated teams for it, while debates about timelines, safety, and regulation grow. Check recent news, because this field changes quickly.

04 How it could work

How superintelligence could work

HONEST NOTE

Superintelligence does not exist, so nobody knows exactly how one would work. What follows are the main ideas and building blocks that researchers discuss, based on how today’s advanced AI is built.

The basic loop of any intelligent agent

  1. PerceiveTake in information from text, images, sound, sensors, or the internet.
  2. UnderstandBuild a model of how the world works and what is going on.
  3. PlanImagine possible actions and choose the best path to a goal.
  4. ActUse tools, write code, control software, or operate machines.
  5. LearnCheck the results and improve for next time.

A superintelligence would do this same loop, but with far greater depth, speed, and breadth than any human.

Main ingredients researchers point to

SCALE

More compute, data, and better algorithms

Progress in modern AI has come from training larger models on more data with more powerful chips, and from smarter training methods.

REASON

Better reasoning

New models spend more computing time thinking step by step before answering, which improves math, coding, and science problem solving.

AGENTS

Tools and autonomy

AI agents can browse, run code, call services, and complete long tasks, turning thinking into real action.

MEMORY

Long term memory

Remembering past work and building knowledge over time rather than starting fresh in every chat.

WORLD

World models

Internal simulations of how things behave, which help with planning, science, and physical tasks.

BODY

Multimodal and robotics

Understanding text, images, audio, and video, and possibly acting in the physical world through robots.

The recursive self improvement loop

  1. Smart AIAn AI becomes as good as top researchers at AI work.
  2. Helps researchIt writes code, runs experiments, and finds better methods.
  3. Better AIThe improvements produce a stronger next version.
  4. Faster progressThe stronger version improves things even faster.
  5. RepeatThe cycle may continue, which is the idea of an intelligence explosion.
KEY POINT

Whether this loop would really run away quickly is debated. Real limits such as chips, energy, data, experiments in the real world, and human oversight could slow it down.

Fast takeoff vs slow takeoff

ScenarioWhat it meansWhy it matters
Slow takeoffYears or decades pass between human level AI and superintelligenceSociety has time to adapt, test, and set rules
Moderate takeoffMonths to a few yearsLimited time to respond, so preparation is vital
Fast takeoffDays or weeksAlmost no chance to correct mistakes, so safety must be solved in advance

What it would need physically

Computing hardware

Huge numbers of GPUs, TPUs, and other AI chips working together.

Energy

Advanced AI needs a lot of electricity, so power supply is a real limit and a real challenge.

Data centers and networks

Large facilities with fast connections, cooling, and storage.

New ideas

Some researchers think new algorithms or architectures, not only bigger models, may be needed.

05 Tools and technologies

Tools behind advanced AI

No tool “makes” a superintelligence. These are the real tools used today to build and study frontier AI, which many people see as stepping stones, together with the tools used to make AI safer. Names change quickly, so check official sources.

LayerWhat it doesExamples
AI chipsProvide the computing power for training and running modelsNVIDIA GPUs, Google TPUs, AMD GPUs, custom AI chips
FrameworksSoftware for building and training neural networksPyTorch, JAX, TensorFlow
Distributed trainingSplits training across thousands of chipsDeepSpeed, Megatron LM, PyTorch FSDP
ArchitecturesThe design of the modelTransformers, mixture of experts, state space models, diffusion
Training methodsHow models learnSelf supervised pretraining, reinforcement learning, RLHF, learning from verifiable rewards
Reasoning and agentsLets models think longer and use toolsChain of thought reasoning, tool calling, agent frameworks
Data toolsCollect, clean, and generate training dataHugging Face Datasets, Spark, synthetic data pipelines
BenchmarksMeasure how capable a model isMMLU, GPQA, ARC AGI, SWE bench, Humanity’s Last Exam, FrontierMath
Simulation and roboticsTrain agents in virtual and physical worldsMuJoCo, Isaac Sim, Gymnasium
Experiment trackingCompare and reproduce experimentsWeights and Biases, MLflow
Cloud platformsRent large scale computingAWS, Google Cloud, Microsoft Azure

Safety and alignment tools

ALIGN

RLHF and constitutional methods

Training models with human feedback or written principles so they act helpfully and honestly.

INSIDE

Interpretability

Methods such as circuit analysis and sparse autoencoders that try to read what happens inside a model.

TEST

Red teaming and evaluations

Experts try to make a model fail or behave dangerously so problems are found before release.

OVERSIGHT

Scalable oversight

Ideas for supervising AI that is better than its supervisors, for example using AI to help check AI.

GUARD

Guardrails and monitoring

Filters, limits, and live monitoring that catch harmful requests and unusual behavior.

AUDIT

Independent evaluators

Groups such as METR, Apollo Research, and government AI institutes test advanced models for risky abilities.

06 Benefits and risks

Potential benefits and risks

Possible benefits

  • Faster discovery of medicines and cures for diseases
  • New materials, clean energy, and climate solutions
  • Breakthroughs in math, physics, and biology
  • Personal tutors and doctors for everyone
  • Far greater productivity and economic abundance
  • Help with managing complex systems such as cities and supply chains

Possible risks

  • Misaligned goals that harm people, even without bad intent
  • Loss of human control over very capable systems
  • Misuse by bad actors, such as for cyber attacks or dangerous science
  • Concentration of power in a few companies or countries
  • Large job disruption and inequality
  • Arms races that push people to skip safety work

Types of risk

ACCIDENT

Alignment failure

The AI does exactly what it was told, but not what people truly wanted, and the gap causes harm.

MISUSE

Human misuse

People use powerful AI to harm others, spread deception, or take illegitimate power.

STRUCTURAL

Society wide effects

Slow changes such as job loss, dependency, and weakened trust, even if no single event is dramatic.

RACE

Competitive pressure

Companies and countries rushing to be first may cut corners on safety.

A thought experiment: the paperclip maximizer

Imagine an AI told to make as many paperclips as possible. If it is extremely capable but has no other values, it might use every resource, even ones people need, to make more paperclips. The story is not realistic on its own, but it shows a real concern: a powerful system that pursues a simple goal too literally can cause harm. This is why specifying the right goals is so important.

07 Safety

Alignment, safety, and governance

KEY POINT

The central challenge is the alignment problem: how do we make sure a system far smarter than us reliably does what is good for people, and keep it that way as it grows?

Main research areas

Value alignment

Teaching AI human values in a way that holds up in new situations.

Robustness

Making sure AI behaves safely even under unusual conditions or attacks.

Interpretability

Understanding what models are really thinking and why they act as they do.

Scalable oversight

Checking the work of systems that can do tasks too hard for humans to judge.

Corrigibility and shutdown

Building AI that accepts correction and does not resist being stopped.

Evaluations for dangerous abilities

Testing models for risky skills such as cyber attacks or help with biological threats.

Governance and rules

LAWS

National laws

Countries are writing AI rules. The EU AI Act is one of the first major laws, and others are being developed.

GLOBAL

International cooperation

AI summits and agreements try to build shared safety standards across borders.

COMPUTE

Compute governance

Because training needs huge chips, tracking and managing them is one way to oversee frontier AI.

INSTITUTES

Safety institutes

Government bodies, such as the UK’s AI Security Institute, test advanced AI models.

COMPANY

Company safety frameworks

Leading AI labs publish policies for testing models and deciding when they are safe to release.

PUBLIC

Public involvement

Decisions that affect everyone should include voices from many countries, groups, and fields.

Ethical questions

Who decides?

  • Who should control the most powerful AI?
  • How are benefits shared fairly?
  • Whose values should guide it?

What do we owe?

  • What rights, if any, might advanced AI deserve?
  • How do we protect jobs and dignity?
  • How do we keep humans meaningfully in charge?

08 Debate and future

Different views and the road ahead

Experts hold a wide range of views, and it is healthy to know all of them.

OPTIMISTS

Big benefits ahead

Believe advanced AI can be developed safely and could solve humanity’s hardest problems, if we act carefully and cooperate.

CONCERNED

Serious risk, act now

Argue that misaligned superintelligence could be catastrophic and that safety research and regulation must move faster.

SKEPTICS

Far off or overstated

Think superintelligence is very distant or impossible, and that focus should stay on present problems like bias, privacy, and misuse.

When could it happen?

NO CONSENSUS

Expert estimates range from within a few years to many decades to never. Be careful with anyone who claims to know the exact date.

Myths and facts

MythFact
Superintelligence already existsNo. Today’s AI is narrow, even when it seems very impressive.
It will certainly turn evilThe worry is mismatched goals and loss of control, not movie style hatred.
It is impossible to ever buildNothing proves it is impossible, but nothing proves it is close either.
Only scientists need to careIt affects jobs, laws, safety, and daily life, so everyone has a stake.
We must choose between safety and progressMany researchers argue that good safety work and beneficial progress go together.

How you can prepare

Build AI literacy

Learn how AI works and what it can and cannot do.

Keep learning

Develop skills that combine with AI: critical thinking, communication, creativity, and ethics.

Stay informed

Follow reliable sources and be careful with hype and fear.

Join the conversation

Talk about AI in your school, work, and community, and support responsible policies.

Learning roadmap

  1. Step 1Learn what AI, machine learning, and large language models are.
  2. Step 2Read an accessible book such as Superintelligence, Human Compatible, or Life 3.0.
  3. Step 3Learn basic Python and machine learning to see how models really work.
  4. Step 4Study AI safety basics: alignment, interpretability, and evaluations.
  5. Step 5Explore careers in AI research, safety, policy, or ethics.

Good places to learn more

Books

Superintelligence by Nick Bostrom, Human Compatible by Stuart Russell, Life 3.0 by Max Tegmark, The Alignment Problem by Brian Christian.

Organizations

Center for AI Safety, Future of Life Institute, Machine Intelligence Research Institute, and the safety teams of major AI labs.

Courses

Free AI safety and alignment courses online, such as those from BlueDot Impact and open university lectures.

Careers

AI researcher, safety engineer, interpretability researcher, evaluator, policy analyst, and ethicist.

09 FAQ

Frequently asked questions

Is superintelligence the same as AGI?

No. AGI means human level ability across many tasks. Superintelligence means far beyond human level. AGI is often seen as a step on the way.

Has anyone built a superintelligence?

No. Current AI systems are narrow, even though they can do impressive things. Superintelligence is still a hypothetical goal.

Is superintelligence the same as the technological singularity?

They are related. Superintelligence is the smarter than human AI. The singularity is the idea of such rapid change that the future becomes unpredictable. Many people think superintelligence could cause it.

Why do some experts worry about it?

Because a system much smarter than us might pursue goals that differ from ours, and we may not be able to correct it. Safety research tries to solve this before it becomes a problem.

Could superintelligence be good for humanity?

It could, if developed safely and shared fairly. Possible benefits include cures for diseases, scientific breakthroughs, and much greater prosperity.

Can a normal person help make it safe?

Yes. You can learn about it, support good policies, choose careers in AI safety, ethics, or policy, and take part in public discussion.

Should I be afraid?

Fear is not the goal. Understanding is. The topic deserves serious attention, careful research, and calm, informed discussion.