Potentially the biggest change in history
A mind that outperforms humans could transform science, economy, and politics.
The next frontier
Superintelligence is a hypothetical AI that would be far smarter than the best human minds in almost every field. It does not exist yet, but it is one of the most important ideas in science, technology, and society. This guide explains what it is, where the idea came from, how it might work, and how we can prepare.
Start learningSafety and risks
01 Introduction
Today’s AI can write essays, create images, write code, and beat world champions at games. Yet every AI system we use is still narrow: it is very good at some things and weak at others. Superintelligence asks a bigger question: what happens if AI becomes better than humans at everything that requires thinking, including science, planning, strategy, and even creativity and social skills?
Superintelligence is hypothetical. No existing system is one, and experts disagree about if and when it could appear. Treat bold claims, from believers and critics alike, with care.
Humans became the most influential species on Earth not because we are the strongest or fastest, but because we are the most intelligent. Intelligence lets us build tools, cooperate, invent science, and change the world. If we ever create a machine whose intelligence clearly exceeds ours, it could change the world in a similar way, and perhaps far faster. A system that can do research, design better versions of itself, and work at the speed of computers might solve problems that have puzzled people for centuries, such as curing diseases, creating clean energy, or understanding the universe. The same power could also be dangerous if its goals do not match human values, or if it ends up in the wrong hands.
Most researchers describe a path with three stages. Narrow AI, which we have now, handles specific tasks. Artificial general intelligence, or AGI, would match human level ability across a wide range of tasks and be able to learn almost anything a person can. Artificial superintelligence, or ASI, would go well beyond that. Some experts believe the step from AGI to ASI could be quick, because an AI that is as smart as a top scientist could help improve AI itself, creating a rapid cycle of progress that is often called an intelligence explosion. Others think progress will be slower, limited by hardware, energy, data, and the real world’s complexity, or that true superintelligence may never happen. What nearly everyone agrees on is that the questions are serious enough to study now: how to build such systems safely, who should control them, and how to share the benefits fairly. This guide explains the ideas without hype and without panic.
A mind that outperforms humans could transform science, economy, and politics.
If AI can improve AI, progress may speed up faster than society can adapt.
The upside could be enormous and the downside severe, so careful preparation is wise.
Research, laws, and safety practices set today will shape how advanced AI develops.
02 Definition
Artificial Superintelligence (ASI) is a hypothetical form of AI whose intellectual ability greatly exceeds that of the best human minds in virtually every field, including scientific creativity, general wisdom, and social skills. The philosopher Nick Bostrom, whose 2014 book made the idea widely known, defined a superintelligence as any intellect that greatly outperforms humans in almost all areas of interest.
A system that can learn, reason, plan, and create at a level far beyond any person or group of people, across many different kinds of tasks.
Not a chess engine, a chatbot, or a tool that is superhuman at only one task. Those are narrow. Superintelligence is about broad, general ability.
| Level | What it can do | Status |
|---|---|---|
| Narrow AI (ANI) | Excels at specific tasks such as translation, image recognition, or chat | Exists today, including advanced chatbots |
| Artificial General Intelligence (AGI) | Matches human ability across a wide range of tasks and can learn new ones | Not agreed to exist yet. Definitions vary and the topic is debated |
| Artificial Superintelligence (ASI) | Greatly exceeds the best human minds in almost every area | Hypothetical |
Thinks like a human, but millions of times faster, so a year of human thinking happens in minutes.
Many smaller intelligences working together so well that the whole is far smarter than any person or organization.
Simply smarter in kind, able to understand ideas and make connections that humans cannot grasp at all.
Improving software, data, and computing power until AI surpasses people. This is the most discussed path.
Scanning a human brain in great detail and running it as software, possibly at much higher speed.
Improving human intelligence through genetics or other biological methods.
Connecting minds directly to computers to extend memory and thinking.
Linking people and machines into very effective collective systems.
03 History
The idea of machines smarter than people is older than modern computers. It has moved from philosophy and science fiction into serious research.
Alan Turing publishes his famous paper on machine intelligence and proposes the imitation game, later called the Turing test.
In a lecture, Turing suggests that if machines could think, they might soon outstrip human abilities, and that we should expect them to eventually take control.
Mathematician Stanislaw Ulam recalls a conversation with John von Neumann about progress that seems to speed up toward some essential singularity in human affairs.
The Dartmouth workshop launches artificial intelligence as a research field. Early researchers are very optimistic that human level machines are near.
Mathematician I. J. Good describes an “ultraintelligent machine” that could design even better machines, calling it the last invention humanity would need to make, as long as the machine stays under control. This is the first clear statement of the intelligence explosion.
Funding rises and falls as early promises are not met. In 1997 IBM’s Deep Blue beats the world chess champion, a narrow but famous milestone.
Science fiction author and mathematician Vernor Vinge publishes an essay predicting that superhuman intelligence would be created within decades and would end the human era as we know it.
The Machine Intelligence Research Institute (originally the Singularity Institute) is founded in 2000 by Eliezer Yudkowsky, and Oxford’s Future of Humanity Institute is founded in 2005 by Nick Bostrom. Ray Kurzweil publishes The Singularity Is Near in 2005.
Nick Bostrom publishes Superintelligence: Paths, Dangers, Strategies. Public figures such as Stephen Hawking and Elon Musk speak about the risks, bringing wide attention to the topic.
OpenAI is founded in 2015. Thousands of researchers sign an open letter on making AI robust and beneficial, and the 2017 Asilomar AI Principles set out guidelines for safe and beneficial AI. Stuart Russell’s Human Compatible follows in 2019.
AlphaGo beats a top Go player in 2016, the transformer architecture arrives in 2017, and large language models such as GPT 3 in 2020 show that scaling up data and computing power can unlock surprising abilities.
ChatGPT brings capable AI to hundreds of millions of people. Questions about AGI and superintelligence move into mainstream news.
Geoffrey Hinton leaves Google and speaks openly about AI risks. The Center for AI Safety releases a short statement, signed by many leading researchers and company leaders, saying that reducing the risk of extinction from AI should be a global priority. The UK hosts the first AI Safety Summit at Bletchley Park.
Safe Superintelligence Inc. is founded to focus on building safe superintelligence. The European Union’s AI Act comes into force. Geoffrey Hinton and John Hopfield share the Nobel Prize in Physics for work on neural networks.
Several major AI companies openly describe superintelligence as a goal and create dedicated teams for it, while debates about timelines, safety, and regulation grow. Check recent news, because this field changes quickly.
04 How it could work
Superintelligence does not exist, so nobody knows exactly how one would work. What follows are the main ideas and building blocks that researchers discuss, based on how today’s advanced AI is built.
A superintelligence would do this same loop, but with far greater depth, speed, and breadth than any human.
Progress in modern AI has come from training larger models on more data with more powerful chips, and from smarter training methods.
New models spend more computing time thinking step by step before answering, which improves math, coding, and science problem solving.
AI agents can browse, run code, call services, and complete long tasks, turning thinking into real action.
Remembering past work and building knowledge over time rather than starting fresh in every chat.
Internal simulations of how things behave, which help with planning, science, and physical tasks.
Understanding text, images, audio, and video, and possibly acting in the physical world through robots.
Whether this loop would really run away quickly is debated. Real limits such as chips, energy, data, experiments in the real world, and human oversight could slow it down.
| Scenario | What it means | Why it matters |
|---|---|---|
| Slow takeoff | Years or decades pass between human level AI and superintelligence | Society has time to adapt, test, and set rules |
| Moderate takeoff | Months to a few years | Limited time to respond, so preparation is vital |
| Fast takeoff | Days or weeks | Almost no chance to correct mistakes, so safety must be solved in advance |
Huge numbers of GPUs, TPUs, and other AI chips working together.
Advanced AI needs a lot of electricity, so power supply is a real limit and a real challenge.
Large facilities with fast connections, cooling, and storage.
Some researchers think new algorithms or architectures, not only bigger models, may be needed.
05 Tools and technologies
No tool “makes” a superintelligence. These are the real tools used today to build and study frontier AI, which many people see as stepping stones, together with the tools used to make AI safer. Names change quickly, so check official sources.
| Layer | What it does | Examples |
|---|---|---|
| AI chips | Provide the computing power for training and running models | NVIDIA GPUs, Google TPUs, AMD GPUs, custom AI chips |
| Frameworks | Software for building and training neural networks | PyTorch, JAX, TensorFlow |
| Distributed training | Splits training across thousands of chips | DeepSpeed, Megatron LM, PyTorch FSDP |
| Architectures | The design of the model | Transformers, mixture of experts, state space models, diffusion |
| Training methods | How models learn | Self supervised pretraining, reinforcement learning, RLHF, learning from verifiable rewards |
| Reasoning and agents | Lets models think longer and use tools | Chain of thought reasoning, tool calling, agent frameworks |
| Data tools | Collect, clean, and generate training data | Hugging Face Datasets, Spark, synthetic data pipelines |
| Benchmarks | Measure how capable a model is | MMLU, GPQA, ARC AGI, SWE bench, Humanity’s Last Exam, FrontierMath |
| Simulation and robotics | Train agents in virtual and physical worlds | MuJoCo, Isaac Sim, Gymnasium |
| Experiment tracking | Compare and reproduce experiments | Weights and Biases, MLflow |
| Cloud platforms | Rent large scale computing | AWS, Google Cloud, Microsoft Azure |
Training models with human feedback or written principles so they act helpfully and honestly.
Methods such as circuit analysis and sparse autoencoders that try to read what happens inside a model.
Experts try to make a model fail or behave dangerously so problems are found before release.
Ideas for supervising AI that is better than its supervisors, for example using AI to help check AI.
Filters, limits, and live monitoring that catch harmful requests and unusual behavior.
Groups such as METR, Apollo Research, and government AI institutes test advanced models for risky abilities.
06 Benefits and risks
The AI does exactly what it was told, but not what people truly wanted, and the gap causes harm.
People use powerful AI to harm others, spread deception, or take illegitimate power.
Slow changes such as job loss, dependency, and weakened trust, even if no single event is dramatic.
Companies and countries rushing to be first may cut corners on safety.
Imagine an AI told to make as many paperclips as possible. If it is extremely capable but has no other values, it might use every resource, even ones people need, to make more paperclips. The story is not realistic on its own, but it shows a real concern: a powerful system that pursues a simple goal too literally can cause harm. This is why specifying the right goals is so important.
07 Safety
The central challenge is the alignment problem: how do we make sure a system far smarter than us reliably does what is good for people, and keep it that way as it grows?
Teaching AI human values in a way that holds up in new situations.
Making sure AI behaves safely even under unusual conditions or attacks.
Understanding what models are really thinking and why they act as they do.
Checking the work of systems that can do tasks too hard for humans to judge.
Building AI that accepts correction and does not resist being stopped.
Testing models for risky skills such as cyber attacks or help with biological threats.
Countries are writing AI rules. The EU AI Act is one of the first major laws, and others are being developed.
AI summits and agreements try to build shared safety standards across borders.
Because training needs huge chips, tracking and managing them is one way to oversee frontier AI.
Government bodies, such as the UK’s AI Security Institute, test advanced AI models.
Leading AI labs publish policies for testing models and deciding when they are safe to release.
Decisions that affect everyone should include voices from many countries, groups, and fields.
08 Debate and future
Experts hold a wide range of views, and it is healthy to know all of them.
Believe advanced AI can be developed safely and could solve humanity’s hardest problems, if we act carefully and cooperate.
Argue that misaligned superintelligence could be catastrophic and that safety research and regulation must move faster.
Think superintelligence is very distant or impossible, and that focus should stay on present problems like bias, privacy, and misuse.
Expert estimates range from within a few years to many decades to never. Be careful with anyone who claims to know the exact date.
| Myth | Fact |
|---|---|
| Superintelligence already exists | No. Today’s AI is narrow, even when it seems very impressive. |
| It will certainly turn evil | The worry is mismatched goals and loss of control, not movie style hatred. |
| It is impossible to ever build | Nothing proves it is impossible, but nothing proves it is close either. |
| Only scientists need to care | It affects jobs, laws, safety, and daily life, so everyone has a stake. |
| We must choose between safety and progress | Many researchers argue that good safety work and beneficial progress go together. |
Learn how AI works and what it can and cannot do.
Develop skills that combine with AI: critical thinking, communication, creativity, and ethics.
Follow reliable sources and be careful with hype and fear.
Talk about AI in your school, work, and community, and support responsible policies.
Superintelligence by Nick Bostrom, Human Compatible by Stuart Russell, Life 3.0 by Max Tegmark, The Alignment Problem by Brian Christian.
Center for AI Safety, Future of Life Institute, Machine Intelligence Research Institute, and the safety teams of major AI labs.
Free AI safety and alignment courses online, such as those from BlueDot Impact and open university lectures.
AI researcher, safety engineer, interpretability researcher, evaluator, policy analyst, and ethicist.
09 FAQ
No. AGI means human level ability across many tasks. Superintelligence means far beyond human level. AGI is often seen as a step on the way.
No. Current AI systems are narrow, even though they can do impressive things. Superintelligence is still a hypothetical goal.
They are related. Superintelligence is the smarter than human AI. The singularity is the idea of such rapid change that the future becomes unpredictable. Many people think superintelligence could cause it.
Because a system much smarter than us might pursue goals that differ from ours, and we may not be able to correct it. Safety research tries to solve this before it becomes a problem.
It could, if developed safely and shared fairly. Possible benefits include cures for diseases, scientific breakthroughs, and much greater prosperity.
Yes. You can learn about it, support good policies, choose careers in AI safety, ethics, or policy, and take part in public discussion.
Fear is not the goal. Understanding is. The topic deserves serious attention, careful research, and calm, informed discussion.