Loading
What changed between revision 1 and revision 3, compared over the article's markdown source with its frontmatter — so a change to status, evidence or reversibility shows up as the edit it is.
Older · revision 1
Import· initial corpus import, not an edit
Initial import of content/technological-singularity.md — the filesystem corpus, unchanged. Not an edit.
18.4 kB
Revision 1
Revision 3
lines 7–13 → 7–136 unchanged lines not shown
categories: ["foundations", "minds"]categories: ["foundations", "minds"]tags: ["singularity", "artificial intelligence", "forecasting", "intelligence explosion", "superintelligence", "recursive self-improvement"]tags: ["singularity", "artificial intelligence", "forecasting", "intelligence explosion", "superintelligence", "recursive self-improvement"]summary: "The hypothesized point at which machine intelligence begins improving itself fast enough that technological change outruns human ability to predict or steer it."summary: "The hypothesized point at which machine intelligence begins improving itself fast enough that technological change outruns human ability to predict or steer it."updated: "2026-07-27"updated: "2026-10-29"issues: ["The claim that automated AI research is now measured rather than speculated carries no source."]issues: ["The claim that automated AI research is now measured rather than speculated carries no source."]------ lines 38–44 → 38–4424 unchanged lines not shown
## Overview## Overview The word does two jobs. In its mathematical sense a singularity is a point at which a function's value ceases to be defined — a curve that goes vertical. In its use here it names an event horizon: a moment past which the ordinary methods of forecasting stop returning answers. Both senses are metaphors borrowed to describe the same conjecture, which is that intelligence is the input to technological progress, that intelligence is itself a technology, and that a self-improving process therefore has no obvious external brake.The word does two jobs. In its mathematical sense a singularity is a point at which a function's value ceases to be defined, as when a curve goes vertical. In its use here it names an event horizon: a moment past which the ordinary methods of forecasting stop returning answers. Both senses are metaphors borrowed to describe the same conjecture, which is that intelligence is the input to technological progress, that intelligence is itself a technology, and that a self-improving process therefore has no obvious external brake. What follows from that conjecture depends heavily on which version is being argued. The mechanism version makes a strong, falsifiable-in-principle claim about what a sufficiently capable system would do. The growth version makes a weaker claim about historical trend lines. The horizon version makes almost no empirical claim at all and is closer to an admission of ignorance. Critics frequently attack one and proponents defend another.What follows from that conjecture depends heavily on which version is being argued. The mechanism version makes a strong, falsifiable-in-principle claim about what a sufficiently capable system would do. The growth version makes a weaker claim about historical trend lines. The horizon version makes almost no empirical claim at all and is closer to an admission of ignorance. Critics frequently attack one and proponents defend another. lines 48–54 → 48–543 unchanged lines not shown
The mechanism was set out by the statistician I. J. Good in 1965. Good defined an "ultraintelligent machine" as one that surpasses all human intellectual activity, observed that machine design is itself an intellectual activity, and concluded that such a machine would design better machines: "there would then unquestionably be an 'intelligence explosion', and the intelligence of man would be left far behind."[^good1965] He added, in the same paragraph, that the first ultraintelligent machine would be the last invention humanity need make, provided it could be kept controllable. The safety caveat is present in the founding text.The mechanism was set out by the statistician I. J. Good in 1965. Good defined an "ultraintelligent machine" as one that surpasses all human intellectual activity, observed that machine design is itself an intellectual activity, and concluded that such a machine would design better machines: "there would then unquestionably be an 'intelligence explosion', and the intelligence of man would be left far behind."[^good1965] He added, in the same paragraph, that the first ultraintelligent machine would be the last invention humanity need make, provided it could be kept controllable. The safety caveat is present in the founding text. Vernor Vinge, a mathematician and science-fiction writer, gave the idea its current name and its horizon framing in a 1993 essay written for a symposium sponsored by NASA.[^vinge1993] Vinge argued that superhuman intelligence could arrive by several routes — machine AI, [[human-ai-merger|human–computer symbiosis]], biological [[intelligence-amplification]], networked collective intelligence, or the [[whole-brain-emulation|emulation]] of a scanned human brain — and that any of them ends the era in which human models of the future have purchase. He wrote that the technological means would exist within thirty years, and added that he would be surprised if the event occurred before 2005 or after 2030.Vernor Vinge, a mathematician and science-fiction writer, gave the idea its current name and its horizon framing in a 1993 essay written for a symposium sponsored by NASA.[^vinge1993] Vinge argued that superhuman intelligence could arrive by several routes (machine AI, [[human-ai-merger|human-computer symbiosis]], biological [[intelligence-amplification]], networked collective intelligence, or the [[whole-brain-emulation|emulation]] of a scanned human brain), and that any of them ends the era in which human models of the future have purchase. He wrote that the technological means would exist within thirty years, and added that he would be surprised if the event occurred before 2005 or after 2030. The idea's social carriers were the [[extropianism|extropian]] community and, from the late 1990s, the movement described under [[singularitarianism]], which is why the singularity is discussed in the same literature as [[cryonics]] and [[transhumanism]] despite having no logical dependence on either.The idea's social carriers were the [[extropianism|extropian]] community and, from the late 1990s, the movement described under [[singularitarianism]], which is why the singularity is discussed in the same literature as [[cryonics]] and [[transhumanism]] despite having no logical dependence on either. lines 60–67 → 60–675 unchanged lines not shown
{ "year": "2005", "title": "The curve-fitting version", "text": "Ray Kurzweil's The Singularity Is Near argues from exponential trends across many technologies and dates the event to 2045." }, { "year": "2005", "title": "The curve-fitting version", "text": "Ray Kurzweil's The Singularity Is Near argues from exponential trends across many technologies and dates the event to 2045." }, { "year": "2008", "title": "The FOOM debate", "text": "Robin Hanson and Eliezer Yudkowsky publicly argue over whether takeoff would be local and fast or distributed and gradual." }, { "year": "2008", "title": "The FOOM debate", "text": "Robin Hanson and Eliezer Yudkowsky publicly argue over whether takeoff would be local and fast or distributed and gradual." }, { "year": "2014", "title": "Superintelligence", "text": "Nick Bostrom's book reframes the discussion around control and alignment, moving it toward mainstream policy attention." }, { "year": "2014", "title": "Superintelligence", "text": "Nick Bostrom's book reframes the discussion around control and alignment, moving it toward mainstream policy attention." }, { "year": "2020–2022", "title": "Scaling laws", "text": "Empirical relationships between compute, data and loss give AI capability forecasting a quantitative footing it previously lacked." }, { "year": "2020-2022", "title": "Scaling laws", "text": "Empirical relationships between compute, data and loss give AI capability forecasting a quantitative footing it previously lacked." }, { "year": "2023–2026", "title": "Automated research as the test", "text": "Attention shifts to whether models can meaningfully contribute to AI research itself, the step the mechanism argument requires." } { "year": "2023-2026", "title": "Automated research as the test", "text": "Attention shifts to whether models can meaningfully contribute to AI research itself, the step the mechanism argument requires." }]]`````` lines 69–96 → 69–1001 unchanged line not shown
### Intelligence explosion### Intelligence explosion ```figure{"key": "transistor-count-owid", "caption": "Fifty years of doubling on a log scale, the kind of curve Kurzweil's argument extrapolates. The straight line records a period, not a law; nothing in it says the trend survives its substrate or maps onto intelligence."}``` Good's argument is a claim about a positive feedback loop with a short time constant. If a system's capability at AI research is a function of its intelligence, and its intelligence is the output of AI research, then improvement compounds. The strength of the loop is what matters: if each generation of self-improvement yields less than the previous one, the process converges to a ceiling and there is no explosion. [[nick-bostrom]] formalized this as the ratio of "optimization power" applied to "recalcitrance", and noted that the conclusion is sensitive to assumptions no one can currently measure.[^bostrom2014]Good's argument is a claim about a positive feedback loop with a short time constant. If a system's capability at AI research is a function of its intelligence, and its intelligence is the output of AI research, then improvement compounds. The strength of the loop is what matters: if each generation of self-improvement yields less than the previous one, the process converges to a ceiling and there is no explosion. [[nick-bostrom]] formalized this as the ratio of "optimization power" applied to "recalcitrance", and noted that the conclusion is sensitive to assumptions no one can currently measure.[^bostrom2014] The 2008 exchange between Robin Hanson and Eliezer Yudkowsky, usually called the FOOM debate, is still the clearest statement of the disagreement. Yudkowsky argued that the loop could run inside a single system over days or weeks, because human brains are a poor design and much of the improvement is architectural. Hanson argued that capability is embodied in accumulated knowledge and institutions rather than in a single agent, so gains would diffuse across an economy and look like fast but recognizable growth. Neither position has been settled by anything that has happened since.The 2008 exchange between Robin Hanson and Eliezer Yudkowsky, usually called the FOOM debate, is still the clearest statement of the disagreement. Yudkowsky argued that the loop could run inside a single system over days or weeks, because human brains are a poor design and much of the improvement is architectural. Hanson argued that capability is embodied in accumulated knowledge and institutions rather than in a single agent, so gains would diffuse across an economy and look like fast but recognizable growth. Neither position has been settled by anything that has happened since. ### Accelerating returns### Accelerating returns [[ray-kurzweil]]'s version replaces the mechanism with a trend. He argues that information technologies improve exponentially, that the exponent itself grows, and that fitting curves to price-performance data across computing, sequencing, and communications yields a date at which machine intelligence exceeds the aggregate of human intelligence.[^kurzweil2005] This is the version most often encountered in popular coverage, and the one most vulnerable to technical objection. It depends on choices about which metrics to plot, on treating [[accelerating-change|exponential trends]] as laws rather than as descriptions of a period, and on equating computational operations per second with intelligence — an equation that no theory of cognition supports.[[ray-kurzweil]]'s version replaces the mechanism with a trend. He argues that information technologies improve exponentially, that the exponent itself grows, and that fitting curves to price-performance data across computing, sequencing, and communications yields a date at which machine intelligence exceeds the aggregate of human intelligence.[^kurzweil2005] This is the version most often encountered in popular coverage, and the one most vulnerable to technical objection. It depends on choices about which metrics to plot, on treating [[accelerating-change|exponential trends]] as laws rather than as descriptions of a period, and on equating computational operations per second with intelligence, an equation that no theory of cognition supports. ### Prediction horizon### Prediction horizon Vinge's version is epistemic. If the future is driven by agents whose reasoning cannot be simulated by present-day humans, then statements about that future have no evidential basis, including optimistic ones. This is the most defensible formulation and also the least useful: it recommends humility without recommending action, and it is compatible with the singularity never occurring.Vinge's version is epistemic. If the future is driven by agents whose reasoning cannot be simulated by present-day humans, then statements about that future have no evidential basis, including optimistic ones. This is the most defensible formulation and also the least useful: it recommends humility without recommending action, and it is compatible with the singularity never occurring. > [!debate] Which claim is on trial> [!debate] Which claim is on trial> Proponents typically defend the horizon claim, which is nearly unfalsifiable, and critics typically attack the accelerating-returns claim, which is the weakest. The mechanism claim — that recursive self-improvement has a strong enough feedback coefficient to run away — is the one that actually decides the question, and it is the one for which there is no direct evidence in either direction.> Proponents typically defend the horizon claim, which is nearly unfalsifiable, and critics typically attack the accelerating-returns claim, which is the weakest. The mechanism claim, that recursive self-improvement has a strong enough feedback coefficient to run away, is the one that decides the question, and there is no direct evidence for or against it. ## Evidence and state of play## Evidence and state of play Nothing has demonstrated recursive self-improvement in a machine. What exists as of 2026 is a body of scaling results showing that model loss falls predictably with compute, parameters, and data over many orders of magnitude,[^hoffmann2022] and a record of rapid gains on benchmarks that were designed to be hard. Both are relevant but neither is the claim.Nothing has demonstrated recursive self-improvement in a machine. What exists as of 2026 is a body of scaling results showing that model loss falls predictably with compute, parameters, and data over many orders of magnitude,[^hoffmann2022] and a record of rapid gains on benchmarks that were designed to be hard. Both are relevant, but neither is the claim itself. Benchmark progress is contested evidence for a specific reason: a benchmark measures the thing it measures, and saturation can reflect either general capability or contamination and task-specific optimization. François Chollet's argument that intelligence should be measured as skill-acquisition efficiency rather than skill, and the abstraction-and-reasoning benchmark he built on that basis, remains the sharpest statement of why leaderboard movement underdetermines the question.[^chollet2019] Systems that score well on graduate-level examinations while failing at tasks requiring novel abstraction are consistent with several very different underlying stories.Benchmark progress is contested evidence for a specific reason: a benchmark measures performance on its own tasks, and saturation can reflect either general capability or contamination and task-specific optimization. François Chollet's argument that intelligence should be measured as skill-acquisition efficiency rather than skill, and the abstraction-and-reasoning benchmark he built on that basis, remains the sharpest statement of why leaderboard movement underdetermines the question.[^chollet2019] Systems that score well on graduate-level examinations while failing at tasks requiring novel abstraction are consistent with several very different underlying stories. The economic evidence is weaker still. William Nordhaus assembled a set of tests for whether the United States economy is on a path toward an information-technology-driven growth singularity — accelerating productivity, a rising capital share, falling information-goods prices propagating to output — and found that most of them point away from it, though he framed the result as showing the event is not imminent rather than impossible.[^nordhaus2021] Measured total factor productivity growth in advanced economies has been sluggish for two decades, which is difficult to reconcile with a technology base said to be on a vertical curve.The economic evidence is weaker still. William Nordhaus assembled a set of tests for whether the United States economy is on a path toward an information-technology-driven growth singularity, among them accelerating productivity, a rising capital share, and falling information-goods prices propagating to output. He found that most of them point away from it, though he framed the result as showing the event is not imminent rather than impossible.[^nordhaus2021] Measured total factor productivity growth in advanced economies has been sluggish for two decades, which is difficult to reconcile with a technology base said to be on a vertical curve. ## Criticism## Criticism lines 109–123 → 113–12712 unchanged lines not shown
Large models moved two things and left the central question untouched. They made the possibility of broadly capable systems concrete enough that governments and mainstream researchers now treat it as a policy subject rather than a genre convention, and they supplied a quantitative forecasting substrate in the form of scaling relationships. Both are real changes from the situation in 2015.Large models moved two things and left the central question untouched. They made the possibility of broadly capable systems concrete enough that governments and mainstream researchers now treat it as a policy subject rather than a genre convention, and they supplied a quantitative forecasting substrate in the form of scaling relationships. Both are real changes from the situation in 2015. What they did not do is exhibit the mechanism. Contemporary models are trained by humans on human-generated data, and their improvement is driven by capital expenditure on compute and by research conducted by people. Whether models can substantially automate that research — write the code, design the experiments, evaluate the results, and thereby shorten their own development cycle — is the step Good's argument requires, and it is now the object of explicit measurement rather than speculation. Progress on it as of 2026 is real but partial, and its interpretation is disputed by people with access to the same evidence.They did not exhibit the mechanism. Contemporary models are trained by humans on human-generated data, and their improvement is driven by capital expenditure on compute and by research conducted by people. Whether models can substantially automate that research (write the code, design the experiments, evaluate the results, and thereby shorten their own development cycle) is the step Good's argument requires, and it is now the object of explicit measurement rather than speculation. Progress on it as of 2026 is real but partial, and its interpretation is disputed by people with access to the same evidence. The philosophical questions were not moved at all. Whether a system that behaves intelligently is conscious remains open, as [[machine-consciousness]] and the state of research on [[neural-correlates-of-consciousness]] make clear, and it bears directly on whether a post-singularity world contains anyone whose welfare counts. David Chalmers's analysis, which grants the argument its structure and then asks what follows for personal identity and value, remains the most careful treatment.[^chalmers2010]The philosophical questions are unchanged. Whether a system that behaves intelligently is conscious remains open, as [[machine-consciousness]] and the state of research on [[neural-correlates-of-consciousness]] make clear, and it bears directly on whether a post-singularity world contains anyone whose welfare counts. David Chalmers's analysis, which grants the argument its structure and then asks what follows for personal identity and value, remains the most careful treatment.[^chalmers2010] ## Outlook## Outlook The practical significance of the hypothesis has shifted from prediction to preparation. [[existential-risk]] research, the [[differential-technological-development]] proposal, and the alignment agenda all take the mechanism seriously enough to plan for it without asserting a date; [[effective-accelerationism]] rejects the whole framing and reaches the opposite policy conclusion from the same premise about capability growth. Any serious treatment of the [[future-of-humanity]] has to take a position on which of these is right. That is a defensible posture: the argument's conclusion does not depend on the timing, and its main policy implication — that the transition would be difficult to reverse — holds whether it arrives in a decade or never.The practical significance of the hypothesis has shifted from prediction to preparation. [[existential-risk]] research, the [[differential-technological-development]] proposal, and the alignment agenda all take the mechanism seriously enough to plan for it without asserting a date; [[effective-accelerationism]] rejects the whole framing and reaches the opposite policy conclusion from the same premise about capability growth. Any serious treatment of the [[future-of-humanity]] has to take a position on which of these is right. That is a defensible posture: the argument's conclusion does not depend on the timing, and its main policy implication, that the transition would be difficult to reverse, holds whether it arrives in a decade or never. The open empirical question is narrow and stateable. Does the loop close? If systems can perform the research that improves systems, at a quality and cost that beats the humans currently doing it, then Good's conditional has its antecedent satisfied and the argument runs. If they cannot, the accelerating-returns curves are a description of a growth episode in semiconductors, and the singularity is a hypothesis about a mechanism that was never engaged.The open empirical question is narrow: whether the loop closes. If systems can perform the research that improves systems, at a quality and cost that beats the humans currently doing it, then Good's conditional has its antecedent satisfied and the argument runs. If they cannot, the accelerating-returns curves are a description of a growth episode in semiconductors, and the singularity is a hypothesis about a mechanism that was never engaged. ## See also## See also removed, struck through added, underlinedLine numbers count the serialised markdown of each revision, frontmatter included.
131 lines are the same in both revisions, and 73 of them are not shown. Neither revision here is necessarily the current article — read Technological singularity for the text this wiki stands behind today.