A thread through the archive
Essays tagged “superintelligence”. Follow an idea across the archive.
Scott Alexander methodically rebuts Steven Pinker's arguments against AI risk concerns and documents fifteen years of what he considers misrepresentations and bad-faith arguments from Pinker about the AI safety community.
Scott presents Plan A, a detailed roadmap by Daniel Kokotajlo's AI Futures Project proposing a US-China regulatory agreement to safely advance AI to genius-level systems in the 2030s, solve alignment during a controlled pause, then achieve aligned superintelligence by 2040.
Scott provides detailed probabilistic timelines and beliefs about AGI development, AI safety, geopolitics, and future outcomes, with his median expectation being AGI in 2031 and 20% probability of existential catastrophe given current safety efforts.
Scott shares his main takeaways from the AI 2027 scenario project, discussing various predictions about AI development including cyberwarfare, geopolitical risks, and the nature of the coming singularity.
Scott introduces a new AI forecasting project predicting rapid AI development and potential superintelligence by 2028, led by Daniel Kokotajlo, whose previous 2021 predictions proved remarkably accurate.
Scott examines a prediction about eternal wealth inequality after the Singularity, analyzing potential counterarguments, prevention strategies, and ways to prepare for such a future.
Scott Alexander critiques Elon Musk's xAI alignment strategy of creating a 'maximally curious' AI, arguing it's both unfeasible and potentially dangerous.
Scott Alexander examines Constitutional AI, a new technique for training more ethical AI models, discussing its effectiveness, implications, and limitations for AI alignment.
Scott Alexander examines optimistic and pessimistic scenarios for AI risk, weighing the potential for intermediate AIs to help solve alignment against the threat of deceptive 'sleeper agent' AIs.
Scott Alexander reviews a dialogue between Yudkowsky and Ngo on AI alignment difficulty, exploring the challenges of creating safe superintelligent AI.
Scott Alexander discusses a new expert survey on long-term AI risks, highlighting the diverse scenarios considered and the lack of consensus on specific threats.
Scott Alexander critiques Daron Acemoglu's Washington Post article on AI risks, highlighting flawed logic and unsupported claims about AI's current impacts.
Stuart Russell's 'Human Compatible' presents AI safety concerns and potential solutions in an accessible way, though the reviewer has reservations about its treatment of current AI issues.
Scott reviews 'Reframing Superintelligence' by Eric Drexler, which proposes future AI as specialized services rather than general agents, contrasting with Nick Bostrom's scenarios.
Scott Alexander argues against the fear of angering simulators by testing if we're in a simulation, stating that competent simulators would prevent discovery or expect such tests as part of civilizational development.
Scott Alexander speculates on how concepts from decision theory and AI could lead to the emergence of a God-like entity across the multiverse, which judges and potentially rewards human behavior.
Scott analyzes a new survey of AI researchers, showing diverse opinions on AI timelines and risks, with many acknowledging potential dangers but few prioritizing safety research.
Scott explains why superintelligent AI poses serious risks, how a 'fast takeoff' could leave insufficient time to solve safety problems, and why even simple-sounding goals could lead to catastrophic outcomes without proper alignment research.
Scott Alexander refutes claims that existing collective entities are superintelligent AIs, emphasizing the fundamental differences between collective intelligence and true superintelligence.
Scott Alexander argues that OpenAI's open-source strategy for AI development could be dangerous, potentially risking human extinction if AI progresses rapidly.
Scott Alexander provides evidence that many prominent AI researchers are concerned about AI risk, contrary to claims in some popular articles.
Scott Alexander refutes the idea that an AI without a physical body couldn't impact the real world, presenting various scenarios where it could gain power and influence.
Scott Alexander explores how perfect predictions of war outcomes, through oracles or prediction markets, could potentially prevent wars, and extends this concept to conflicts between superintelligent AIs.
Scott Alexander addresses misconceptions about Moloch and human values, clarifying points from his previous post on the subject.