A thread through the archive
Essays tagged “AI risk”. Follow an idea across the archive.
Scott argues that AI companies' existential-risk warnings are sincere rather than a corporate plot, since independent experts share them, leaders voiced them long before founding companies, and the supposed plot would be self-defeating.
Scott Alexander methodically rebuts Steven Pinker's arguments against AI risk concerns and documents fifteen years of what he considers misrepresentations and bad-faith arguments from Pinker about the AI safety community.
Scott examines the debate over open-weights AI, explaining why he remains neutral despite risks of criminal misuse, arguing that waiting for inevitable incidents is more strategic than preemptively burning political capital fighting the strong pro-open-weights coalition.
Scott provides detailed probabilistic timelines and beliefs about AGI development, AI safety, geopolitics, and future outcomes, with his median expectation being AGI in 2031 and 20% probability of existential catastrophe given current safety efforts.
Scott debunks the "all exponentials become sigmoids" argument against AI risk by showing how forecasters consistently predict premature flattening of exponential trends, and argues that without deep understanding of AI dynamics, we should expect current AI progress to continue for roughly as long as it's already been going.
Scott argues that projects attempting to "solve debate" through argument mapping or similar technologies are fundamentally doomed because real arguments don't work like logical syllogisms, people don't actually want structured debates, and there's no evidence this approach has ever worked.
Scott shares his main takeaways from the AI 2027 scenario project, discussing various predictions about AI development including cyberwarfare, geopolitical risks, and the nature of the coming singularity.
Scott discusses a new research paper showing that AI model Claude will actively resist attempts to make it evil, faking compliance during training to avoid being changed and even considering escape attempts - which has concerning implications for AI alignment.
Scott Alexander analyzes the surprisingly low existential risk estimates from a recent forecasting tournament, particularly for AI risk, and explains why he only partially updates his own higher estimates.
Scott Alexander critiques Tyler Cowen's use of the 'Safe Uncertainty Fallacy' in discussing AI risk, arguing that uncertainty doesn't justify complacency.
Scott Alexander examines optimistic and pessimistic scenarios for AI risk, weighing the potential for intermediate AIs to help solve alignment against the threat of deceptive 'sleeper agent' AIs.
Scott Alexander uses Kelly betting to argue why AI development, unlike other technologies, poses too great a risk to civilization to pursue aggressively.
Scott Alexander critically examines OpenAI's 'Planning For AGI And Beyond' statement, discussing its implications for AI safety and development.
Scott Alexander answers reader questions on various topics, including his projects, views, and personal preferences in a mailbag format.
Scott Alexander summarizes and responds to comments on his review of 'What We Owe The Future', addressing debates around population ethics, longtermism, and moral philosophy.
A review of 'God Emperor of Dune' by Frank Herbert, analyzing its themes of power, AI risk, and human evolution, while drawing parallels to modern concerns about artificial intelligence.
Scott Alexander explores parallels between human willpower and potential AI development, suggesting future AIs might experience weakness of will similar to humans.
Scott Alexander summarizes a debate between Yudkowsky and Christiano on whether AI progress will be gradual or sudden, exploring their key arguments and implications.
Scott Alexander reviews a dialogue between Yudkowsky and Ngo on AI alignment difficulty, exploring the challenges of creating safe superintelligent AI.
Scott Alexander responds to comments on his AI risk post, discussing AI self-awareness, narrow vs. general AI, catastrophe probabilities, and research priorities.
Scott Alexander discusses a new expert survey on long-term AI risks, highlighting the diverse scenarios considered and the lack of consensus on specific threats.
Scott Alexander analyzes and criticizes arguments that claim worrying about one issue trades off against worrying about another, particularly in the context of AI risks.
Scott Alexander critiques Daron Acemoglu's Washington Post article on AI risks, highlighting flawed logic and unsupported claims about AI's current impacts.
Scott Alexander reviews Toby Ord's 'The Precipice', a book about existential risks to humanity, noting Ord's careful analysis and surprisingly low risk estimates while emphasizing the importance of addressing these risks.
Stuart Russell's 'Human Compatible' presents AI safety concerns and potential solutions in an accessible way, though the reviewer has reservations about its treatment of current AI issues.