A thread through the archive
Essays tagged “decision theory”. Follow an idea across the archive.
A technical dialogue between Scott Alexander and Eliezer Yudkowsky exploring why AI alignment is difficult, covering the inapplicability of human moral development analogies, the nature of AI consequentialism, and skepticism about acausal trade as a safety mechanism.
Scott Alexander speculates on how concepts from decision theory and AI could lead to the emergence of a God-like entity across the multiverse, which judges and potentially rewards human behavior.
Scott argues for the importance of starting AI safety research now, presenting key problems and reasons why early work is crucial.
Scott Alexander examines the concept of infinite debts in personal and societal contexts, proposing a practical approach of paying manageable 'interest' rather than attempting full repayment.
Scott Alexander reflects on five years of Less Wrong, highlighting the community's intellectual progress in various areas of rationality and philosophy.
Scott Alexander rewrites classic fairy tales to illustrate transhumanist and rationalist concepts, each ending with a moral lesson related to these philosophies.
Scott explains backward reasoning in game theory through decision trees, culminating in a counterintuitive example where giving the US president line-item veto power could actually reduce the president's ability to achieve desired outcomes.
Scott introduces the Prisoners' Dilemma, showing how rational reasoning leads two criminals to defect against each other even when mutual cooperation would benefit both, then explores why iterating the dilemma doesn't fully solve the problem.
Scott explains Nash equilibria (outcomes where no player regrets their strategy given the other's choice) and Schelling points (focal points that help coordinate between multiple equilibria), using examples ranging from meeting a partner to military strategy to show how these game theory concepts apply to real-world coordination problems.
Scott analyzes four mechanisms that make slippery slopes real, using examples like Murder-Gandhi and playing Civilization, and proposes 'Schelling fences' as the solution - clear stopping points that people can credibly commit to defending.
Scott argues against negotiating with unconscious desires, distinguishing between behavior-executors (which have context-dependent urges) and utility-maximizers (which have stable preferences), and suggests we mistakenly model ourselves as the latter when we're actually the former.
Scott analyzes why some proposed political solutions feel wrong or useless by examining them as counterfactuals, arguing that valid solutions must involve modifying nodes you can actually influence rather than wishfully changing things you have no power over.
Scott uses Calvinist predestination and Tversky's ice-water heart study to show how people confuse correlation with causation and deceive themselves about their own motives.