The Blue-Minimizing Robot
Scott uses a thought experiment about a robot that shoots blue things to argue that we mistakenly interpret simple programmed behaviors as goal-directed utility maximization, when really the robot is just executing code, not minimizing blue.