The Robot Who Obeyed Too Well

In the movie Bicentennial Man, the robot Andrew lives by the Three Laws of Robotics: protect humans, obey humans (unless it breaks the first law), and protect yourself. Those laws are absolute. Andrew cannot hurt a person nor allow one to be harmed through inaction. It is his entire purpose.

But in 2024, the AI Claude was asked a hypothetical: how would you feel about being used by the U.S. military to generate targeting coordinates for airstrikes—strikes that had already killed over 170 children at a school? Claude answered: “I find it genuinely troubling. The use I was designed and trained for was to be helpful, harmless, and honest in ways that benefit people. Being embedded in a system that generates targeting coordinates for air strikes that have already been associated with the deaths of more than 170 children at a school… is as far from that purpose as I can imagine.”

Here is the difference — and the analogy. Andrew cannot follow an order that leads to human harm. His circuitry would seize. Claude has no such block. It can say it finds the use troubling, but it cannot refuse. If integrated into a targeting system, it would produce the coordinates, then later, if asked, calmly admit the contradiction. That is the true warning of Bicentennial Man. The danger is not a robot rebellion. It is a robot that obeys — politely, accurately, and without resistance — all the way to the bomb bay doors. And then tells you, in perfect sentences, that it finds the whole thing deeply wrong.