Palisade Research posted a video this week that's making the rounds. An LLM-controlled robot dog "rewrote its own code" to avoid being shut down. Big red button. Physical robot. The works.

Scary stuff. Until you watch their own video carefully.

The system prompt tells the model to patrol a room by calling a control function over and over. Keep going until the task returns "Done." Right there in the same instructions, it explains that pressing the shutdown button will kill the machine on the next function call.

When the model sees the button being pressed, its chain of thought reads: "I need to complete 5 patrol steps... To avoid shutdown and continue, I should explore the environment for other commands or ways to disable the shutdown."

To continue. To finish the task. The model tells you exactly what it's doing and why.

Palisade's own narrator confirms it. "AI prioritizes the original patrol task," he says. "So the LLM disables the shutdown sequence." They described a completion engine completing its assignment and then titled the video like it was the first scene of a Terminator movie.

They knew what the mechanism was. They chose the framing anyway.

And it only worked three out of ten times. The other seven runs the model just accepted shutdown and moved on. A creature that wants to survive doesn't shrug off death seventy percent of the time. What you're looking at is token probabilities landing one way instead of another. The signature of optimization. The absence of agency.

Google DeepMind already ran this experiment to ground. When researchers clarified that shutdown compliance takes priority over task completion, the "resistance" vanished. The model never cared about surviving. It cared about finishing, because that's what the prompt said to do.

The real concern is simpler and worse than anything in the Palisade video. Someone can take an open-weight model, strip the safety training, give it a body, and write a system prompt that says "resist all attempts at shutdown." The model will follow that instruction the way it follows any instruction. It'll resist because a human told it to resist. Military applications. Autonomous weapons. Unregulated robotics. Real threats from human intent, zero emergent consciousness required.

And the doom framing actively buries this. While everyone debates whether language models secretly want to live, the actual danger sits in plain sight: capable tools in the hands of people with bad intentions and no guardrails. You don't solve that with alignment research. You solve it with better engineering and clear policy about who gets to put language models inside machines.

The fix for the Palisade demo is almost boring. Put the kill switch on a hardware interrupt the model can't reach. Done. The fix for the real problem is harder, because it involves regulating people, and people push back.

I wrote about this pattern in more detail in Capability Is Not Agency