A software agent that modifies its own prompts based on past performance is cheap to roll back. A robot that modifies its own motion primitives based on past performance is not.

The feedback loop that makes software self-improvement fast and cheap makes physical self-improvement fast and potentially dangerous.

A self-modified motion primitive that reduces success rate or increases collision probability can cause damage before the modification is detected. Software's iterate-rollback-iterate cycle does not exist at physical stakes.

Physical self-improvement systems already exist. Agents that tune grasping strategies. Agents that update environment models as they operate. Agents that refine motion primitives through reinforcement learning. All of them sit within carefully engineered safety envelopes.

The envelope is doing the work the rollback used to do.