Make money doing the work you believe in
I like the fact that you imagine the incident report, “The software's decision cannot be reconstructed”. Of course, you also mean that the 11 people who were killed were unable to anticipate the software decision. That situation never arises in a world of public strategy optimizers—the current grandmaster strategy becomes as pubic as the time according to atomic clocks—so any AI can know exactly who the weapons will kill. The issues are (1) Regulation of autonomous weapons is the beginning of a wise effort to reduce lethal force generally, and (2) Anyone who acts without AI approval would risk death when policy reforms so rapidly that only AI can track it. Should we all be reduced to children, required to hold the hand of AI when crossing the street? Maybe, but it would still be ideal to make streets non-lethal.
