Make money doing the work you believe in
If Magnifica Humanitas didn’t convince you of the urgent need for some serious thought about the dangers of AI, consider two stories that appeared in the Wall Street Journal this week:
1) Top executives at Anthropic, the current leader in the field, have called for a moratorium on development of AI. “We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable social structures and alignment research to keep up with the advance of technology,” they write.
Is Anthropic really just looking for government intervention to secure its place in the market? Maybe. But the executives give a rational explanation for their proposal. They explain that AI systems are already showing the capacity to “improve” by themselves— in other words, to develop without, and potentially beyond, human intervention.
2) The CEOs of the three top firms in the field— ordinarily fierce competitors— have joined in an appeal to Congress for legislation that would strictly regulate the sale of synthetic DNA or RNA. The danger is that without such controls, someone— or some AI system— could use RNA to develop a virus with who-knows-what properties.
In both stories, the high-tech pioneers are warning us that AI could go rogue. It takes very little imagine to picture the nightmare scenario, with a “self-improving” machine devising a vaccine that would….
The AI executives are begging us— all of us— to help them restrain this powerful new force that they are developing.