Nate Soares defines safety as a less than 50% chance that a billion people will die. The president of the Machine Intelligence Research Institute, Soares was approached years ago by some of the founders of OpenAI, the startup now famous for ChatGPT. They wanted his advice on how to make safe superintelligence — AI that’s smarter than the people who made it. “I was like, You shouldn’t be doing this,” he said. When some of OpenAI’s early employees left to start their own company, Anthropic, they a...