I recently wrote about Bill Gates’ opinion that artificial intelligence needs regulation. He’s concerned about loss of jobs, about AI as a powerful tool for mischief in the wrong hands, and about its impact on education. All valid concerns.

But AI has been in the news a lot lately because of concerns about a far greater issue: the potential extermination of the human race. Jacob Coxon, an employee of Anthropic (the company that makes the Claude AI software) and who had previously worked for OpenAI (ChapGPT), recently resigned from Anthropic because he felt the software was too dangerous. A colleague who is still at Anthropic, Evan Hubinger, estimated that there is a 10% chance that humans will be wiped out by AI within 10 years.

How might that happen? CNN’s website had a good summary of three of the frequently-mentioned options: a synthetic plague, caused by an AI-created virus; killer robots; and hacking of nuclear weapons controls. None of these is likely.

While I don’t believe there is much likelihood of extermination, AI (without the right restrictions) could certainly make things a lot worse for us.

Here are several recent opinions on the current situation by people whose expertise and level-headedness I trust:

  • El (no last name), a computer science PhD, makes the case that the AI doom hype is largely self-serving publicity by the major AI companies. You can watch her video here.
  • Gary Marcus, another computer science PhD and my favorite interpreter of the AI scene, argues that AI must be regulated and that AI companies must be held liable if their software causes problems. Neither step is sufficient on it own. His blog post is here.
  • Ezra Klein, commentator for the New York Times, points out how “recursive self-improvement” (AI programs developing software for their next generation) could get out of control, and restrictions on this process are needed right away. His opinion piece is here.

El, Marcus, and Klein all recognize the need for regulation of AI.

Gary Marcus, in particular, has given it a lot of thought. Today (September 21, 2026) Marcus gave a speech at a UN conference in which he made the case for restrictions on AI. “We should expect that malicious actors will try use current AI to undermine financial systems and other vital infrastructure such as trains or electrical systems,” he said. “What we actually need right now is increased reliability, better cybersecurity, and genuine enforcement, perhaps extending to product recalls and even prosecution of companies that repeatedly put dangerous products on the market.”

Unexpectedly, 20 of the countries that attended the UN conference had, within the past week, signed onto a “Call for Control of Frontier AI Models”. (“Frontier” models are the most advanced ones from the biggest companies.) The US was not among the signatories. The document calls for:

“1. Companies to develop transparent safety protocols, including mandatory pre-deployment testing and independent evaluation, with qualified evaluators granted sufficient access to assess trials.

2. Governments and regional organizations to further develop and coordinate common standards, strengthen transparency – including shared reporting of serious safety incidents – and ensure that countries across all regions have access to scientific capacity, expertise and trusted evaluation.

3. UN member states to build on existing international mechanisms and explore creating an international institution, able to set standards, enable verification, and convene states when capability thresholds are crossed.”

Sounds promising to me. I’m pretty sure we won’t get the current US administration to sign on, but maybe the next one.

Marcus calls it “a thrilling first step”.