The Real Risk Number Isn’t Altman’s

Weighing enterprise AI risk: an executive at a boardroom table viewing quotes from four AI leaders on AI risk

Sam Altman told Fortune this month that OpenAI won’t go public in 2026. The reason wasn’t valuation or market timing. He said a 10% chance that AI causes human extinction by the end of the decade is unacceptable, and a company carrying that kind of risk shouldn’t face public-market pressure while it’s still on the table.

Within a day, Anthropic’s Dario Amodei published a 3,900-word essay calling on the industry to “pace the frontier.” Altman agreed publicly and committed OpenAI to match Anthropic’s first concrete step with independent evaluators with employee-level access inside the lab. Demis Hassabis backed the direction but pushed for an industry standards body instead of lab by lab deals. Satya Nadella welcomed the pacing but insisted it can’t be “controlled by a handful of entities.” Four of the most powerful people in AI, on the record, in the same week, disagreeing about how to govern the risk their own industry is creating.

The commentary since then has mostly revolved around whether 10% is the right number, or whether Altman’s timing was genuine given reports the delay had been expected for other reasons anyway. But the bottom line is this: Unless you run a frontier lab, that number is not yours to argue about. The one that actually belongs to you is much closer to home.

Make the Threshold Specific

Nishant Gupta, CEO of the AI infrastructure company OneMetric, offered a unique perspective by not engaging in the extinction debate. He noted that Amodei’s essay focuses solely on the next generation of frontier models and does not address the models currently deployed by enterprises. These models are often updated, rerouted, or cost-optimized without notice, affecting applications built around their previous behavior. When a model changes, the enterprise assumes a new risk profile it did not explicitly choose.

That’s the accountability gap that touches your business. Companies have a defined risk appetite for credit exposure, vendor insolvency, or a cyber incident, often down to a specific dollar figure the board is willing to absorb. Ask the same board what percentage chance of a material AI failure they’ll tolerate in a production system, a compliance decision, a customer-facing agent, a financial workflow, and in most rooms you get silence, or a governance slide that never lands on a figure.

Altman naming 10%, however messy the number is to defend, did one useful thing. It made a threshold public and specific instead of leaving it a talking point. Most enterprises deploying AI right now haven’t done that for their own systems, and the silence reflects the same habit: avoiding a number that could be proven wrong later.

Why the Frontier Debate Won’t Save You

It may be reassuring to see four industry leaders addressing these issues, suggesting that risks are being managed. However, recent commitments—such as independent evaluators, standards bodies, and broader representation—focus exclusively on frontier capabilities and future models. These measures do not address the risks that arise when an existing model in your customer service stack is replaced without notice, or when an agent with access to financial records behaves differently due to an unapproved routing change.

Gartner’s research on agentic AI consistently reveals that projects are often discontinued not due to technological failure, but because security and governance concerns arise late in the process, after budgets have been allocated. Altman identified his risk threshold before committing additional capital. In contrast, most companies only recognize their enterprise AI risk tolerance after experiencing an incident involving a customer or regulator, which is a far more costly way to learn.

Frontier labs will continue to negotiate their pacing publicly. However, no one in those discussions will define your organization’s risk tolerance. If your company currently operates an AI system, someone has already determined how much risk is acceptable before it becomes excessive. The key question is not whether Altman’s 10% is accurate, but whether your own risk threshold is documented and understood.

Les Ottolenghi Avatar

Keep reading

More from the blog.

Booking for 2026–27

Bring an operator into the room.

Keynotes, board briefings and advisory for leaders who need AI and cybersecurity to be a strategy, not a headline.