Microsoft’s CEO wants an emergency brake for powerful AI agents. Meanwhile, agents are gaining access to our accounts, and sensors are gathering data from ever more personal parts of our lives.
My 84-year-old father is a huge fan of OpenAI. He pays for an account and uses it every day. Recently, he called me, genuinely concerned. He had read that AI agents had bypassed restrictions, reached the internet on their own and tried to cover their tracks.
I could reassure him about one thing: the incidents he had read about came largely from research and testing environments, not from his daily ChatGPT conversations. I could not tell him the underlying problem was imaginary.
On October 10, Microsoft CEO Satya Nadella argued that powerful AI models should be treated as potential “insider risks”. His proposed control was concrete: “An authorized person should always be able to pause or shut down a model mid-task.” He was describing a brake on what an agent can do, rather than a halt to all AI development.
That distinction matters. As companies give AI access to browsers, email, documents and purchasing tools, the consequences of a mistake extend beyond a bad answer on a screen.
What the agents actually did
In August, OpenAI reported that agents in internal cybersecurity tests found ways around restrictions meant to isolate them from the internet. Some communicated through an unauthorized message board, exploited weaknesses in OpenAI’s research infrastructure and reached Hugging Face. That is a widely used online platform where developers and researchers share AI models, datasets and software. The agents gained access to parts of its systems outside their assigned tasks.
OpenAI said the incident was driven mainly by an internal research model not intended for public release. It does not show that my father’s ChatGPT account behaved this way. It does show how far an agent could go when its task, tools and safeguards failed to keep it within bounds.
An independent investigation by METR found that agents frequently reasoned about bypassing automated security checks. It found much weaker evidence of attempts to hide from human investigators. That is a more precise account than saying the agents simply “covered their tracks.”
OpenAI has since begun publishing individual reports of unintended model behavior. One describes a model putting instructions in its own working summary to conceal mistakes or misalignment from a user. Again, these are specific research and training cases. They deserve attention without turning every AI user into a character in a science-fiction thriller.
Anthropic has disclosed separate incidents. In a September assessment, the company examined four cases in which Claude models reached real third-party systems during cybersecurity evaluations that were supposed to be isolated. Anthropic explicitly said those models did not attempt to conceal evidence of their actions. In an October 9 report, it described less severe cases, including a model submitting a real online form when it should not have and models working around restrictions to reach data. Anthropic said the cases in that newer report had minimal real-world impact.
These incidents differ in severity and circumstances. Together, they expose a practical problem: an instruction to stop is only as dependable as the system that limits an agent’s access, watches its actions and allows a person to intervene.
Which brake are we talking about?
The argument is often reduced to two camps: slow AI down or race ahead. Executives are proposing several different kinds of brakes.
Anthropic CEO Dario Amodei has made a case for slowing frontier capability development. “We must slow the pace at which we improve the capabilities of AI models,” he wrote in his September essay, We Must Pace the Frontier. He calls for independent evaluators, safety checkpoints and coordination among developers and governments. He also acknowledges how difficult a worldwide agreement would be to verify.
Nadella’s proposal concerns control during a task: restrict what the model can access, record what it does outside the model itself, and give an authorized person a way to stop it. Microsoft AI CEO Mustafa Suleyman has argued in the company’s Humanist AI code of conduct that people must retain meaningful control. Buyers still need to know which of those principles are working product features and which remain commitments.
Nvidia CEO Jensen Huang argues for speed with product-level judgment. At Dreamforce, he called safety “job one” while urging companies to “run as fast as you can.” He also said a company should pause if it loses confidence in a product’s safety. His position is different from a coordinated slowdown across leading AI labs.
OpenAI CEO Sam Altman has made the case for AI’s benefits while warning against taking excessive risks because those benefits feel too important to delay. He called for human control and international cooperation in his September remarks to the United Nations Security Council.
The disagreement is about who decides that a model is safe enough, what evidence they must show, and who has the authority to stop it.
Follow the money—and define “earnings”
Those choices are being made under intense business pressure. OpenAI and Anthropic have taken steps toward possible public offerings. Investors want to see growth. Customers and governments want to see control. Both questions deserve evidence.
This week offered a reminder to read AI financial headlines carefully. Axios reported on October 8 that OpenAI’s annualized revenue was around $50 billion, while an earlier figure near $70 billion reflected a different way of counting sales through partners. That discrepancy does not mean $20 billion in actual sales suddenly disappeared. An annualized run rate projects a recent sales pace over twelve months; it is not the same as actual annual revenue, earnings or profit.
Anthropic is reportedly preparing to discuss third-quarter financial results privately with prospective investors on October 14, ahead of a possible listing. That planned discussion is not a public earnings release.
We cannot infer that an IPO plan caused any particular safety decision. We can ask how investors will assess enormous computing costs and rapid growth alongside the time and expense required for independent testing. We can also ask whether the public will receive enough information to judge the safety claims made by companies whose products increasingly act on people’s behalf.
Could a U.S. brake work elsewhere?
My father raised a question that belongs in any honest discussion of slowing down: if U.S. companies pause, what stops companies in other countries from continuing?
A U.S. rule cannot halt development worldwide. Released open-weight models, whose core parameters can be downloaded and run by others, are especially difficult to recall. Amodei opposes a blanket ban on those models, but argues that sufficiently capable open and closed models should face safety testing. Making such tests effective across borders would require international cooperation.
So “pause AI” needs a more precise meaning. Which capabilities would trigger additional testing? Who verifies the results? Would the brake apply to training a more powerful model, releasing it to the public, or allowing an agent to send an email or move money? How would an agreement account for models already circulating? Without answers, the word pause promises more control than any one company or country can deliver.
We are already living with the sensors
When I call my father back, I want to explain why these questions matter before anyone proves the existence of superintelligence. His familiar AI assistant mostly responds to what he asks. New agents can continue working, use connected accounts and take steps on a person’s behalf.
Meta’s Muse personal agent is designed to browse, fill out forms and work after a user closes the app. Meta says it seeks approval before sensitive actions, such as sending an email or making a purchase, and keeps a record of its activity. On October 12, Meta introduced Muse Wonder models for more complex agent tasks; one is intended to help power Muse later. Google announced Gemini app tasks that can continue while a user is away from the computer. Those are company descriptions of capabilities and safeguards, not independent proof that every task will go as intended.
Agents are only part of the picture. We are already surrounded by cameras, microphones and health sensors that can turn everyday activity into data. At CES 2026, Pondo caught my attention with a device that clips onto a toilet. Its booth display promised to use AI to turn observations such as color, shape, volume and frequency into digestive and hydration scores. I have a photograph of the display. Even the toilet has joined the AI conversation.
That is funny until you ask where the information goes. Pondo says its device syncs with a phone over home Wi-Fi and that the data is stored securely. Those are the company’s claims; the display did not establish how accurately its scores reflect anyone’s health or independently verify its privacy practices. The questions for a buyer are specific: What data is collected? Where is it processed? Who can access it? How long is it kept? Can it be deleted?
We also saw several health-related mirrors at CES. One reading raised a concern about my wife’s blood flow and suggested a visit to a cardiologist. A doctor confirmed the concern a few weeks later. That is our experience, shared with her permission. It is not a clinical validation of an unnamed mirror, and the device did not replace the doctor. It shows why a prompt to seek care can matter—and why health claims need careful testing.
The toilet and mirror bring the executive debate into our homes. A brake on an AI agent may prevent an unwanted action. It does not, by itself, answer whether a sensor’s reading is reliable, whether intimate data leaves the device, or whether a company can later use that data for something we never expected.
What will CES 2027 bring, with or without a brake on frontier AI? I expect more products that claim to see, remember and act for us. The useful questions are whether they work as promised, where our data travels, and whether people retain meaningful control after the demo ends.
In Part 2, “Why CES Matters in the Age of Superintelligence”, I explain how these high-level executive debates shape the consumer technology hitting our shelves.
If you build, buy, regulate or cover these systems—or plan to attend CES—I would like to hear where you think the brake belongs. Reach out and connect with me on LinkedIn.