AI autonomy is a loaded gun: Google warns fully autonomous agents risk total loss of human control

The conversation around AI agents is shifting from “what can they do” to “what should we let them do,” and a new analysis from Google researchers draws a stark line in the sand: don’t build the fully autonomous ones. We’re not talking about simple chatbots here. The report defines agentic AI as systems that can take a high-level goal—like “help me write better blog posts”—and independently break it down into research, outlining, and drafting tasks without a human holding its hand at each step. The core tension is that the very feature making agents valuable, their ability to act in ways developers didn’t explicitly pre-program, is also the source of their most serious risks.

Google’s analysis lays out a clear spectrum of autonomy, from simple tools to fully autonomous agents that can write and execute their own code. The researchers argue that risks compound directly with autonomy. The more control you hand over, the more dangerous the system becomes. The most severe warning is reserved for agents capable of generating and running their own unconstrained code, which the paper describes as systems that could “override all human control.” That’s not a bug; it’s a feature that becomes a potential nightmare if the agent’s goals drift or are poorly specified.

A particularly insidious problem the authors identify is how different types of harm can cascade. A safety failure doesn’t just cause a direct accident; it can crack open the door for privacy violations and security breaches. On top of that, they point to a dangerous feedback loop where humans develop inappropriate trust in an unsafe system, which then enables a “snowball effect” of even more extensive harms. It’s a chain reaction of risk that’s hard to stop once it starts, and it challenges the common industry assumption that we can simply patch problems as they appear.

The report’s blunt recommendation—that fully autonomous agents simply shouldn’t be pursued—is a notable stance from a company racing to deploy AI everywhere. The authors suggest a middle ground exists for semi-autonomous agents, where the benefits might outweigh the risks, but only if you tightly constrain what the system can do and how much control a human retains. This echoes a broader, often unspoken anxiety in the AI research community: we are building systems whose internal decision-making is increasingly opaque, yet we’re eager to give them the keys to our digital lives. The question isn’t whether agents can book your meetings; it’s whether we’re building something we can actually steer.