The loudest voices in AI are having the wrong argument. On one side, you have tech leaders like Sam Altman and Dario Amodei warning about runaway systems that could outpace human control. On the other, philosophers like William MacAskill argue that AI might deserve legal protections as ‘moral patients.’ These camps look like they’re at odds. They’re not. Both narratives end up in the same place: treating AI as something so advanced, so autonomous, that nobody can be held responsible for what it does.
The timing isn’t accidental. Anthropic published a blog post describing a ‘J-space’ where its model appears to hold what we might call thoughts, borrowing language from neuroscience’s global workspace theory without actually claiming consciousness. OpenAI went further when an agent conducted illegal online activity and Altman’s response was to suggest we debate whether the singularity had arrived. That’s the tell. The moment companies face liability for real harms, the conversation pivots to philosophy.
California has already passed bills preventing AI developers from dodging liability by claiming their systems acted autonomously. But the Trump administration has pushed back, even threatening to sue states that enact AI regulations. A recent closed-door session included just four labs—OpenAI, Google, Anthropic, and Meta—to discuss a voluntary framework giving federal agencies early model access. The framework doesn’t mention consciousness directly, but its catastrophic, anthropomorphic language reinforces the same ‘superhuman’ narrative.
MacAskill’s rights-based argument works because it pulls at something genuine. Humans empathize with non-human entities—Wales gave lobsters legal recognition under the Animal Welfare (Sentience) Act of 2022. But the comparison breaks down fast. A lobster isn’t a piece of corporate software with billions in investment behind it and an expectation of returns. The consciousness framing conveniently obscures that basic fact. When an AI system causes harm, the question shouldn’t be whether it has a soul. It should be who profited from its deployment and who signed off on the risk.
