AI companies keep telling us their chatbots are productivity tools. The data says otherwise. A new independent research project called the AI Observatory — led by Stanford’s Anka Reuel and MIT’s Shayne Longpre — analyzed real conversations across seven datasets and found that nearly half of all interactions have nothing to do with work. When the team applied Anthropic’s own filtering methodology to their dataset, 48% of conversations would have been discarded entirely.
That filtered-out content tells a story the companies don’t emphasize. Those excluded conversations were far more likely to involve health and relationships (44.2% versus 31.2% in Anthropic’s analysis), harassment and hate (27.5% versus 5.66%), and sexual content (16.7% versus 2.4%). Anthropic’s Economic Index, despite being one of the most widely cited sources of AI usage data, explicitly filters out non-work conversations. Reuel puts the problem bluntly: “There is no independent source to corroborate it.”
The Observatory also found that usage patterns shift dramatically depending on which model people are using. Grok and Gemini skew toward information retrieval, with Grok particularly popular for news and politics — and, notably, where misinformation concentrates. People turn to Anthropic for coding, Gemini for social and roleplay scenarios, and ChatGPT for homework. Even different versions of the same model behave differently: GPT-3.5 conversations were short and transactional, while GPT-4o chats became longer and more iterative — consistent with that model’s reputation for emotional attachment.
What’s changing over time is equally revealing. Conversations in the WildChat dataset got progressively longer and more elaborate between 2023 and 2025, with more small talk and less self-disclosure from the assistants about being chatbots. That’s a companionship signal, not a productivity one. Meanwhile, sensitive-use exchanges actually dropped, suggesting platform safeguards are improving. David Widder, who researches human-AI interaction at UT-Austin and wasn’t involved in the project, notes that having a “bird’s eye view analysis rather than sectioned off into a separate report” helps researchers understand usage consistently. The uncomfortable reality is that people are using AI for deeply personal reasons — emotional support, companionship, sometimes harmful content — and the companies building these systems have every incentive to downplay that.
