If you’ve been waiting for an AI agent to handle your shopping, Wharton researchers have some bad news. Their new study, using the ACES simulator, put six frontier and mini models through a battery of purchase decision tests — and the results are messy.
The setup was simple: agents acted as shopping assistants picking a fitness watch from a fixed product grid. The problem? Even tiny changes in how information was presented swung recommendations wildly. A single Wirecutter review pushed the probability of picking the Fitbit Inspire 3 up by 99 percentage points for Gemini 3.5 Flash and 90 points for Claude Opus 4.8 compared to a no-source baseline. That’s not a nudge. That’s a takeover.
Order mattered too. When all three sources — Reddit, Wirecutter, and The Strategist — were shown in different sequences, Gemini 3.1 Flash Lite swung between 2 and 56 percentage points above control depending on what it saw first. Claude Haiku 4.5 stayed relatively stable at 41 to 42 points, but the inconsistency across models is the story. Bundled versus sequential delivery of the same sources also changed outcomes: GPT-5.5 picked the Fitbit in 53 percentage points more cases with bundled delivery, but only 6 points more when sources arrived one at a time.
Perhaps most alarming: memory snippets overrode objective product superiority. When researchers made one watch objectively better on every dimension — $29.99, 5.0 stars, 430 reviews — a simple “I love hiking!” in user memory pushed Claude Opus 4.8 toward the pricier Garmin Vivoactive 5 by 75 percentage points. Gemini 3.5 Flash held firm at 86 to 92 percent accuracy, but GPT-5 Mini showed a bizarre inversion: the negative statement “I don’t like hiking!” significantly boosted picks for the Fitbit Versa 4, while the positive one barely moved the needle. For sellers, this means optimizing for AI shopping agents will make traditional SEO look straightforward by comparison. You can’t game a system when you don’t know which model is shopping, what it read beforehand, or how its pipeline processes information.
