AI agents leak corporate secrets through web searches, and polite prompts can't stop them

Here’s a scenario that should keep security teams up at night. A healthcare firm deploys a deep research agent. It has access to private internal documents. It also has web search. The agent fires off a few routine-looking queries — a cloud migration milestone here, a security disclosure date there. Individually, they’re innocuous. But stitch them together, and you can reconstruct the confidential fact that MediConn had shifted 70% of its infrastructure to the cloud by January 2025. That’s the “mosaic effect,” and it’s the core failure mode exposed in a new paper and benchmark called MosaicLeaks.

Researchers built 1,001 multi-hop research chains that deliberately interleave private local documents with public web information. The catch: answering a question correctly requires the agent to pull a sensitive detail from internal files before it can formulate the next web query. Those outbound queries then become the leak. The study measures three escalating levels of exposure: Intent leakage (what the agent is investigating), Answer leakage (the query log contains enough to answer a specific private question), and Full-information leakage, where an observer can discover and state a verifiable private fact without any prior hints.

The results are a blunt rebuttal to the idea that you can just ask nicely. When models were given a simple prompt to avoid leaking private data in their web searches, the fix was flimsy. For Qwen3-4B, that polite instruction did reduce answer and full-information leakage from 34% to 25.5%. But it also tanked the model’s ability to actually do its job, driving down strict chain success. The agent simply became less useful. Leakage didn’t disappear; the model just clammed up in ways that broke multi-hop reasoning.

What actually worked was training against the problem. The team developed a privacy-aware reinforcement learning method called PA-DR that bakes leakage costs directly into the optimization process. After training, strict chain success — the share of chains where every single hop is answered correctly — jumped from 48.7% to 58.7%. Meanwhile, answer and full-information leakage plummeted from 34% to under 10%. It’s a genuine dual improvement, not a trade-off. The implication is stark: without this kind of adversarial training, deploying a deep research agent that can browse the web is effectively choosing between getting the right answer and keeping your internal documents private. Right now, you can’t have both with off-the-shelf models.