The era of politely asking AI crawlers to back off is over for Patreon. The membership platform has stopped relying solely on robots.txt files—the digital equivalent of a “no trespassing” sign that bots can easily ignore—and started actively blocking AI training bots through Cloudflare’s AI Crawl Control technology.
Patreon’s product chief Drew Rowny didn’t mince words about the shift. “Consent shouldn’t depend on whether a scraper chooses to behave,” the company wrote in a blog post announcing the crackdown. That blunt assessment came after testing revealed a stark pattern: individual AI training crawlers were making thousands of weekly attempts to scrape Patreon content, despite the company’s existing requests to stop. When the new blocking measures went live, those attempts dropped to zero.
The timing makes sense. Patreon recently rolled out more public-facing discovery features—a redesigned Home Feed and tweet-like Quips—which threaten to expose creator content that was previously locked behind its paywall. More surface area means more scrapers sniffing around. The company says AI scraping has grown considerably more sophisticated since it first tried to deter crawlers back in 2023.
Patreon isn’t throwing out all bots, though. Indexing crawlers that help organize information and send users back to the platform still get a pass. But the distinction is clear: bots that train AI models on creator work without permission are now being met with a locked door rather than a polite request. Cloudflare’s broader push into this space—including a marketplace that lets websites charge bots for scraping, called Pay Per Crawl—suggests the infrastructure layer is finally catching up to the consent problem that has infuriated publishers and creators for years. Whether other platforms follow Patreon’s lead or continue treating robots.txt as sufficient is an open question, but the “thousands to zero” stat makes the old approach look increasingly naive.
