I think lying is a bit strong, I think they're potentially incorrect at worst.
The Cloudflare blog post where they announced this a few weeks ago stated "Cloudflare, Inc. (NYSE: NET), the leading connectivity cloud company, today announced it is now the first Internet infrastructure provider to block AI crawlers accessing content without permission or compensation, by default." [1]
I was also a bit confused by this wording and took it to mean Cloudflare was blocking AI traffic by default. What does it mean exactly?
Third party folks seemingly also interpreted it in the same way, eg The Verge reporting it with the title "Cloudflare will now block AI crawlers by default" [2]
I think what it actually means is that they'll offer new folks a default-enabled option to block ai traffic, so existing folks won't see any change. That aligns with text deeper in their blog post:
> Upon sign-up with Cloudflare, every new domain will now be asked if they want to allow AI crawlers, giving customers the choice upfront to explicitly allow or deny AI crawlers access. This significant shift means that every new domain starts with the default of control, and eliminates the need for webpage owners to manually configure their settings to opt out. Customers can easily check their settings and enable crawling at any time if they want their content to be freely accessed.
Not sure what this looks like in practice, or whether existing customers will be notified of the new option or something. But I also wouldn't fault someone for misinterpreting the headlines; they were a bit misleading.
I recently created a new Cloudflare account for a project I’m working on and moved two domains into it, and the settings were both on by default without asking me about it at all. The original press release specifically mentioned enabling it by default.
> Cloudflare, Inc. (NYSE: NET), the leading connectivity cloud company, today announced it is now the first Internet infrastructure provider to block AI crawlers accessing content without permission or compensation, *by default*.
> companies who want AI to recommend their products need to turn this off before it starts hurting them financially
Content marketing, gamified SEO, and obtrusive ads significantly hurt the quality of Google search. For all its flaws, LLMs don’t feel this gamified yet. It’s disappointing that this is probably where we’re headed. But I hope OpenAI and Anthropic realize that this drop in search result quality might be partly why Google’s losing traffic.
KomoD|6 months ago
Now you're just lying.
I checked several of my Cloudflare sites and none have it enabled by default:
"No robots.txt file found. Consider enabling Cloudflare managed robots.txt or generate one for your website"
"A robots.txt was found and is not managed by Cloudflare"
"Instruct AI bot traffic with robots.txt" disabled
cdrini|6 months ago
The Cloudflare blog post where they announced this a few weeks ago stated "Cloudflare, Inc. (NYSE: NET), the leading connectivity cloud company, today announced it is now the first Internet infrastructure provider to block AI crawlers accessing content without permission or compensation, by default." [1]
I was also a bit confused by this wording and took it to mean Cloudflare was blocking AI traffic by default. What does it mean exactly?
Third party folks seemingly also interpreted it in the same way, eg The Verge reporting it with the title "Cloudflare will now block AI crawlers by default" [2]
I think what it actually means is that they'll offer new folks a default-enabled option to block ai traffic, so existing folks won't see any change. That aligns with text deeper in their blog post:
> Upon sign-up with Cloudflare, every new domain will now be asked if they want to allow AI crawlers, giving customers the choice upfront to explicitly allow or deny AI crawlers access. This significant shift means that every new domain starts with the default of control, and eliminates the need for webpage owners to manually configure their settings to opt out. Customers can easily check their settings and enable crawling at any time if they want their content to be freely accessed.
Not sure what this looks like in practice, or whether existing customers will be notified of the new option or something. But I also wouldn't fault someone for misinterpreting the headlines; they were a bit misleading.
[1]: https://www.cloudflare.com/en-ca/press-releases/2025/cloudfl...
[2]: https://www.theverge.com/news/695501/cloudflare-block-ai-cra...
rzz3|6 months ago
> Cloudflare, Inc. (NYSE: NET), the leading connectivity cloud company, today announced it is now the first Internet infrastructure provider to block AI crawlers accessing content without permission or compensation, *by default*.
fourside|6 months ago
Content marketing, gamified SEO, and obtrusive ads significantly hurt the quality of Google search. For all its flaws, LLMs don’t feel this gamified yet. It’s disappointing that this is probably where we’re headed. But I hope OpenAI and Anthropic realize that this drop in search result quality might be partly why Google’s losing traffic.
ipaddr|6 months ago
gcbirzan|6 months ago
Edit: And, btw, that statement was true before the default was changed. So, your comment is doubly false.