.avif)
This article published by Agence Norry analyzes the Cloudflare settings change that took effect on September 15, 2026. It explains why sites that enabled AI bot blocking also block Googlebot, Bingbot and Applebot as a side effect, which sites are affected, and how to check and fix your settings in under five minutes.
The numbers to remember
IndicatorFigureSourceDate of the changeSeptember 15, 2026CloudflareOfficial announcementJuly 1, 2026Cloudflare blogAffected botsGooglebot, Bingbot, ApplebotCloudflareAffected sites (new)All new Cloudflare domainsCloudflareAffected sites (existing)Those that enabled Block Training or Block AI BotsCloudflareVisibility in logsNone (network-level block)Crawl LabError returnedHTTP 403 on the sitemap and pagesSearch Engine JournalDelay before visible SEO impactSeveral weeksField observations
What Cloudflare changed and why
On July 1, 2026, Cloudflare announced what they called their “Content Independence Day”: a complete overhaul of their AI bot management system. Instead of a single “Block AI bots” toggle, Cloudflare now sorts crawlers into three categories.
CategoryWhat the bot doesExamplesDefault since Sept. 15SearchIndexes for search enginesGooglebot (Search), Bingbot (Search)AllowedAgentActs in real time on behalf of a userAI agents, conversational crawlersBlocked on pages with adsTrainingCollects data to train modelsGPTBot, CCBot, Google ExtendedBlocked on pages with adsMixed (Search + Training)Indexes AND collects for AI trainingGooglebot, Bingbot, ApplebotBlocked if Training = Block
The problem lies in the last row. Googlebot, Bingbot and Applebot are mixed-purpose crawlers: they index for classic search, but they also collect data used for the engines' AI features (AI Overviews, AI Mode). Cloudflare can't separate the two uses at the request level. Rule applied by Cloudflare since September 15: a multi-purpose bot is treated according to the most restrictive rule. If Training = Block, Googlebot is blocked. Even if Search = Allow.
Why it's silent and dangerous
The block doesn't happen on your server. It happens at the Cloudflare network level, upstream of your infrastructure.
- Your server logs show nothing. Googlebot's request never reaches your server. No error in your Apache or Nginx logs. Just an absence.
- Your robots.txt is useless. The block is applied before Googlebot can read your robots.txt.
- Google Search Console doesn't trigger an immediate alert. The URL Inspection tool may show 403 errors if you test manually, but there's no automatic notification for a gradual block.
- The SEO impact arrives with a delay. Pages that are already indexed stay in Google for weeks while the cache expires. You only see the traffic drop after the problem has existed for a long time.
One site owner reported the problem as early as August 2026, before the official effective date: after setting AI Training = Block, Googlebot and Bingbot were receiving HTTP 403 errors on his sitemap.
Which sites are affected
Site profileSituationRiskNew domain added since Sept. 15Training and Agent blocked by default on pages with adsMedium: Googlebot blocked if Training is enabledExisting Free tier account with no changesNew defaults applied automaticallyHigh if monetized pagesExisting site with Block AI Bots enabledGooglebot blocked on pages where the rule appliesVery high: probably already activeExisting site with Block Training enabledSame effectVery highPaying customer with no settings changesExisting configuration keptLow if no Training block is active
What this changes for SEO and GEO
The impact goes beyond classic search ranking. Googlebot now crawls for two uses simultaneously: the classic search index and the AI features (AI Overviews, AI Mode). Blocking Googlebot therefore means blocking both.
For sites working on their visibility in AI answers (GEO strategy), this is doubly problematic. Not only are your pages no longer indexed for Google Search, but they can no longer be read to feed the AI Overviews live in France since July 22, 2026. At a time when being cited by AI is becoming a major strategic issue, a box ticked as a precaution can shut you out completely.
How to check your settings in under 5 minutes
- Step 1: access your Cloudflare dashboard. Log in and select the relevant site.
- Step 2: go to Security. Click Security, then Bot Management or AI Crawlers depending on your interface version.
- Step 3: check the Training setting. If Training is set to Block, you are potentially blocking Googlebot. Switch it to Disallow, not Block.
- Step 4: check the Block AI Bots (legacy) setting. If the old Block AI Bots toggle is enabled, it also triggers the Training block. Disable it or migrate to the new granular controls.
- Step 5: enable Bot Preference Sync. This setting ensures consistency between your rules and the new categories. New domains have it enabled by default, older ones may need to enable it manually.
- Step 6: check in Google Search Console. Use the URL Inspection tool on several key pages. If Googlebot returns 403 errors, the block is active.
Watch out for the distinction between Disallow and Block. Disallow sends a signal that well-behaved bots respect. Block is a hard network block: Googlebot can't get through. Confusion between the two is behind most of the reported incidents.
Want to block AI AND stay visible on Google: it's possible
It is entirely possible to protect your content from AI model training without blocking Google. You just need to use the right tools.
- Google Extended (recommended). This is Google's specific crawler for AI training and products like Gemini. Blocking it in your robots.txt prevents your content from being used for training, without touching Googlebot and therefore without any impact on your Search indexing.
- OpenAI's GPTBot. Dedicated to ChatGPT training. Distinct from the bot that powers ChatGPT Search. Blocking it in robots.txt protects your content with no consequence for your Google visibility.
- CCBot (Common Crawl). Used by many open-source models. Easily blocked in robots.txt.
What this incident reveals
This incident illustrates a real tension setting in during 2026: site owners must now manage simultaneously their visibility on classic search engines, their visibility in AI answers, and the protection of their content against model training. These three goals can conflict. The right approach isn't to block everything as a precaution, nor to let everything through by default. It's to understand which bots do what, and to configure your tools accordingly.
