Google-Extended — Google's robots.txt control token

Because it is usage governance rather than a crawler, you will never see 'Google-Extended' in server logs, and there are no IPs to verify. It exists purely as a line you write in robots.txt.

Its scope is specific: Gemini training and Gemini grounding. It does not opt you out of AI Overviews or AI Mode, which are Search features governed by Googlebot and snippet controls.

Google-Extended never appears in your logs — it is a robots.txt token, not a crawler. Fetching is done by Google's existing crawl infrastructure; the token only governs how the crawled data may be used.

To refuse Google-Extended access to your entire site, add this to your robots.txt:

User-agent: Google-Extended
Disallow: /

To restrict it from specific sections only (for example, premium content) while leaving the rest open:

User-agent: Google-Extended
Disallow: /premium/
Disallow: /members/

The case for blocking: Since blocking costs nothing in search visibility, disallowing Google-Extended is the rational default for any publisher who doesn't want to donate training data to Gemini.

The case for allowing or monetizing: Withholding free training access is a precondition for licensing it. As with GPTBot and ClaudeBot, the token is your consent lever — keep it closed until access is worth something.