Meta-ExternalAgent — Meta's ai training crawler

Meta-ExternalAgent is a declared bulk training crawler, introduced after Meta consolidated its AI data collection away from the older FacebookBot token.

Its sibling Meta-ExternalFetcher performs user-initiated fetches and — like Perplexity-User — Meta indicates robots.txt may be bypassed for those, on the user-request argument. Control the two separately.

Meta-ExternalAgent identifies itself with the following user-agent string:

meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)

Never trust the user-agent string alone. Scrapers routinely impersonate well-known crawlers to inherit their access. Meta documents its crawlers and publishes IP ranges for verification on its developer site; check suspect traffic against those ranges.

Note that most AI crawlers and fetchers, Meta-ExternalAgent included, do not execute JavaScript — so this traffic is invisible to GA4 and every script-based analytics tool. Server logs, CDN analytics, or a dedicated bot-analytics layer are the only places you will see it. For the full picture of measuring AI-driven visits, see our guide on how to track AI traffic.

To refuse Meta-ExternalAgent access to your entire site, add this to your robots.txt:

User-agent: meta-externalagent
Disallow: /

To restrict it from specific sections only (for example, premium content) while leaving the rest open:

User-agent: meta-externalagent
Disallow: /premium/
Disallow: /members/

The case for blocking: Meta AI's answers return essentially no referral traffic to publishers, and Llama training access is otherwise free of compensation. The standard training-crawler logic applies with no offsetting search surface at all.

The case for allowing or monetizing: Meta licenses enormous amounts of data commercially in other contexts; withholding free access is the only leverage a publisher has toward any paid arrangement.