Verified Bots Directory

Directory / AI assistants

Meta External Fetcher

Verifiable

Meta's on-demand fetcher that retrieves a single link at a user's request to support agentic AI features (e.g. an AI assistant navigating a page a user asked about), rather than broad indexing. Because its fetches are requested by a user, Meta documents that this crawler may bypass robots.txt rules. Verified by ASN lookup (AS32934).

Operated by Meta · Official documentation

User agents

Patterns this directory matches, with real observed strings.

  • regexmeta-externalfetcher/\d
  • observedmeta-externalfetcher/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)
  • observedmeta-externalfetcher/1.1

How to verify

Look up the announced ranges by ASN

whois -h whois.radb.net -- '-i origin AS32934' | grep route

Good Bot Practices scorecard

  • Identifies honestly

    Stable UA token documented (1 pattern)

  • Verifiable

    Verifiable

  • Respects robots.txt

    Does not honor robots.txt

  • Behaves

    Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset

  • Reachable operator

    Operator and documentation published

Measured against the Good Bot Practices.