Facebook External Hit
VerifiableMeta's crawler that fetches shared links from Facebook, Instagram, and Messenger to generate rich link previews. Meta documents one exception to its robots.txt compliance: the crawler might bypass robots.txt when performing security or integrity checks, such as looking for malware or malicious content. Verified by ASN lookup (AS32934); Meta's own docs describe IP-based allowlisting but publish no IP feed.
Operated by Meta · Official documentation
User agents
Patterns this directory matches, with real observed strings.
- regexfacebookexternalhit/\d
- observedfacebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
- observedfacebookexternalhit/1.1
How to verify
Look up the announced ranges by ASN
whois -h whois.radb.net -- '-i origin AS32934' | grep routeGood Bot Practices scorecard
- Identifies honestly
Stable UA token documented (1 pattern)
- Verifiable
Verifiable
- Respects robots.txt
Honors robots.txt (token: facebookexternalhit)
- Behaves
Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset
- Reachable operator
Operator and documentation published
Measured against the Good Bot Practices.