Verified Bots Directory

Directory / Search engines

Baiduspider

Verifiable

Baidu's primary web crawler, fetching pages for Baidu Search indexing.

Operated by Baidu · Official documentation

User agents

Patterns this directory matches, with real observed strings.

  • regexBaiduspider/\d
  • observedMozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
  • observedMozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html)

How to verify

Reverse-DNS check a client IP

dig -x <client-ip> +short   # expect a hostname matching *.baidu.com

Reverse-DNS suffixes: *.baidu.com, *.baidu.jp

Good Bot Practices scorecard

  • Identifies honestly

    Stable UA token documented (1 pattern)

  • Verifiable

    Verifiable

  • Respects robots.txt

    Honors robots.txt (token: Baiduspider)

  • Behaves

    Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset

  • Reachable operator

    Operator and documentation published

Measured against the Good Bot Practices.