Directory / AI crawlers

Cloudflare Browser Run Crawler

Fully verifiable

The crawler behind the /crawl endpoint of Cloudflare's Browser Run (Browser Rendering) developer product, which crawls third-party websites on behalf of Cloudflare customers building applications on the platform. Its user agent is not configurable, and every request is signed with Web Bot Auth HTTP message signatures that site owners can verify against Cloudflare's published key directory.

Operated by Cloudflare · Official documentation

User agents

Patterns this directory matches, with real observed strings.

  • regexCloudflareBrowserRenderingCrawler/[\d.]+
  • observedCloudflareBrowserRenderingCrawler/1.0

How to verify

Fetch the Web Bot Auth key directory

curl -s https://web-bot-auth.cloudflare-browser-rendering-085.workers.dev/.well-known/http-message-signatures-directory

Good Bot Practices scorecard

  • Identifies honestly

    Stable UA token documented (1 pattern)

  • Verifiable

    Fully verifiable

  • Respects robots.txt

    Honors robots.txt (token: CloudflareBrowserRenderingCrawler)

  • Behaves

    Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset

  • Reachable operator

    Operator and documentation published

Measured against the Good Bot Practices.