Directory / SEO tools

OnCrawl

Listed only

OnCrawl's SEO crawler, used to analyze a customer's own site structure and content for technical SEO reporting. OnCrawl's help docs describe no fixed IP range; the bot's identity is user-configurable per crawl.

Operated by OnCrawl · Official documentation

User agents

Patterns this directory matches, with real observed strings.

  • regexOncrawl/\d
  • observedMozilla/5.0 (compatible; Oncrawl/1.0; +http://www.oncrawl.com/)

How to verify

No verification recipe published by the operator.

Good Bot Practices scorecard

  • Identifies honestly

    Stable UA token documented (1 pattern)

  • Verifiable

    Operator publishes no verification path

  • Respects robots.txt

    Honors robots.txt (token: Oncrawl)

  • Behaves

    Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset

  • Reachable operator

    Operator and documentation published

Measured against the Good Bot Practices.