SiteGuru Crawler

Verifiable

The crawler behind SiteGuru, a hosted SEO auditing tool. It crawls a site either because someone added that site to SiteGuru to monitor its SEO health, or because a site already being audited links to it and SiteGuru is checking that link for breakage. The operator documents that it reads only publicly available content, uses the HTML source to derive meta descriptions, page titles and missing alt text, submits no forms, crawls only pages that are in the sitemap or linked from another page, and limits concurrent requests to bound the load it puts on an origin. For sites it is auditing it also calls the W3 Validator and Google's PageSpeed API.

Operated by SiteGuru · Official documentation

User agents

Patterns this directory matches, with real observed strings.

regexSiteGuruCrawler
observedSiteGuruCrawler

How to verify

Documented static ranges: 54.78.126.239/32, 52.17.9.255/32, 34.246.125.226/32, 3.248.73.25/32, 34.253.76.180/32

IP ranges (5)

Refreshed daily from the operator's feed. Also available as /data/ips/siteguru.ips and /data/bots/siteguru.json.

3.248.73.25/32
34.246.125.226/32
34.253.76.180/32
52.17.9.255/32
54.78.126.239/32

Good Bot Practices scorecard

Identifies honestly
Stable UA token documented (1 pattern)
Verifiable
Verifiable
Respects robots.txt
Honors robots.txt (token: SiteGuruCrawler)
Behaves
Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset
Reachable operator
Operator and documentation published

Measured against the Good Bot Practices.