Verified Bots Directory

Directory / SEO tools

Siteimprove Crawler

Verifiable

Siteimprove's content-suite crawler, which fetches pages of sites its customers have configured in their account to run quality-assurance, accessibility, policy and SEO checks. Companion agents (LinkCheck, Image size, Probe) fetch links and resources for the same checks and crawl from the same published address list.

Operated by Siteimprove · Official documentation

User agents

Patterns this directory matches, with real observed strings.

  • regexSiteCheck-sitecrawl by Siteimprove\.com
  • regexLinkCheck by Siteimprove\.com
  • regexImage size by Siteimprove\.com
  • regexProbe by Siteimprove\.com
  • observedMozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0) SiteCheck-sitecrawl by Siteimprove.com
  • observedMozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0) LinkCheck by Siteimprove.com
  • observedMozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.0) LinkCheck by Siteimprove.com
  • observedMozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0) Image size by Siteimprove.com
  • observedMozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0) Probe by Siteimprove.com

How to verify

Documented static ranges: 52.55.30.145/32, 52.20.183.91/32, 13.58.165.213/32, 18.116.191.222/32, 18.116.197.208/32, 18.189.206.159/32, 18.190.68.80/32, 18.216.137.252/32, 18.223.191.8/32, 3.13.121.241/32, 3.133.38.181/32, 3.135.49.180/32, 3.136.111.218/32, 3.138.54.100/32, 18.219.35.44/32, 185.229.145.22/32, 3.129.126.175/32, 52.7.141.1/32, 52.4.143.42/32, 52.60.34.56/32, 52.57.25.76/32, 35.157.236.87/32, 35.157.42.7/32, 18.157.140.51/32, 18.159.218.224/32, 18.196.205.2/32, 18.198.120.55/32, 3.124.26.114/32, 3.125.99.135/32, 3.64.159.177/32, 3.66.247.32/32, 3.68.122.244/32, 52.57.167.198/32, 35.158.180.204/32, 18.192.147.131/32, 52.58.146.230/32, 35.156.240.123/32, 185.229.144.10/32, 52.47.166.84/32, 3.10.81.130/32, 52.48.230.149/32, 35.204.24.17/32, 52.64.209.168/32, 52.199.41.160/32

IP ranges (44)

Refreshed daily from the operator's feed. Also available as /data/ips/siteimprove.ips and /data/bots/siteimprove.json.

13.58.165.213/32
18.116.191.222/32
18.116.197.208/32
18.157.140.51/32
18.159.218.224/32
18.189.206.159/32
18.190.68.80/32
18.192.147.131/32
18.196.205.2/32
18.198.120.55/32
18.216.137.252/32
18.219.35.44/32
18.223.191.8/32
185.229.144.10/32
185.229.145.22/32
3.10.81.130/32
3.124.26.114/32
3.125.99.135/32
3.129.126.175/32
3.13.121.241/32
3.133.38.181/32
3.135.49.180/32
3.136.111.218/32
3.138.54.100/32
3.64.159.177/32
3.66.247.32/32
3.68.122.244/32
35.156.240.123/32
35.157.236.87/32
35.157.42.7/32
35.158.180.204/32
35.204.24.17/32
52.199.41.160/32
52.20.183.91/32
52.4.143.42/32
52.47.166.84/32
52.48.230.149/32
52.55.30.145/32
52.57.167.198/32
52.57.25.76/32
52.58.146.230/32
52.60.34.56/32
52.64.209.168/32
52.7.141.1/32

Good Bot Practices scorecard

  • Identifies honestly

    Stable UA token documented (4 patterns)

  • Verifiable

    Verifiable

  • Respects robots.txt

    Honors robots.txt (token: SiteimproveBot-Crawler)

  • Behaves

    Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset

  • Reachable operator

    Operator and documentation published

Measured against the Good Bot Practices.