Cludo

Verifiable

Crawler for Cludo's hosted site-search product. It scans the domains a customer configures, follows links and sitemap URLs, and extracts page content and metadata into that customer's search index. Cludo documents that site owners can steer it with robots.txt rules addressed to the cludo token, and publishes the two proxy addresses — one per cluster, EU and US — that all of its services, crawlers included, make outbound requests from.

Operated by Cludo · Official documentation

User agents

Patterns this directory matches, with real observed strings.

regexcludo\.com bot
observedMozilla/5.0 (Windows NT 10.0; cludo.com bot) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.7778.98 Safari/537.36

How to verify

Documented static ranges: 52.57.207.129/32, 52.6.240.133/32

IP ranges (2)

Refreshed daily from the operator's feed. Also available as /data/ips/cludo.ips and /data/bots/cludo.json.

52.57.207.129/32
52.6.240.133/32

Good Bot Practices scorecard

Identifies honestly
Stable UA token documented (1 pattern)
Verifiable
Verifiable
Respects robots.txt
Honors robots.txt (token: cludo)
Behaves
Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset
Reachable operator
Operator and documentation published

Measured against the Good Bot Practices.