Dataproviderbot

Verifiable

Dataprovider.com's in-house crawler. It indexes more than 400 million domains each month and structures what it finds into the company's web dataset (business information, technology detection, classifications and risk signals). The operator documents that it follows the robot exclusion protocol and that its crawlers can be identified by a reverse DNS lookup.

Operated by Dataprovider.com · Official documentation

User agents

Patterns this directory matches, with real observed strings.

regexDataprovider\.com
observedMozilla/5.0 (compatible; Dataprovider.com)

How to verify

Reverse-DNS check a client IP

dig -x <client-ip> +short   # expect a hostname matching *.dataproviderbot.com

Reverse-DNS suffixes: *.dataproviderbot.com

Good Bot Practices scorecard

Identifies honestly
Stable UA token documented (1 pattern)
Verifiable
Verifiable
Respects robots.txt
Honors robots.txt (token: dataprovider)
Behaves
Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset
Reachable operator
Operator and documentation published

Measured against the Good Bot Practices.