Directory / Security scanners

SurdotlyBot

Listed only

Sur.ly's crawler. The operator runs a spam-fighting link-safety service and uses this bot to fetch third-party sites and build a short security profile for each one, querying metadata and favicons and taking a screenshot of the homepage. The operator states it never harvests e-mail addresses or content unrelated to security.

Operated by Sur.ly · Official documentation

User agents

Patterns this directory matches, with real observed strings.

  • regexSurdotlyBot
  • observedMozilla/5.0 (compatible; SurdotlyBot/1.0; +http://sur.ly/bot.html; Linux; Android 4; iPhone; CPU iPhone OS 6_0_1 like Mac OS X)

How to verify

No verification recipe published by the operator.

Good Bot Practices scorecard

  • Identifies honestly

    Stable UA token documented (1 pattern)

  • Verifiable

    Operator publishes no verification path

  • Respects robots.txt

    Honors robots.txt (token: SurdotlyBot)

  • Behaves

    Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset

  • Reachable operator

    Operator and documentation published

Measured against the Good Bot Practices.