magpie-crawler
Listed onlymagpie-crawler is Brandwatch's web crawler. It downloads publicly available blog, forum, news, and social-media pages to be indexed and analysed by Brandwatch's social-listening platform. It respects robots.txt.
Operated by Brandwatch · Official documentation
User agents
Patterns this directory matches, with real observed strings.
- regexmagpie-crawler
- observedmagpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net)
How to verify
No verification recipe published by the operator.
Good Bot Practices scorecard
- Identifies honestly
Stable UA token documented (1 pattern)
- Verifiable
Operator publishes no verification path
- Respects robots.txt
Service traffic you configured — robots.txt does not apply
- Behaves
Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset
- Reachable operator
Operator and documentation published
Measured against the Good Bot Practices.