Amazon Quick Web Crawler

Listed only

The web crawler AWS runs for Amazon Quick, which fetches the website content an AWS customer has configured as a data source and indexes it into that customer's knowledge base. Like the Bedrock and Q Business crawlers it is customer-directed but operator-run. AWS documents that it respects the robots.txt protocol, honours user-agent, allow, disallow and crawl-delay fields, reads robots.txt per host, and that it crawls anyway when robots.txt cannot be fetched because of blocking, parsing errors or timeouts.

Operated by Amazon · Official documentation

User agents

Patterns this directory matches, with real observed strings.

regexamazon-Quick-on-behalf-of-
observedamazon-Quick-on-behalf-of-<UUID>

How to verify

No verification recipe published by the operator.

Good Bot Practices scorecard

Identifies honestly
Stable UA token documented (1 pattern)
Verifiable
Operator publishes no verification path
Respects robots.txt
Honors robots.txt
Behaves
Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset
Reachable operator
Operator and documentation published

Measured against the Good Bot Practices.