Amazon Kendra Web Crawler
Listed onlyThe web crawler AWS runs for Amazon Kendra, its enterprise search service, which fetches the site URLs an AWS customer has configured as a data source and indexes them for that customer. Like the Bedrock knowledge-base crawler it is customer-directed but operator-run, and AWS states that customers must only point it at their own pages or pages they are authorised to index, and that using it to aggressively crawl sites the customer does not own is not acceptable use.
Operated by Amazon · Official documentation
User agents
Patterns this directory matches, with real observed strings.
- regexamazon-kendra
- observedamazon-kendra
How to verify
No verification recipe published by the operator.
Good Bot Practices scorecard
- Identifies honestly
Stable UA token documented (1 pattern)
- Verifiable
Operator publishes no verification path
- Respects robots.txt
Honors robots.txt (token: amazon-kendra)
- Behaves
Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset
- Reachable operator
Operator and documentation published
Measured against the Good Bot Practices.