Directory / AI crawlers

GPTBot

Fully verifiable

OpenAI's web crawler that gathers publicly available data used to train OpenAI's models.

Operated by OpenAI · Official documentation

User agents

Patterns this directory matches, with real observed strings.

  • GPTBot/\d
  • Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.2; +https://openai.com/gptbot

How to verify

Fetch the official IP range feed

curl -s https://openai.com/gptbot.json

IP ranges (21)

Refreshed daily from the operator's feed. Also available as /data/ips/gptbot.ips and /data/bots/gptbot.json.

132.196.86.0/24
172.182.202.0/25
172.182.204.0/24
172.182.207.0/25
172.182.214.0/24
172.182.215.0/24
20.125.66.80/28
20.171.206.0/24
20.171.207.0/24
4.227.36.0/25
52.230.152.0/24
74.7.175.128/25
74.7.227.0/25
74.7.227.128/25
74.7.228.0/25
74.7.230.0/25
74.7.241.0/25
74.7.241.128/25
74.7.242.0/25
74.7.243.128/25
74.7.244.0/25

Good Bot Practices scorecard

  • Identifies honestly

    Stable UA token documented (1 pattern)

  • Verifiable

    Fully verifiable

  • Respects robots.txt

    Honors robots.txt (token: GPTBot)

  • Behaves

    Crawl-rate behavior is operator-declared; not machine-verifiable from this dataset

  • Reachable operator

    Operator and documentation published

Measured against the Good Bot Practices.