GPTBot
GPTBot is OpenAI’s web crawler for gathering publicly available content that may be used to train its generative AI foundation models. It is controlled through its own robots.txt token and is separate from OpenAI’s search and user-triggered crawlers.
GPTBot in Practice
OpenAI documents three distinct user agents: GPTBot for training, OAI-SearchBot for surfacing sites in ChatGPT search, and ChatGPT-User for actions a user initiates. The settings are independent, so a site can allow search while disallowing training ([OpenAI](https://developers.openai.com/api/docs/bots)). A full opt-out from training looks like `User-agent: GPTBot` followed by `Disallow: /`. The common mistake is treating OpenAI as one bot and blocking all of it, which also removes the site from ChatGPT search eligibility ([iCoda](https://icoda.io/blog/gptbot-guide/)). A publisher like Ledgerly that wants ChatGPT citations but not training use would disallow GPTBot and leave OAI-SearchBot allowed.
How GPTBot Activity Is Tracked
Filter server logs for the GPTBot user-agent string. Because user-agent strings can be spoofed, verify requests against the IP ranges OpenAI publishes — it maintains separate JSON files for GPTBot, OAI-SearchBot, and ChatGPT-User ([xSeek](https://www.xseek.io/docs/openai-crawlers-and-user-agents)).
Why GPTBot Matters
GPTBot hits are exposure to training, not citations — a visit from GPTBot doesn’t mean ChatGPT will cite or recommend you. Blocking it limits training use of your content but, per OpenAI’s own separation of controls, doesn’t by itself remove you from ChatGPT’s search results.
Terms Related to GPTBot
OAI-SearchBot ·
robots.txt for AI Crawlers ·
Parametric Memory
ClaudeBot
FAQ
Does blocking GPTBot remove my site from ChatGPT answers?
Not by itself. Search features run on OAI-SearchBot, a separate crawler with its own robots.txt token, so training and search eligibility are independent decisions.
