GPTBot, OAI-SearchBot and ChatGPT-User: what each one does

By Michael Nefedov, noetio · Updated 21 October 2026

GPTBot is the OpenAI crawler for content that may be used in training. It is not the bot that decides if you show up in ChatGPT search. That job belongs to OAI-SearchBot.

GPTBot vs OAI-SearchBot vs ChatGPT-User

OpenAI lists four user agents, and each one has a different job. They are on one page: developers.openai.com/api/docs/bots (“Overview of OpenAI Crawlers”, no date on the page), read on 21 October 2026. The user-agent versions below are what the page showed that day.

Does GPTBot affect ChatGPT search visibility?

No: by OpenAI's own description, search appearance is tied to OAI-SearchBot, not GPTBot.ChatGPT-User is “not used to determine whether content may appear in Search”.

OpenAI does not say that allowing OAI-SearchBot will get you cited, or how it picks pages. For what may help, read how to get cited by ChatGPT and SEO vs GEO.

GPTBot robots.txt: copy and paste

Pick the snippet that matches your goal and paste it into robots.txt. The copy of the OpenAI page we read had no sample robots.txt. The snippets below use the user-agent names OpenAI lists. They follow the normal robots.txt format.

No training, visible in ChatGPT search

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

Allow all OpenAI bots

User-agent: GPTBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

Block all OpenAI bots

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

By OpenAI's description, this removes you from ChatGPT search answers, though your pages can still appear as navigational links. Because of the ChatGPT-User note above, it may also not stop user-started fetches.

Keep bots out of one folder only

User-agent: GPTBot
Disallow: /private/

User-agent: OAI-SearchBot
Disallow: /private/

Under RFC 9309, section 2.2.1, a crawler obeys the User-agent: * group only when no group matches its own token. If your * group hides folders, repeat those lines in each bot group.

What we ran

All three user-agent strings got HTTP 200 from our own site. On 21 October 2026 we requested the home page of getcompassseo.com with curland the user-agent strings from OpenAI's page for GPTBot, OAI-SearchBot and ChatGPT-User. OpenAI labels the GPTBot and OAI-SearchBot strings as examples and the ChatGPT-User string as the full string. Use this to test your own site (GPTBot example string copied from OpenAI's page):

curl -s -o /dev/null -w "%{http_code}\n" \
  -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \
  https://yoursite.com/

That only shows our server answers them. A CDN or firewall rule can return 403 to a bot that robots.txt allows, so run the test against your own site.

Check your logs

One awk command counts how often each OpenAI bot hits your site.

awk 'BEGIN{n=split("gptbot oai-searchbot chatgpt-user oai-adsbot",b," ")} {l=tolower($0); for(i=1;i<=n;i++) if(index(l,b[i])) c[b[i]]++} END{for(k in c) print c[k], k}' access.log | sort -rn

This counts each request once per bot, whatever the case. A plain grep -o would count GPTBot twice per request, because OpenAI's user-agent string contains the name twice (GPTBot/1.4; +https://openai.com/gptbot, copied from the OpenAI page).

Then compare IPs against OpenAI's lists: gptbot.json, searchbot.json, chatgpt-user.json. OpenAI also asks you to allow requests from its published IP ranges, so a firewall that blocks them can keep OAI-SearchBot out even when robots.txt allows it. More log tips are in the AI crawlers guide.

Caveats

Blocking is a request, not a lock.

• OpenAI says that when it fetches robots.txt files it may add a robots.txt marker to the user-agent string. The marker helps you tell robots.txt requests from other requests in your logs.
• If both GPTBot and OAI-SearchBot are allowed, OpenAI says it may use the results of one crawl for both uses.
• OpenAI documents blocking as an indication for training use. It is a request, not a technical lock.

Anthropic splits its bots in a similar way: see ClaudeBot. For all owners side by side, see the AI crawlers table.

Sources

All read on 21 October 2026.

1. OpenAI, Overview of OpenAI Crawlers, developers.openai.com/api/docs/bots (no date on the page). Used for every bot description, quote, user-agent version, the 24-hour note and the robots.txt marker.
2. OpenAI, IP lists: openai.com/gptbot.json, openai.com/searchbot.json, openai.com/chatgpt-user.json. All three returned HTTP 200 on 21 October 2026.
3. IETF, RFC 9309, Robots Exclusion Protocol (September 2022), section 2.2.1. Used for how a crawler picks its group.
4. Our own curl tests of getcompassseo.com, run 21 October 2026, described in “What we ran”.

Questions

More guides

• ClaudeBot: What It Is and How to Block or Allow It
• AI Crawlers: The Main Bots, robots.txt and Log Checks
• llms.txt: What It Is and Whether You Need One (2026)
• How to Rank in AI Overviews: What Google Says (2026)
• How to Get Cited by ChatGPT: What Is Documented (2026)

Turn your next topic into a useful page

Compass reads your website, helps you plan relevant topics and creates sourced article drafts. Review and approve the work, then send it to a connected CMS or export it.

Written by Michael Nefedov, founder of noetio. Updated 21 October 2026.