How do I block OAI-SearchBot in robots.txt?

To control OAI-SearchBot access, add the following to your robots.txt file: # Allow search indexing but block training: User-agent: OAI-SearchBot Allow: / User-agent: GPTBot Disallow: /

Does OAI-SearchBot respect robots.txt?

Yes, OAI-SearchBot respects robots.txt directives.

What are OAI-SearchBot's IP ranges?

Known OAI-SearchBot IP ranges include: 20.15.240.64/28, 20.15.240.80/28, 20.15.240.96/28

OAI-SEARCHBOT

Q: What is the OAI-SearchBot User-Agent string?

The OAI-SearchBot User-Agent string is: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot)

LOW RISK🔍 SEARCH & AI CRAWLER

OpenAI's search grounding crawler — fetches pages for ChatGPT Search and SearchGPT results

ORGANIZATION

OpenAI

FIRST SEEN

2024-07

RESPECTS ROBOTS.TXT

✓ YES

DOCUMENTATION

platform.openai.com

—

DAILY VISITS

—

COUNTRIES ACTIVE

TRACKING

STATUS

—

LAST SEEN

📡 OAI-SEARCHBOT USER-AGENT STRING

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot)

This is the User-Agent header sent by OAI-SearchBot in HTTP requests. Use this to identify OAI-SearchBot in your server access logs.

📋 ABOUT OAI-SEARCHBOT

OAI-SearchBot is OpenAI's dedicated search indexing crawler, introduced alongside ChatGPT's integrated search capabilities (initially launched as SearchGPT). Unlike GPTBot, which collects training data for model development, OAI-SearchBot indexes pages specifically to power real-time search results within ChatGPT's search feature.

This distinction is critical for website operators: blocking GPTBot prevents your content from being used in AI model training, while blocking OAI-SearchBot removes your site from ChatGPT's search results entirely. OpenAI designed these as separate User-Agent tokens specifically to give site operators granular control over different types of AI data access.

NORAD.io tracks OAI-SearchBot separately from GPTBot to help site operators make informed decisions about their OpenAI access policies. With ChatGPT Search growing as a traffic source, understanding OAI-SearchBot activity is increasingly important for content publishers who want search visibility without contributing to AI training.

🎯 HOW TO DETECT OAI-SEARCHBOT

▸User-Agent contains 'OAI-SearchBot'
▸Shares IP ranges with GPTBot but has distinct User-Agent
▸Blocking OAI-SearchBot removes your site from ChatGPT Search results
▸Separate robots.txt control from GPTBot training crawler
▸Newer bot — first seen mid-2024 with SearchGPT launch

🌐 OAI-SEARCHBOT KNOWN IP RANGES

20.15.240.64/2820.15.240.80/2820.15.240.96/28

Use these CIDR ranges to verify OAI-SearchBot identity at the network level. Always combine with User-Agent verification for accurate detection.

🔄 CRAWL BEHAVIOR

Indexes web pages for OpenAI's search product. Moderate crawl frequency. Respects robots.txt with its own User-Agent token. Does not execute JavaScript. Follows sitemaps.

PURPOSE

Builds the search index for ChatGPT's integrated search functionality (formerly SearchGPT). Allows OpenAI's products to provide web search results with citations. Separate from GPTBot's training data collection.

🤖 ROBOTS.TXT CONFIGURATION

# Allow search indexing but block training:
User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

OAI-SearchBot respects robots.txt directives. Add this to your robots.txt file at the root of your domain.

→ Complete Guide: robots.txt for AI Bots

🗺️ WHERE IS OAI-SEARCHBOT ACTIVE?

⚠️ RELATED THREATS

Prompt Injection

Attempts to override bot instructions via malicious content embedded in web pages

Data Exfiltration

Bots attempting to extract sensitive data from websites including PII and credentials

Credential Stuffing

Automated login attempts using leaked credentials from data breaches

Aggressive Content Scraping

Bots aggressively scraping content beyond robots.txt limits and ToS

🔗 RELATED BOTS

GPTBotLOW

OpenAI · OpenAI's web crawler used for training GPT models and improving AI capabilities

ChatGPT-UserLOW

OpenAI · ChatGPT's real-time web browsing mode that fetches pages during conversations

📂 MORE 🔍 SEARCH & AI CRAWLERS

GPTBotOpenAI ClaudeBotAnthropic GooglebotGoogle Google-ExtendedGoogle BingbotMicrosoft BytespiderByteDance

📚 RELATED GUIDES

How to Detect AI Bots →robots.txt for AI Bots →NORAD API Docs →

PROTECT YOUR WEBSITE

Deploy SiteTrust to monitor and control AI bot access to your site with the Agent Passport Standard.

INSTALL SITETRUST →