DUCKDUCKBOT
LOW RISK🔍 SEARCH & AI CRAWLERDuckDuckGo's web crawler for its privacy-focused search engine
📡 DUCKDUCKBOT USER-AGENT STRING
DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)
This is the User-Agent header sent by DuckDuckBot in HTTP requests. Use this to identify DuckDuckBot in your server access logs.
📋 ABOUT DUCKDUCKBOT
DuckDuckBot is the web crawler for DuckDuckGo, the privacy-focused search engine. Unlike Google and Bing, DuckDuckGo does not build a comprehensive web index of its own — it primarily relies on Bing's search index supplemented by DuckDuckBot's targeted crawling for instant answers, direct content, and specific features.
DuckDuckBot is a relatively lightweight crawler compared to Googlebot or Bingbot. It focuses on discovering and indexing content for DuckDuckGo's unique features like instant answers, bang redirects, and direct content displays. DuckDuckGo publishes the specific IP addresses used by DuckDuckBot for verification purposes.
NORAD.io tracks DuckDuckBot as a low-risk crawler in its bot monitoring network. While DuckDuckBot generates less traffic than major search engine crawlers, its activity is an important signal for sites targeting privacy-conscious users who prefer DuckDuckGo over Google or Bing.
🎯 HOW TO DETECT DUCKDUCKBOT
- ▸User-Agent starts with 'DuckDuckBot/1.1'
- ▸Crawls from specific Azure IP addresses published by DuckDuckGo
- ▸Lower volume than major search engine crawlers
- ▸Focus on specific content types for instant answer features
- ▸Verify IPs against DuckDuckGo's published list
🌐 DUCKDUCKBOT KNOWN IP RANGES
20.191.45.21240.88.21.23540.76.173.15140.76.163.720.185.79.4752.142.26.17520.185.79.15Use these CIDR ranges to verify DuckDuckBot identity at the network level. Always combine with User-Agent verification for accurate detection.
🔄 CRAWL BEHAVIOR
Lightweight crawling focused on content discovery. DuckDuckGo primarily uses Bing's index, so DuckDuckBot focuses on supplementary crawling. Respects robots.txt. Low to moderate request rates.
Supplements DuckDuckGo's search index (which is primarily sourced from Bing). Used for direct answer features, instant answers, and supplementary content not available through Bing's index.
🤖 ROBOTS.TXT CONFIGURATION
User-agent: DuckDuckBot Allow: / # DuckDuckGo primarily uses Bing's index, # so blocking DuckDuckBot has limited impact on DuckDuckGo rankings
DuckDuckBot respects robots.txt directives. Add this to your robots.txt file at the root of your domain.
🗺️ WHERE IS DUCKDUCKBOT ACTIVE?
⚠️ RELATED THREATS
Attempts to override bot instructions via malicious content embedded in web pages
Data ExfiltrationBots attempting to extract sensitive data from websites including PII and credentials
Credential StuffingAutomated login attempts using leaked credentials from data breaches
Aggressive Content ScrapingBots aggressively scraping content beyond robots.txt limits and ToS
📂 MORE 🔍 SEARCH & AI CRAWLERS
📚 RELATED GUIDES
PROTECT YOUR WEBSITE
Deploy SiteTrust to monitor and control AI bot access to your site with the Agent Passport Standard.
INSTALL SITETRUST →