How do I block DotBot in robots.txt?

To control DotBot access, add the following to your robots.txt file: User-agent: DotBot Allow: / # To block: # User-agent: DotBot # Disallow: /

Does DotBot respect robots.txt?

Yes, DotBot respects robots.txt directives.

DOTBOT

Q: What is the DotBot User-Agent string?

The DotBot User-Agent string is: Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)

LOW RISK📊 SEO & DATA SCRAPER

Moz's SEO analysis crawler for domain authority and link analysis

ORGANIZATION

Moz

FIRST SEEN

2010-01

RESPECTS ROBOTS.TXT

✓ YES

DOCUMENTATION

moz.com

—

DAILY VISITS

—

COUNTRIES ACTIVE

TRACKING

STATUS

—

LAST SEEN

📡 DOTBOT USER-AGENT STRING

Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)

This is the User-Agent header sent by DotBot in HTTP requests. Use this to identify DotBot in your server access logs.

📋 ABOUT DOTBOT

DotBot is the web crawler operated by Moz, the company behind some of the most widely-used SEO metrics including Domain Authority (DA) and Page Authority (PA). DotBot crawls the web to build the link index that powers Moz's suite of SEO analysis tools, including Link Explorer and the Moz Pro platform.

DotBot is a relatively polite crawler compared to other SEO bots, generating moderate traffic volumes and respecting robots.txt directives including Crawl-delay. Its primary focus is discovering and analyzing link structures to calculate Moz's proprietary authority metrics, which are used by hundreds of thousands of SEO professionals worldwide.

NORAD.io tracks DotBot alongside other SEO crawlers to give site operators a complete picture of SEO tool access to their content. Understanding the collective impact of SEO crawlers (AhrefsBot, SemrushBot, MJ12bot, DotBot) helps operators balance visibility in SEO tools against server resource consumption.

🎯 HOW TO DETECT DOTBOT

▸User-Agent contains 'DotBot/1.2'
▸References opensiteexplorer.org in the User-Agent string
▸Moderate crawl volumes — less traffic than AhrefsBot
▸Focus on link-related page analysis
▸Respects robots.txt and Crawl-delay directives

🔄 CRAWL BEHAVIOR

Moderate crawl rates with robots.txt compliance. Focuses on link discovery and page analysis for SEO metrics. Supports Crawl-delay. Less aggressive than AhrefsBot or SemrushBot.

PURPOSE

Powers Moz's SEO tools including Domain Authority (DA), Page Authority (PA), and Link Explorer. Used by SEO professionals for competitive analysis and link building research.

🤖 ROBOTS.TXT CONFIGURATION

User-agent: DotBot
Allow: /

# To block:
# User-agent: DotBot
# Disallow: /

DotBot respects robots.txt directives. Add this to your robots.txt file at the root of your domain.

→ Complete Guide: robots.txt for AI Bots

🗺️ WHERE IS DOTBOT ACTIVE?

⚠️ RELATED THREATS

Prompt Injection

Attempts to override bot instructions via malicious content embedded in web pages

Data Exfiltration

Bots attempting to extract sensitive data from websites including PII and credentials

Credential Stuffing

Automated login attempts using leaked credentials from data breaches

Aggressive Content Scraping

Bots aggressively scraping content beyond robots.txt limits and ToS

📂 MORE 📊 SEO & DATA SCRAPERS

AhrefsBotAhrefs SemrushBotSemrush ScrapyOpen Source Python RequestsUnknown cURLOpen Source MJ12botMajestic

📚 RELATED GUIDES

How to Detect AI Bots →robots.txt for AI Bots →NORAD API Docs →

PROTECT YOUR WEBSITE

Deploy SiteTrust to monitor and control AI bot access to your site with the Agent Passport Standard.

INSTALL SITETRUST →