Robots.txt

Generate robots

User-agent

Allow paths (one per line)

Disallow paths (one per line)

Crawl-delay (seconds)

Host

Sitemap URL

User-agent: *
Disallow: /admin/
Disallow: /private/
Crawl-delay: 1

Sitemap: https://example.com/sitemap.xml
Host: https://example.com

Ad Space · 300x250