Free SEO Tool
Robots.txt Checker
Enter any domain to inspect its robots.txt file. See crawler rules at a glance and find issues that could be blocking search engines from indexing your content.
What is robots.txt?
robots.txt is a plain-text file at the root of your domain that tells web crawlers which pages or directories they can and cannot access. It is a critical file for SEO because a misconfigured rule can block Googlebot from crawling your entire site.
What this tool checks
- Whether robots.txt exists and is publicly accessible
- Allow and Disallow rules per user-agent
- Whether Googlebot or all crawlers are fully blocked
- Sitemap directives embedded in robots.txt