What is the Robots.txt Downloader & Checker?
Every website can keep a robots.txt file at example.com/robots.txt. It tells search engines and other bots which parts of the site they may visit. A small mistake in this file can hide a whole website from Google.
Our tool downloads the file and explains it in simple words. You get a quick table that shows if Googlebot, Bingbot and AI bots like GPTBot, ClaudeBot and PerplexityBot are allowed or blocked. You also see every rule and every sitemap listed in the file.
How to use it
- Type or paste the website address, like
example.com. You can also paste the full link to the file. - Press the button. We look for
/robots.txtin the right place on that website. - Read the simple summary to see what the file says.
- Press Download to save the file, or Copy text to paste it anywhere.
Why people like this tool
AI bot check
See at a glance if the site allows GPTBot, ClaudeBot, Google-Extended, CCBot and others.
Rule table
Every Allow and Disallow line in a clean table, grouped by user-agent.
Sitemap finder
All sitemap links from the file, ready to open or export.
Mistake warnings
We warn you if the whole site is blocked or no sitemap is listed.
