Skip to content
Index Console
Start free
Free tool

Robots Checker

I show which pages your robots.txt blocks from Google. Type your site, and optionally a page, to test it the way Googlebot would.

How Google reads robots.txt

Google uses the group written for "Googlebot" if there is one, otherwise the group for all bots ("*"). The longest matching rule wins, and if an Allow and a Disallow rule are the same length, Allow wins.

Blocking a page in robots.txt stops Google from reading it, but it can still appear in search without a description. To keep a page out of Google, use a noindex tag and let Google read the page.