Robots.txt checker
See which robots.txt rule applies to a page path for a selected crawler.
Use this information as a starting point, then verify the page-level evidence available to you.
See which robots.txt rule applies to a page path for a selected crawler.
Use this information as a starting point, then verify the page-level evidence available to you.