Run, don't crawl, to the new Site Diagnostics page

If you've ever wondered whether our crawler is able to reach all of your pages, you should try out the new Site Diagnostics tool. To determine the meaning of your site and show relevant ads, our crawlers need to be able to access your pages. Site Diagnostics enables you to see some of the errors our crawler finds while it's crawling. You can find the Site Diagnostics page listed under the Reports tab of your account.



If our crawler is having trouble reaching any of your pages, you'll see those URLs listed in the Blocked URLs section of the Site Diagnostics table. At the moment, we're displaying two types of errors:

URL not found – this means our crawler wasn't able to reach your pages, most likely because your server was temporarily down

Robots.txt file – this indicates that your robots.txt file was blocking our crawler from visiting

For help resolving either of these issues, just head over to the the AdSense Help Center. You can also access more general information about our crawler.

If you don't see any errors listed on your Site Diagnostics page but still want to improve the relevance of your Google ads, visit the following entries:

Why are my ads not relevant?

Why am I getting PSAs?

Posted by Nikhil Achwal - AdSense Publisher Support