You must Sign In to post a response.
  • Category: Blog and websites

    How to fix crawler errors in google adsense?

    Often we find crawler errors and blocked URLs in Google AdSense. Find out how to fix crawler errors in Google AdSense.

    How to fix crawlers errors in Google adsense? I have a website in blogspot and i have faced crawling errors problem. the blocked URL is like and the reason is "Page not found". Can anyone tell me how to resolve this problem?

    Waiting for your valuable suggestions.

  • #98189
    A crawler error means that Google AdSense crawler couuld not view your site's pages and index your site. By fixing these crawler errors, it will ensure that Google crawler can access your site and it will help to display more relevant ads.

    1. First solution to fix this error is to grant the crawler access in your robots.txt file which is located at the domain level of your website. You can do this by adding two lines of text to the top of your robots.txt file:

    User-agent: Mediapartners-Google

    This change will allow Google crawler to index the content of your site and provide you with the most relevant Google ads for that content. This change will not impact your Google search rankings. Google AdSense crawler will only attempt to crawl pages that have adsense ad code and are requesting ads.

    2. In case of Page not found error Google Adsense Crawler is attempting to crawl a page that no longer exists. Ther are receiving a 404 error specifically for this page. So you have to make sure this URL is serving correctly.

    3. Third issue is about the Content behind the login. Google deliver ads located behind a login. Sometimes account do not have crawler logins enabled and it prevents Google AdSense crawler from targeting the right content. So if your content and ads are login protected then you have to setup a crawler login. For login setup please visit

    With Regards

    Ajay Kumar

  • #98849
    Yes,you must modify the robot.txt for your blog that is by default provided in blogger platform for crawling and also use google webmaster from google.

  • #99126
    As you have a blogspot blog, there is no way you can fix this error. Plus, this error does not have any effect on your website. So ignore it.

  • #99281
    Thanks to all for your valuable responses.


    Priyabrata Das
    "Be Honest to You,not to Others"

  • Sign In to post your comments