Ask Experts » Computers » Blog and websites »

How to fix crawler errors in google adsense?

Date: 26 Dec 2012   Posted By: Priyabrata Das     Group: Computers    Category: Blog and websites   

How to fix crawlers errors in Google adsense? I have a website in blogspot and i have faced crawling errors problem. the blocked URL is like and the reason is "Page not found". Can anyone tell me how to resolve this problem?

Waiting for your valuable suggestions.


Often we find crawler errors and blocked URLs in Google AdSense. Find out how to fix crawler errors in Google AdSense.

Author: Ajay Kumar    27 Dec 2012      Member Level: Gold     Points : 5  (Rs 5)    Voting Score: 0

A crawler error means that Google AdSense crawler couuld not view your site's pages and index your site. By fixing these crawler errors, it will ensure that Google crawler can access your site and it will help to display more relevant ads.

1. First solution to fix this error is to grant the crawler access in your robots.txt file which is located at the domain level of your website. You can do this by adding two lines of text to the top of your robots.txt file:

User-agent: Mediapartners-Google

This change will allow Google crawler to index the content of your site and provide you with the most relevant Google ads for that content. This change will not impact your Google search rankings. Google AdSense crawler will only attempt to crawl pages that have adsense ad code and are requesting ads.

2. In case of Page not found error Google Adsense Crawler is attempting to crawl a page that no longer exists. Ther are receiving a 404 error specifically for this page. So you have to make sure this URL is serving correctly.

3. Third issue is about the Content behind the login. Google deliver ads located behind a login. Sometimes account do not have crawler logins enabled and it prevents Google AdSense crawler from targeting the right content. So if your content and ads are login protected then you have to setup a crawler login. For login setup please visit


Author: Keyur    19 Jan 2013      Member Level: Bronze     Points : 1    Voting Score: 0

Yes,you must modify the robot.txt for your blog that is by default provided in blogger platform for crawling and also use google webmaster from google.

Author: shivam chawla    25 Jan 2013      Member Level: Bronze     Points : 1    Voting Score: 0

As you have a blogspot blog, there is no way you can fix this error. Plus, this error does not have any effect on your website. So ignore it.

Author: Priyabrata Das    30 Jan 2013      Member Level: Bronze     Points : 0    Voting Score: 0

Thanks to all for your valuable responses.


Post Answer

You must Sign In to post a response.

Post Answer        

Read related posts: Google AdSense    

Related Questions
  • Return to Question Index
  • Ask Question to Experts

  • Awards & Gifts
    Top Contributors
    TodayLast 7 Daysmore...

    ISC Technologies, Kochi - India. © All Rights Reserved.