Ask Experts » Computers » Blog and websites »

How to fix crawler errors in google adsense?

Date: 26 Dec 2012   Group: Computers    Category: Blog and websites   

How to fix crawlers errors in Google adsense? I have a website in blogspot and i have faced crawling errors problem. the blocked URL is like and the reason is "Page not found". Can anyone tell me how to resolve this problem?

Waiting for your valuable suggestions.


Find Answers
Often we find crawler errors and blocked URLs in Google AdSense. Find out how to fix crawler errors in Google AdSense.

Author: Ajay Kumar    27 Dec 2012      Member Level: Gold     Points : 5  (Rs 5)    Voting Score: 0

A crawler error means that Google AdSense crawler couuld not view your site's pages and index your site. By fixing these crawler errors, it will ensure that Google crawler can access your site and it will help to display more relevant ads.

1. First solution to fix this error is to grant the crawler access in your robots.txt file which is located at the domain level of your website. You can do this by adding two lines of text to the top of your robots.txt file:

User-agent: Mediapartners-Google

This change will allow Google crawler to index the content of your site and provide you with the most relevant Google ads for that content. This change will not impact your Google search rankings. Google AdSense crawler will only attempt to crawl pages that have adsense ad code and are requesting ads.

2. In case of Page not found error Google Adsense Crawler is attempting to crawl a page that no longer exists. Ther are receiving a 404 error specifically for this page. So you have to make sure this URL is serving correctly.

3. Third issue is about the Content behind the login. Google deliver ads located behind a login. Sometimes account do not have crawler logins enabled and it prevents Google AdSense crawler from targeting the right content. So if your content and ads are login protected then you have to setup a crawler login. For login setup please visit

Author: Keyur    19 Jan 2013      Member Level: Bronze     Points : 1    Voting Score: 0

Yes,you must modify the robot.txt for your blog that is by default provided in blogger platform for crawling and also use google webmaster from google.

Author: shivam chawla    25 Jan 2013      Member Level: Bronze     Points : 1    Voting Score: 0

As you have a blogspot blog, there is no way you can fix this error. Plus, this error does not have any effect on your website. So ignore it.
Author: Priyabrata Das    30 Jan 2013      Member Level: Bronze     Points : 0    Voting Score: 0

Thanks to all for your valuable responses.



Post answer or solution to "How to fix crawler errors in google adsense?"

You can earn revenue online from Google AdSense and our revenue sharing programs by post valid, self written answers for the problem How to fix crawler errors in google adsense?

You must Sign In to post a response.

Next Question: Facing error while Login to Facebook
Previous Question: Which social networking site allows to share our article link?
Return to Question Index
Ask Question to Expert
Category: Blog and websites

Related Questions
Popular Tags   What are tags ?   Search Tags  

Awards & Gifts
Active Members
TodayLast 7 Daysmore...

ISC Technologies, Kochi - India. Copyright © All Rights Reserved.