Seocheckout

Your site has been assigned special crawl rate settings. You will not be able to change the...



Write the reason you're deleting this FAQ

Your site has been assigned special crawl rate settings. You will not be able to change the...

I noticed that my crawl rate was down, I use to see new content appear in seconds or minutes but now it is a couple of days. I went to Google Webmaster Tools to change the crawl rate. Under settings in Google Webmaster Tools I see this:

Your site has been assigned special crawl rate settings. You will not be able to change the crawl rate


What does that mean? Is that a good thing or a bad thing?

EDIT: I found this from a JohnMu at Google:
The special crawl rate settings apply to sites where we are manually optimizing the crawl rate on our side. This is often due to feedback from the webmaster, the hoster or just generally because we've determined that the site can profit from a manual crawl rate setting by our engineers. In general, you do not have to worry about the crawl rate setting if this is the case with your site.


Also, when looking at the site health under Crawl Stats, I see pages has dropped from 90,000 per day to about 60,000 per day. Not idea why.

KB per day has increased from 1,200,000 to 1,600,000 though. No idea why.

EDIT #2: Apparently if you are on a CDN, google will manually set your crawl rate for some reason. Here is what Gary from Google says about it:
When you're using a CDN, we may assign a special crawl rate for your site so that we can crawl as much as possible without affecting negatively others who're using the same service. It's best to not tackle that setting, especially since we're already accessing your site more than 4000 times a day.
If you want to lower your crawling rate, let me know, but I definitely not recommend increasing it. I'd rather focus on posting unique and compelling content and on user experience.


Kinda odd though that my crawl rate is now lower when I've had a CDN for... always..

Comments

Please login or sign up to leave a comment

Join