Who should read it: Whoever is interested in getting traffic from Search Engine esp Google.
Google Bot is a welcome visitor to all the websites and in recent time I have covered few topics on Google Bot. Here is the list:
Thanks to Matt cutts for helping me continue with the series. As I have explained earlier that you can fake Google bot by changing the user agents. Firefox extensions are allowing you do this easily, try http://chrispederick.com/work/useragentswitcher/.
Matt and his team says,
Telling webmasters to use DNS to verify on a case-by-case basis seems like the best way to go. I think the recommended technique would be to do a reverse DNS lookup, verify that the name is in the googlebot.com domain, and then do a corresponding forward DNS->IP lookup using that googlebot.com name; eg:
> host 220.127.116.11
18.104.22.168.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com.
> host crawl-66-249-66-1.googlebot.com
crawl-66-249-66-1.googlebot.com has address 22.214.171.124
I don’t think just doing a reverse DNS lookup is sufficient, because a spoofer could set up reverse DNS to point to crawl-a-b-c-d.googlebot.com.