What's the hidden secret of the review for ASIN B0069IY63Y?(amazon.com)
amazon.com
What's the hidden secret of the review for ASIN B0069IY63Y?
http://www.amazon.com/robots.txt
2 comments
I don't get it. The link points to www.amazon.com/robots.txt.
if you look in the file you'll see a Disallow entry for /product-reviews/B0069IY63Y.
honeypot entry to ensure that crawlers are complying with robots.txt?