Craigslist reportedly blocks search engines to kill competitors(arstechnica.com)
arstechnica.com
Craigslist reportedly blocks search engines to kill competitors
http://arstechnica.com/tech-policy/2012/08/craigslist-reportedly-blocks-search-engines-to-kill-competitors/
2 comments
As a PadMapper user this saddens me, but I can only wonder why they didn't do it sooner. Craigslist posts are so short-lived, I don't know why they would want them indexed.
Did these people do any fact checking? I just ran another test:
visit http://sfbay.craigslist.org/bia/
spot ad for a kids Giant MTX 250 bike http://sfbay.craigslist.org/nby/bik/3189980073.html
perform a google search for the same thing http://www.google.com/search?q=site%3Acraigslist.org+giant+m...
result number two in the set is that posting, with the following summary:
Giant MTX 250 Kid's Mountain Bike sfbay.craigslist.org/nby/bik/3189980073.html 4 minutes ago ... Giant kid sized hardtail mountain bike for sale. It is a 21 speed, and it has front suspension. The bike was always taken care of, and is in perfect ...
"4 minutes ago" tells me Google is having no problems hitting craigslist.
visit http://sfbay.craigslist.org/bia/
spot ad for a kids Giant MTX 250 bike http://sfbay.craigslist.org/nby/bik/3189980073.html
perform a google search for the same thing http://www.google.com/search?q=site%3Acraigslist.org+giant+m...
result number two in the set is that posting, with the following summary:
Giant MTX 250 Kid's Mountain Bike sfbay.craigslist.org/nby/bik/3189980073.html 4 minutes ago ... Giant kid sized hardtail mountain bike for sale. It is a 21 speed, and it has front suspension. The bike was always taken care of, and is in perfect ...
"4 minutes ago" tells me Google is having no problems hitting craigslist.
...Do you do any fact checking?
I just checked the source:
<meta name="robots" content="NOARCHIVE,NOFOLLOW">
I just checked the source:
<meta name="robots" content="NOARCHIVE,NOFOLLOW">
Nothing new. That tag has always been in place. It's to prevent using an ad for SEO purposes
Those tags aren't as obvious in interpretation as they first appear. A full write-up is a http://noarchive.net/meta/ for completeness, but in short:
NOARCHIVE: ok to index this page, but don't cache the result permanently
NOFOLLOW: don't follow any links on this page
Note that these two things don't prevent a search engine from visiting a page, indexing it, and providing search results for it.
NOARCHIVE: ok to index this page, but don't cache the result permanently
NOFOLLOW: don't follow any links on this page
Note that these two things don't prevent a search engine from visiting a page, indexing it, and providing search results for it.