Web-scale crawling at cable modem speedsblog.metadatalabs.com8 points·by jmischel·vor 16 Jahren·0 comments
jmischel·vor 15 Jahren·discussI ran into this with my crawler, too. See http://blog.mischel.com/2008/08/08/hey-you-deleted-my-files/