HackerDalius·7 anni fa·discussCan the techniques be applied for other scrapers ? Because in the article they are discussed in the context of Web Robots scraping platform.
HackerDalius·7 anni fa·discussGood article, useful general tips that work for many scraping tools and platforms.