Ask HN: Are there any HN mashups?

1 pointsby SudarshanP16 anni fa2 comments

A lot of precious jewels lie buried deep inside HN. A few tools like like Search YC, Ask HN, Hacker Monthly exist to bring that to the surface.

What other existing tools can you think of?

Are HN archives available anywhere for HN enthusiasts/arc hackers to slice and dice?

If someone wanted to create such tools, is it generally encouraged or discouraged?

Is there some particular tool you wished, was available to surface the content of HN?

Has PG ever discouraged or prevented HNers from creating such tools?

1 comments

petercooper16 anni fa

A notable one: http://hacker-newspaper.gilesb.com/

Has PG ever discouraged or prevented HNers from creating such tools?

The only thing that I recall was discouraged was a Greasemonkey script that resulted in HN getting hammered with lots of requests on every page (I forget what it did but it did a lot of lookups on HN). People who used it had their IPs banned for a day or so.

SudarshanP16 anni fa

http://hacker-newspaper.gilesb.com/ is awsome!!!

I am planning to crawl HN using http://scrapy.org Strictly adhering to http://news.ycombinator.com/robots.txt will severely limit the usefulness of the data scraped.

User-Agent: * Disallow: /x? Disallow: /vote? Disallow: /reply? Disallow: /submitted? Disallow: /threads?

So I hope to crawl HN /x,/threads etc. Of course I plan to do it as slowly and unobtrusively as possible. If a database dump or archive is available, the whole crawling could be skipped.