Hacker News All-Time Top Links(lab.varun.io)
lab.varun.io
Hacker News All-Time Top Links
http://lab.varun.io/hn_all_time_top_links/
30 comments
I wonder what this would look like if you normalized the number of points to the total number of HN users. There is a huge bias here toward newer stories.
Interesting how few of the links are about programming. I expect this is b/c programming is largely about specific technologies, so will appeal to only segments of the readership, whereas Steve Jobs dying has a more universal interest.
I do a rolling daily list of the ones that have the most active comments at http://knowabout.it in case anyone is interested (it's how this thread caught my attention)
It'd be cool to see this adjusted for "vote inflation" -- the HN population was much smaller 2 years ago than it is now.
As another poster pointed out down-thread, this list really needs to be adjusted for inflation.
It seems dominated by links about people dying and things shutting down.
Too bad I didn't get the karma for "Show HN: This up votes itself". I think I at least deserved a few points... :)
:) I did notice it when I was making it. In fact the idea to link to google cache popped when I saw that the second highest link was dead, and figured there might be others.
Static content shouldn't require javascript.
Or cookies :(
Kind of funny that the user who posted the top ranking link -- the news of Steve Jobs death -- that was just about the only thing he posted:
https://news.ycombinator.com/submitted?id=patricktomas
https://news.ycombinator.com/submitted?id=patricktomas
It's also rather funny (or sad) that a lot of the posts are about the death of a person/project/business.
"If it bleeds, it leads".
Is the score scaled in any way, or just most points?
I'm not sure how you're collecting this data but any chance you could do (points)/(# of [registered or active] users at the time it was posted)?
Yea that would actually give a better dynamic of the number of votes. Right now the top ones are the recent ones which makes sense since the number of active users here is growing.
Exactly - the denominator is missing. But still a cool little project.
Wow, the amount of death posts.
There's an easier way to find this list.
https://www.hnsearch.com/search#request/all&q=+&sortby=point...
But, it's cool to see old posts :)
https://www.hnsearch.com/search#request/all&q=+&sortby=point...
But, it's cool to see old posts :)
For anyone who wants this for themselves, here's a Python script [0] that generates similar output [1]. It doesn't have the nice Google Cache feature from ajani's version, but it does produce static HTML which some may prefer.
[0] https://gist.github.com/tav/5545779
[1] http://tav.espians.com/temp/hacker-news-top-1000.html
[0] https://gist.github.com/tav/5545779
[1] http://tav.espians.com/temp/hacker-news-top-1000.html
An option to detect and remove broken links could be nice.
Very morbid - a lot of endings and very few beginnings. I think that makes a case for the inauspicious HN post not giving a great indicator of start-up success. For example if it were the other way around I would expect to see many many more show HN posts on this list.
I think it's just much easier to know what was than what's going to be.
"The Dennis Ritchie of this generation was just born."
I think that's not the only way to look at things. Death may be a "bad" thing, but its a natural event that, for the most part, happens on a regular cycle. The fact that these posts get up voted so much isn't an indicator of morbidity, but a sign of how much the community appreciated a person's accomplishments.
Why should Show HN posts dominate this list? By their very nature, things shown in their alpha or beta state should have a select, limited base of fans..they are at the starting point, with no full accomplishments at that point.
By when someone dies, the fact that they merit a news post, and then an HN submission, signifies that they had accomplished very much in life...the up votes are well-deserved and come out of genuine appreciation as much as remorse.
The most up voted obituary posts should be up voted the most, as they, by their nature, are about the most accomplished of our peers, idols, and mentors.
Why should Show HN posts dominate this list? By their very nature, things shown in their alpha or beta state should have a select, limited base of fans..they are at the starting point, with no full accomplishments at that point.
By when someone dies, the fact that they merit a news post, and then an HN submission, signifies that they had accomplished very much in life...the up votes are well-deserved and come out of genuine appreciation as much as remorse.
The most up voted obituary posts should be up voted the most, as they, by their nature, are about the most accomplished of our peers, idols, and mentors.
"No one wants to die. Even people who want to go to heaven don’t want to die to get there. And yet death is the destination we all share. No one has ever escaped it. And that is as it should be, because Death is very likely the single best invention of Life. It is Life’s change agent. It clears out the old to make way for the new. Right now the new is you, but someday not too long from now, you will gradually become the old and be cleared away. Sorry to be so dramatic, but it is quite true.
Your time is limited, so don’t waste it living someone else’s life. Don’t be trapped by dogma — which is living with the results of other people’s thinking. Don’t let the noise of others’ opinions drown out your own inner voice. And most important, have the courage to follow your heart and intuition. They somehow already know what you truly want to become. Everything else is secondary."
Your time is limited, so don’t waste it living someone else’s life. Don’t be trapped by dogma — which is living with the results of other people’s thinking. Don’t let the noise of others’ opinions drown out your own inner voice. And most important, have the courage to follow your heart and intuition. They somehow already know what you truly want to become. Everything else is secondary."
This. Steve Jobs dying is a bigger event that effects more people on HN than just about any Show HN ever will.
Half the posts are less than 1 year old. Wish there was a way to normalize it against active users around the posting date.
It might be interesting to plot the top 4-5 posts per month.
It might be interesting to plot the top 4-5 posts per month.
When I wrote "Don't Fly During Ramadan" (#3 post), I anticipated it'd get a few hundred hits from my friends if I was lucky. I almost didn't even post it to HackerNews, until a friend told me I should.
....Boy was I wrong.
That aside, it's sad how many of the top posts are negative in some way, even if you exclude the death announcements. The exact count depends on whether you consider things like PG banning SOPA companies to be "negative" news, but only a small handful of the top 30 links are truly positive (eg, Light Table).
It's also interesting how many (or few) of the posts are technical in their nature. I believe I count two that are completely non-technical (mine, and the other one about the TSA), and three or four that are about software that people have built (Meteor, etc.).
The rest are "in the middle" - mostly about people important to the tech community (Dennis Ritchie, Aaron Swartz, etc.), or about political issues related to technology, but not talking about the technology itself in any depth.
This pattern continues even if you look beyond the top 30.
I wonder if this has changed over time as the HN community has grown - perhaps in the morning I'll try filtering out posts less than X days old and see how that affects the top results.
....Boy was I wrong.
That aside, it's sad how many of the top posts are negative in some way, even if you exclude the death announcements. The exact count depends on whether you consider things like PG banning SOPA companies to be "negative" news, but only a small handful of the top 30 links are truly positive (eg, Light Table).
It's also interesting how many (or few) of the posts are technical in their nature. I believe I count two that are completely non-technical (mine, and the other one about the TSA), and three or four that are about software that people have built (Meteor, etc.).
The rest are "in the middle" - mostly about people important to the tech community (Dennis Ritchie, Aaron Swartz, etc.), or about political issues related to technology, but not talking about the technology itself in any depth.
This pattern continues even if you look beyond the top 30.
I wonder if this has changed over time as the HN community has grown - perhaps in the morning I'll try filtering out posts less than X days old and see how that affects the top results.
I think it's that software, even cool, amazing, never-done-before software, happens incrementally. So you see one post with a few votes announcing an MVP, another for v0.1, 0.2, 0.7, "finally 1.0", etc. Each small change is expected and there isn't really that much to see from one revision to the next. And the intial version, on its own, isn't really usable enough for anyone to immediately switch to it from what they've got, even if it does something cool, so it gets few evangelists.
Light Table and Meteor are cases where that wasn't true; where the initial version came out of nowhere and did something amazing enough that people did immediately switch to it, and evangelize it. The problem is that, for most things, this sort of "Apple press event" release requires a level of development-in-secret which I don't think is possible for FOSS software developed by more than one person.
On the other hand, really negative events--deaths, disasters, etc.--tend to be unexpected and happen all at once, so they get their impact (and "evangelism", in a sense) all network-effected together.
Light Table and Meteor are cases where that wasn't true; where the initial version came out of nowhere and did something amazing enough that people did immediately switch to it, and evangelize it. The problem is that, for most things, this sort of "Apple press event" release requires a level of development-in-secret which I don't think is possible for FOSS software developed by more than one person.
On the other hand, really negative events--deaths, disasters, etc.--tend to be unexpected and happen all at once, so they get their impact (and "evangelism", in a sense) all network-effected together.
Well, interestingly enough, not one of the top posts was created earlier than 2 years ago. This is obviously due to karma inflation and slightly akin to saying that Avatar is the top grossing movie of all time (hint: it's not[1] when you adjust for inflation). It'd be cool to see a normalized version of this. If that's even possible...
[1]-http://boxofficemojo.com/alltime/adjusted.htm
[1]-http://boxofficemojo.com/alltime/adjusted.htm
Normalizing per total users or karma has also some side effects. The demographics of HN user base has most probably changed during the past years as well as number of interesting stuff produced also changed. Now there is more submissions more competition.
I guess that is quite expected. I for one, do no care about LightTable or Meteor and haven't used either yet. So I probably wouldn't bother looking into such a link when it comes on HN. But on the other hand, a loss of a person important to tech community is something that probably affects everyone equally regardless of their technology preferences.
I would only expect this pattern to remain this way.
I would only expect this pattern to remain this way.
I wonder if this has changed over time as the HN community has grown - perhaps in the morning I'll try filtering out posts less than X days old and see how that affects the top results.
Note that popularity in terms of votes should also be normalized with respect to traffic/users.
Note that popularity in terms of votes should also be normalized with respect to traffic/users.
That'd be an interesting exercise.
Here's the fetched dump: Might be quicker to use:
http://lab.varun.io/hn_all_time_top_links/top.json
Thank you - you saved me having to bust out an ad-hoc web scraper. :)
To answer the questions around source and weighting etc:
1: I'm using https://www.hnsearch.com/api, which the page itself seems to indicate is used by HN for certain views.
2: The url is http://api.thriftdb.com/api.hnsearch.com/items/_search?prett... The API won't let you fetch more than 1000 results in total (using limit and start in tandem).
3: I'm fairly certain the API is fairly new because I never came across any results older than 2010 while fiddling with it. As such the couple of suggestions of weighting by active users might not return the desired balanced result.
4: I'm fetching the results at one go (all 1000) from the API through a cron the runs once every 48 hours.
1: I'm using https://www.hnsearch.com/api, which the page itself seems to indicate is used by HN for certain views.
2: The url is http://api.thriftdb.com/api.hnsearch.com/items/_search?prett... The API won't let you fetch more than 1000 results in total (using limit and start in tandem).
3: I'm fairly certain the API is fairly new because I never came across any results older than 2010 while fiddling with it. As such the couple of suggestions of weighting by active users might not return the desired balanced result.
4: I'm fetching the results at one go (all 1000) from the API through a cron the runs once every 48 hours.
18th most popular HN story ever... w00t :)
How is this calculated? By raw points I should have somewhere around #60, but my top post is not listed. I don't think it's a great post (just me bitching about scores being taken away) so I guess I'm glad, but still curious.
2 of the top 4 posts, and 4 of the top 20 posts are from less than one month ago. A sign of HN's rising popularity.
Yes. A "best of all time" should account for "inflation".
Which are the most commented posts?
Wow, I'd never read A Sister's Eulogy for Steve Jobs before.
"His breathing changed. It became severe, deliberate, purposeful. I could feel him counting his steps again, pushing farther than before.
This is what I learned: he was working at this, too. Death didn’t happen to Steve, he achieved it." -Mona Simpson
"His breathing changed. It became severe, deliberate, purposeful. I could feel him counting his steps again, pushing farther than before.
This is what I learned: he was working at this, too. Death didn’t happen to Steve, he achieved it." -Mona Simpson
Glorify more?
Whoever did this, I'm a newbie to coding, and I'd want to see the code used to pull this. I'm assuming it wasn't done manually. I've been thinking I'd want to do something similar once I get the hang of it, and to play around with it. "Hall of fame" comments or articles aren't the only ones someone new to HN would be interested in, right? How about, for example, the speed at which an article rises to the top? Or how long it remains at the top? And so on and so forth. You could probably try to draw an analogy to music charts. I don't know. But I'm working on knowing. So if you don't mind sharing, I'd appreciate.
Oh yeah, plus I keep hearing about how the quality of HN has dropped. I imagine you could selectively pull stuff from back in the day, apply the same popularity criteria to today, and compare the quality of what comes up. It's small curiosities like this I want to learn how to pull off.
Steve Jobs obituary is the #1 post. Hats off to one of the greatest innovators of our lifetime.
This post is broken to me right now. (Monday, 9/9 5am EST)
The URL it points to is: http://lab.varun.io/hn_all_time_top_links/
But the page looks like this in my browser:
{{key + 1}}. {{link.item.title}}
({{link.item.domain}})
{{link.item.points}} points by {{link.item.username}} {{link.item.create_ts | fromNow}} | Google cache | {{link.item.num_comments}} comments on HN
The URL it points to is: http://lab.varun.io/hn_all_time_top_links/
But the page looks like this in my browser:
{{key + 1}}. {{link.item.title}}
({{link.item.domain}})
{{link.item.points}} points by {{link.item.username}} {{link.item.create_ts | fromNow}} | Google cache | {{link.item.num_comments}} comments on HN
As dumb as it sounds this is because the site requires Javascript to load its static content.
Unfortunately sad titles prevail.
If you like this, you might enjoy my Wayback Letter (http://waybackletter.com) which sends you the best articles from each year of HN every week. Here is last week's issue: http://eepurl.com/EGWOv.
Not surprised that the most pandering stuff (especially "Don't Fly During Ramadan") is at the top.