Why HN Got Slow

152 pointsby pgvor 15 Jahren58 comments

We got MSMed. The post about baseball bat sales in the UK was the perfect media bait and they all linked to the item page.

I turned on comment thread pagination to decrease the load. I'll turn it off tonight.

59 comments

hoodoofvor 15 Jahren

Forgive my lack of worldliness but what is "MSMed"?

shadowmattervor 15 Jahren

Mainstream media'd. Meaning someone from the mainstream media posted a link to that post, and the surge in traffic made everything slow.

stevenpvor 15 Jahren

Main-stream media.

mainguyvor 15 Jahren

I was searching on google trying to figure this out myself...

samstavevor 15 Jahren

Slashdotted Digged Reddited

Pick one

yuvipandavor 15 Jahren

They aren't considered MSM.

NYT, Slate, Guardian, etc., rather.

samstavevor 15 Jahren

It was a historical reference

icefoxvor 15 Jahren

2nded, I haven't heard 'Slashdotted' to mean getting traffic from slashdot.org in a while, but it does seem to have stuck as the means of saying we got a ton of traffic from some other website linking to us.

watmoughvor 15 Jahren

Your point is pretty reasonable, considering the meme sprang to popular awareness in it's 'Slashdotted' form, doubtless melting many servers down prior to appearing in the MSM.

jerryavor 15 Jahren

We got MSMed. The post about baseball bat sales in the UK was the perfect media bait and they all linked to the item page.

Hmm. The past 48 hours a lot of sites have been slow for me: HN, Salon, FARK, ...

I figured I was getting Cox'd. http://cox.com

loire280vor 15 Jahren

Cable providers provide terrible DNS service. You may notice a big difference in performance by changing your DNS settings to http://code.google.com/speed/public-dns/ or http://www.opendns.com/. As a bonus, I've noticed that many service interruptions are DNS failures, not routing issues.

dvdhsuvor 15 Jahren

You might want to try NameBench, which will find the fastest DNS server for you.

http://code.google.com/p/namebench/

ghshephardvor 15 Jahren

Be Careful about changing your DNS settings if you use a CDN - iTunes, in particular, offers horrible performance if you use googles DNS.

ComputerGuruvor 15 Jahren

Why do you think iTunes would care what DNS server you are using? I'm genuinely curious, as I would think that the DNS lookups are abstracted by the OS, and even if they were not, I cannot see how using a different (esp. a better performing) DNS server would hinder iTunes performance — especially since the original DNS isn't provided by Apple or anything.

ryannielsenvor 15 Jahren

CDNs often rely on your DNS info to determine which node is closest to you and, thus, which node should offer the best performance.

Using a central DNS like Google's can do two things:

1) Break locality. Using 8.8.8.8 may cause CDNs to potentially direct you to a node that's nowhere near your actual physical location

2) Aggregate traffic on a node. The more people using 8.8.8.8 for DNS, the more people who may land on the CDN nodes associated with 8.8.8.8's location and thus that node may be more loaded than the one closer to you

[edit] Google even covers this in their Public DNS FAQ: http://code.google.com/speed/public-dns/faq.html#cdn. They offer 31 different locations, while Akamai may offer thousands of nodes (I can't find hard numbers, but they refer to 1,200+ points of presence here http://www.akamai.com/hdonline and 1,000 networks here http://www.akamai.com/html/technology/edgeplatform.html). You're basically constraining yourself to a small subset of at least Akamai's CDN by using Google Public DNS.

ComputerGuruvor 15 Jahren

That completely slipped my mind. You are, of course, correct.

An alternative suggestion here is to use Comcast's 75.75.75.75 IP address; while it may seem static, it's actually an anycast address, and will resolve to the nearest node.

One thing many people forget when choosing a DNS server is that it's not just the ping time that counts - the time it takes the server to reply to the actual DNS request is what matters.

e.g. while 75.75.75.75 for me (Comcast Chicago) is a 15ms trip and 8.8.8.8 is a 28ms ping, the lookup on the former (dig x.com @dns_ip) can reach 120ms while on Google's DNS it never exceeds ~35ms.

michaelcampbellvor 15 Jahren

> CDNs often rely on your DNS info to determine which node is closest to you and, thus, which node should offer the best performance.

Why do they do this? Wasn't DNS SUPPOSED to be something that wasn't in any way tied to locality? Are CDN's just using the side effect/fact of life that they happened to fall out that way, because it's easier?

ComputerGuruvor 15 Jahren

Pretty much, yes. But it's not just easier, it's also simpler. If you do the distribution at the DNS level (which is pretty much as low as you can go), you don't have to deal with the much messier geo-distribution on a higher level.

blinkingledvor 15 Jahren

Absolutely - change the DNS _and_ put something like DD-WRT on your router with dnsmasq as the local caching proxy server. Nothing beats that per Steve Gibson's DNS benchmark!

Terrettavor 15 Jahren

Was this an inside joke?[1]

1. http://allthatiswrong.wordpress.com/2009/10/11/steve-gibson-...

2. http://news.ycombinator.com/item?id=2510614

blinkingledvor 15 Jahren

It wasn't - this is just a DNS benchmark that does DNS resolution in loop and measures how long it took etc. Nothing requiring security expertise.

Besides I don't understand why few people are after Gibson - he is generally on the mark - that's not to say he doesn't make mistakes - but nothing that makes me want to totally ignore him.

jerryavor 15 Jahren

Thanks, but I actually do that using both Google's (8.8.8.8), as well as Level 3's (4.2.2.1)

knownvor 15 Jahren

Debug the connection with http://netalyzr.icsi.berkeley.edu/

jerryavor 15 Jahren

Thank you.

Vexenonvor 15 Jahren

I was wondering why the load times have been ridiculously long.

Also: I've been getting Cox'd as well the past three or so days. Maybe I should complain, or something.

jerryavor 15 Jahren

fyi, cox has their own local speed tests which can be informative as to where the hangup might be. The local one to me is:

http://azspeedtest.cox.com/

It's actually somewhat frustrating to see that register phenomenal speeds when the web past their internal routers seems ... buffering ... buffering.

kleegvor 15 Jahren

I figured I was getting Cox'd

That's what she said...

I noticed the same with Fark as well. Also HuffPo.

rdlvor 15 Jahren

Will we have International Haskell Day later this week?

dschoonvor 15 Jahren

You know, I still have a 50-odd Erlang article queue. So I'm not saying I want every day to be Haskell Day, but I'm also not saying I'd be terribly upset if I had a few hundred articles on Haskell to keep me warm at night...

niels_olsonvor 15 Jahren

What do you use for this queue? I can't possibly keep up with all the reading I come across, and often just relegate it to Readability, Instapaper, Delicious, Xmarks, etc. Would be very interested to hear how someone else handles this.

bluehatvor 15 Jahren

I have a historical Erlang article http://news.ycombinator.com/item?id=2866337

Locke1689vor 15 Jahren

Now there's a joke for the old-timers.

nitrogenvor 15 Jahren

Those front pages filled with articles on relatively obscure programming languages are actually what got me to start reading HN.

Joakalvor 15 Jahren

PG, can you do the complimentary blog post "Well, these are my statistics to HN before and after getting MSM'd"

[deleted]vor 15 Jahren

fnamevor 15 Jahren

Link to the article?

nm, here it is: http://www.pcworld.com/article/237616/baseball_bat_sales_ris...

InfinityX0vor 15 Jahren

Crazy that a site such as PCWorld could bog down a site as popular (or so it seemed) as Hacker News. Goes to show what kind of a small fish Hacker News is in a much bigger pond..

ehsanu1vor 15 Jahren

Well, I'd be surprised too if it was just due to PCWorld, but it's more than that. The guardian and the telegraph, and many other mainstream news outlets have gotten wind of this story: http://www.google.com/search?client=opera&rls=en&q=b...

InfinityX0vor 15 Jahren

Ah, I was using Yahoo! Site Explorer as a judge.. which only showed PCWorld. http://siteexplorer.search.yahoo.com/search;_ylt=A0oGk.jo20F...

JoeCortopassivor 15 Jahren

I think you missed the point. What happened, was everyone (actual news organizations) linked to HN because the top link had a "scandalous" title. This wasn't because of a link from PCWorld

jvehentvor 15 Jahren

I'd be curious to see some numbers. Just for fun, and also to get an idea of the amount of visitors that kind of link can bring.

Capacity planning, they say :)

jbseekvor 15 Jahren

Thanks PG, you must wear many hats.

LearnYouALispvor 15 Jahren

What ever it is, it's always a white one. ("Yeaaaaah")

I was looking at [a list of hats](http://en.wikipedia.org/wiki/Hat) recently, by the way.

kunleyvor 15 Jahren

Looks like HN has now some focus of link spammers.. see recent submissions.

dennisgorelikvor 15 Jahren

Can caching help?

pitdesivor 15 Jahren

Interesting that that post itself (http://news.ycombinator.com/item?id=2864557) is now on the 2nd page despite having 400some upvotes in the past 7 hours (odd because there are several posts with many less votes in a similar timeframe on the homepage).

Does that happen due to being flagged too many times or some other reason?

pbhjpbhjvor 15 Jahren

Speculation: there is an authority metric which is used as a strong quality indicator. If you're considered an authority then your vote counts a lot more.

7 votes from folks like patio11 counts for about a million votes by the likes of me.

Like I said, speculation.

alex_cvor 15 Jahren

I'm pretty sure it has to do with flagging. (that, as well, is speculation combined with personal observation).

niels_olsonvor 15 Jahren

Would norvig be on the list? aaronsw? I keep meaning to build a list of celebrity CS people on HN. Anybody already got one?

nitrogenvor 15 Jahren

This would be a good start: http://news.ycombinator.com/leaders

What amazes me is that as much as I read HN, there are still people in the top 10 whose comments and submissions I've never seen. I wonder if they got all their karma years ago and it's taking this long for everyone else to catch up.

JoachimSchippervor 15 Jahren

To be fair, some of them mostly submit stories (compare e.g. tptacek and fogus; the first has lots of comments but few stories, the other is the other way round.)

niels_olsonvor 15 Jahren

That's almost exactly what I'm not interested in. There are some very interesting people with account who post very infrequently, but their comments, in the context of who they are, is very interesting indeed.

d_rwinvor 15 Jahren

true to the word. " What would you believe , maximus?"

community tagging could not be the solution here- what do you think?

--Stackoverflow blog offers some explanation.

Sundogvor 15 Jahren

So...we're Digg then?

Sundogvor 15 Jahren

Yikes...THAT wasn't the right to say

zobzuvor 15 Jahren

I'm wondering also if some who down vote get a bigger down vote ratio. usually i lose points 1 by 1 and sometimes i just reload and lose 10 or more. Or that's cache. ;-)

And yeah I've strong opinions so while sometimes i get crazy upvotes I also get crazy downvotes hehe.

graywhvor 15 Jahren

There's no down voting on articles. And for karma, 1 vote = 1 point.

mellingvor 15 Jahren

Any time to address the endless number of reposts and the increase in volume of submissions?

benologistvor 15 Jahren

I'd like to see accounts that only submit the same site over and over again and don't otherwise participate penalized. That'd clear up the 'new' list a lot as well as dampen the duplicate submissions.