Are HN submissions dominated by a small elite of users?
11 comments
If you're talking about submission karma, clearly no. I follow the submissions of a selection of HN users I've known for awhile, all of whom are karma leaders, and their submissions are often as not buried on page 4 or 5 with 2-3 votes.
If you're talking about comment karma, clearly yes. I could write a comment consisting of randomly chosen names from the Chicago phone book and someone would upvote it. Name recognition has that effect, as does a small group of people who seem to follow my comments deliberately (as I do for the list of people in my profile).
If you're talking about comment karma, clearly yes. I could write a comment consisting of randomly chosen names from the Chicago phone book and someone would upvote it. Name recognition has that effect, as does a small group of people who seem to follow my comments deliberately (as I do for the list of people in my profile).
Not just name recognition, but a tptacek/pg/patio11 comment is landed #1 on the comment page and stays there a long time just because of karma. Then, occupying the most privileged real state on the page, after thousands of views, even if it is a small percentage that likes the comment, it is a lot of upvotes.
Not that I disagree with this algorithm, I actually agree. Just saying that I think the position of the comment is more important than name recognition in this case.
Not that I disagree with this algorithm, I actually agree. Just saying that I think the position of the comment is more important than name recognition in this case.
grellas posts are particularly buried because they often don't subscribe to the hivemind, but they are always enlightening.
Keep in mind that karma includes both submissions and comments (unlike reddit, where they are separate).
For comments, at least, there is a clear bias where comments from users with higher karma tend to stay at the top of the comment page longer, regardless of the votes for other comments. This is intentional and definitely contributes to the effect
For comments, at least, there is a clear bias where comments from users with higher karma tend to stay at the top of the comment page longer, regardless of the votes for other comments. This is intentional and definitely contributes to the effect
Sort of a "founder effect" built into a site for founders. How meta.
I suspect that you'll find it to be some kind of power law distribution, which implies that measures like Standard Deviation won't be so useful. A quick look at the top 1000 that you provided suggests this might be the case. But the top 100 users don't dominate the top 1000, let alone the entire population. There does seem to be some leveling off, though it's unclear how steady that will be.
Net - domination is too strong a word. Dominated might imply 20% of the karma in 20 users, and this clearly isn't the case. Even defining the Elite as 100 doesn't get us this.
Net - domination is too strong a word. Dominated might imply 20% of the karma in 20 users, and this clearly isn't the case. Even defining the Elite as 100 doesn't get us this.
Yes, take a look at the "new" page. On it you'll see 30 recent submissions, most of which are just as appropriate as those on the home page. But, they don't get any votes, why not?
I don't think it typically matters whether the submitter has name recognition or karma, but rather how many friends and colleagues the person can rally to their cause. All it takes is about 10 points in the first hour to get to the front page and then it can "ride the wave" from there. This is an easy threshold for people with large professional networks, companies, with friends in SV et al. Many people here know each other. Yes, there are some technical counter-measures, but there is only so much that can be done.
This is not to say that a user acting alone has no chance of reaching the home page. However, after a few years of frequenting this site, I believe that most of the posts hitting the front page have had a boost such as I described above.
My personal experience anecdote is that, over several years (also considering prior accounts), working as a "loner", that I've never been able to get more than a few meager votes on a submission. Even if they were highly on topic, even if I got up at 6am Pacific, supposedly the best time to do it. Maybe I'm unlucky or my submissions suck, but I rarely bother any more.
I don't think it typically matters whether the submitter has name recognition or karma, but rather how many friends and colleagues the person can rally to their cause. All it takes is about 10 points in the first hour to get to the front page and then it can "ride the wave" from there. This is an easy threshold for people with large professional networks, companies, with friends in SV et al. Many people here know each other. Yes, there are some technical counter-measures, but there is only so much that can be done.
This is not to say that a user acting alone has no chance of reaching the home page. However, after a few years of frequenting this site, I believe that most of the posts hitting the front page have had a boost such as I described above.
My personal experience anecdote is that, over several years (also considering prior accounts), working as a "loner", that I've never been able to get more than a few meager votes on a submission. Even if they were highly on topic, even if I got up at 6am Pacific, supposedly the best time to do it. Maybe I'm unlucky or my submissions suck, but I rarely bother any more.
My impression is that suppression of voting rings, which is what you're describing with your "group of 10 friends" example, is something that the HN maintainers actually spend a lot of effort on; I get this both firsthand from the maintainers that I actually know, and from the fact that I feel like I regularly manage to do (innocuous) things that trip the detector.
HN voting is a trivial example of selection bias. If frontpage was determined by a random sampling of the user base (weighted by karma or age if you like), instead of whoever may go looking for the post, it would be more fair. One of the reddit devs suggested something like this recently, putting a New post on every page.
They do, but avoiding the counter-measures isn't that hard if you're aware of them, and importantly, only needs to be done a few times a year for each combination of friends.
I could be mistaken, but the hypothesis fits so far.
I could be mistaken, but the hypothesis fits so far.
I have no friends on HN and I tell no one when I'm submitting something and yet I've managed to post things that get to the front page fairly regularly.
Looks like you have a roughly ten percent success rate. Sounds reasonable.
A little late to this, but for my Hacker Newsletter project I keep track of of the number of times a user gets on the top three pages:
337 shawndumas
312 ColinWright
298 iProject
291 Libertatea
214 danso
197 llambda
165 Lightning
163 tokenadult
158 sethbannon
147 mtgx
144 evo_9
120 DanielRibeiro
115 denzil_correa, jonbaer
111 lelf
102 001sky
101 dmor
98 coloneltcb
97 Tsiolkovsky, craigkerstiens, dsr12, mh_
95 ssclafani
90 jseliger
89 jamesbritt
87 JumpCrisscross
85 Garbage
83 wallflower
82 luu
81 antr, protomyth
77 anigbrowl, codegeek
74 aaronbrethorst
73 jgrahamc, pg
71 uladzislau
70 sinak, steveklabnik
69 RougeFemme
68 cleverjake, derpenxyne, swohns
67 _pius
66 jamesjyu, rosser, wslh
64 Pr0, eplanit
63 Brajeshwar, sk2code, uptown
62 bane, stfu
61 teawithcarl
59 ghosh, joeyespo, ph0rque
58 gnosis
57 r0h1n, scholia
56 AndrewDucker, cpeterso, ohjeez
55 bpierre, cyphersanctus, gruseom
54 conductor
52 nikunjk
51 Suraj-Sun, kirillzubovsky
50 lispython
49 MikeCapone, WestCoastJustin, co_pl_te, duck, robin_reala
48 Anon84, ca98am79, taylorbuley
47 fraqed, morphics, nkurz, rpm4321
46 jmduke, kumarski, nreece
45 georgecmu
44 daegloe, rdl
43 donohoe, kunle, olivercameron
42 tanglesome
41 austengary, ck2, jnazario
40 interconnector, rnyman, sonabinu, wikiburner, zoowar
39 cdvonstinkpot, pwg, revorad
38 1337biz, hepha1979, jeffbarr, mhb, zdw
37 fogus, kjhughes, vectorbunny
36 Cbasedlifeform, DanBC, DiabloD3, Reltair, acremades, geetarista
35 AndreyKarpov, jessaustin, mxfh, prostoalex, raganwald, rbanffy
34 wglb
33 CrankyBear, cryptoz, hackhackhack, kposehn, nvk
32 MarlonPro, bcn, npguy, qubitsam, techinsidr
31 Ashuu, Baustin, caffeinewriter, drucken, edent, iamtechaddict, ilamont, mariuz, mikecane, rmah, rohshall, shill, whoishiring
30 eguizzo, eksith, gklein, lukashed, talhof8
29 Sami_Lehtinen, ari_elle, cwan, espeed, gwern, jacquesm, mitmads, sgdesign, superchink, velodrome, wyclif
28 Quekster, X4, clicks, esalazar, iamwil, jaf12duke, javinpaul, laurent123456, recoiledsnake, vanwilder77, zt
27 6thSigma, ValentineC, aespinoza, ananyob, anu_gupta, awwstn, bavidar, dave1010uk, ekm2, pajju, rkudeshi, sdoering, yapcguy
26 DanielBMarkham, bitsweet, brudgers, hornokplease, jcr, tptacek, whalesalad
25 EzGraphs, Hirvesh, adventured, albertzeyer, barredo, johns, jpadilla_, kevin_morrill, kmfrk, mmastrac, mooreds, mrb, niggler, ot, prateekj, saurabh, shrikant, tambourine_man, thibaut_barrere, tjaerv, usaphp
24 amerf1, coffeemug, dangoldin, neya, qiqing, signa11, twapi, w1ntermute, zrail
23 Dekku, Shivetya, duggieawesome, ingve, jalanco, mikeevans, ninthfrank07, npalli, olalonde, playhard, rachbelaid, rb2e, subsystem
22 adamnemecek, amarsahinovic, aritraghosh007, chmars, colinprince, ctoth, dutchbrit, eranation, jashkenas, jmacd, joshfraser, kine, lowglow, mike_esspe, muratmutlu, napolux, redDragon, rrhoover, tareqak, youngerdryas, yread, ytNumbers
21 T-A, akandiah, anon1385, breck, dpaluy, glazskunrukitis, gridscomputing, homakov, ianstormtaylor, jusben1369, kunai, mindcrime, msvan, onosendai, peterkchen, prajjwal, pron, schrofer, sp332, treskot, ttunguz, urlwolf, ximeng
20 BerislavLopac, BruceM, Jaigus, a3voices, abdophoto, angersock, boh, brandnewlow, chinmoy, chrisacky, cperciva, fekberg, ivoflipse, jasonshen, julien, llamataboot, mjn, mmahemoff, platz, shakes, smit, soundsop, speeder, tmoretti, tocomment, vellum
19 RobAley, SanderMak, WadeF, bencevans, bitcartel, clarkm, declan, experiment0, gasull, goronbjorn, hartleybrody, jejune06, jkuria, k-mcgrady, kevingibbon, mijustin, minimaxir, njoglekar, obilgic, ohadfrankfurt, pclark, rainmaker23, robdoherty2, rubikscube, sciwiz, stevewilhelm, tdrnd, timf, twakefield, wamatt
18 10char, ForFreedom, Pasanpr, aynlaplant, bsg75, cs702, deviceguru, dotmanish, ezl, jheitzeb, jstreebin, julien421, orrsella, pain_perdu, philfreo, polskibus, rblion, ridruejo, rmason, sew, swombat
17 MaysonL, ananddass, barmstrong, bconway, bergie, crabasa, dmmalam, dsego, dshankar, equilibrium, fejr, ferdo, gebe, jkopelman, josephby, mikeleeorg, mkrecny, nashequilibrium, petenixey, petercooper, petrel, profquail, ra, replicatorblog, songzme, spking, taytus, ujeezy, wrongc0ntinent
16 Adrock, Impossible, LiveTheDream, StavrosK, aashaykumar92, afshinmeh, ajaymehta, cdl, choult, damian2000, danboarder, daw___, electic, esolyt, followmylee, franze, giis, greenyoda, gregpurtell, hunvreus, jdorfman, jdp23, jellyksong, jorde, jordn, karamazov, linux_devil, lleims, mahmoudimus, maskofsanity, mjhea0, narad, nate, p4bl0, peter123, rfreytag, sciurus, sebg, sharkweek, spindritf, stevewillensky, tosh, tshtf, xmpir, zengr
15 Ataub24, FredericJ, MIT_Hacker, Mithrandir, adulau, apress, areski, austenallred, babawere, bhauer, bitops, bmmayer1, bradleybuda, ckelly, cooldeal, czr80, dan1234, danielpal, gits1225, grey-area, groundCode, hamidr, hawkharris, jeffreyfox, lucb1e, mayop100, mikeknoop, pdknsk, relation, robbiet480, rodriguezcommaj, rpsubhub, sasvari, sheri, smacktoward, thejteam, thinkcomp, vinhnx, xtraclass
14 0cool, Alex3917, ComputerGuru, Peroni, SparksZilla, amazedsaint, anons2011, apoorvamehta, bifrost, blacktulip, bradly, brandonb, bslatkin, c-oreills, carlosgg, charlieirish, chwolfe, codelion, davidroberts, dbaupp, dendory, dennybritz, draegtun, dsl, e1ven, edwintorok, enneff, glazemaster, grej, harryzhang, ishener, ivankirigin, japhyr, jarederondu, jkaljundi, johndcook, joshuacc, justincormack, kamaal, kanamekun, lisper, mathattack, networked, nickmain, pjvds, pragmatictester, pykello, robg, sabalaba, samspenc, sinnerswing, technologizer, ternaryoperator, thewarrior, timr, titlex, watermel0n, wlll
13 6ren, AlexMuir, CrunchyJams, DavidChouinard, Kopion, acav, alter8, asanwal, benjlang, bra-ket, brettcvz, bryanh, bussetta, chrismealy, cinquemb, cpleppert, creamyhorror, cwilson, davidw, detcader, dfc, digisth, ekianjo, endtwist, gatsby, glaugh, gulbrandr, ics, jfaucett, jlongster, kn0thing, larrys, lenkendall, libovness, lukaseder, mactitan, marban, mgunes, michaelrbock, neeee, noinput, palidanx, paulschlacter, plessthanpt05, porker, pytrin, rjsamson, rohin, rpledge, samsolomon, sebkomianos, selmnoo, sethev, smaili, srathi, t0dd, techaddict009, tellarin, tilt, tlrobinson, tomasien, trendspotter, turoczy, vyrotek, waffle_ss, waterlesscloud, weu
12 31reasons, ChuckMcM, CrazedGeek, DaNmarner, Ecio78, Fletch137, GuiA, ISL, IgorPartola, JacksonGariety, Kilo-byte, Rickasaurus, RockyMcNuts, Sealy, SteliE, aaronpk, abraham, achalkley, akos, alexholehouse, ankitoberoi, anujkk, bberson, bcl, bjansn, bjonathan, brianchu, btilly, bwertz, casca, cgi_man, changdizzle, chaz, chriscampbell, cocoflunchy, cramforce, d4vlx, daigoba66, dandrewsen, dctoedt, deusclovis, dmoney67, doh, dwynings, eladgil, enmaku, evolve2k, feelthepain, flippyhead, ghshephard, googletron, gz5, hansy, hkmurakami, hornbaker, irollboozers, jnoller, joxie, juliangamble, kareemm, kitcar, metajack, microwise, mindstab, mmariani, morisy, mpweiher, negrit, nekojima, nqureshi, paraschopra, pascal07, pauljonas, pbiggar, pearjuice, peterkelly, poissonpie, robheaton, seminatore, sherm8n, spooneybarger, stesch, taigeair, thegarside, theoutlander, tikhonj, tnorthcutt, uvdiv, vog, vu0tran, weisser, yeleti, zan2434
11 Amadou, BIackSwan, Cieplak, Swizec, TDL, adrianhoward, aelaguiz, afschwartz, alok-g, aram, benhowdle89, benwerd, cstross, darxius, davidsmith8900, dboles99, decklin, devx, diggan, dkasper, dohertyjf, eduardordm, edw519, eibrahim, ekpyrotic, elleferrer, epenn, fatiherikli, frankdenbow, friism, guiseppecalzone, haven, hudibras, iand, igrigorik, immad, infoman, jacoblyles, jamest, jbaudanza, jdmitch, jgv, jipumarino, johnjlocke, jordanmessina, jpmc, judegomila, kloncks, knes, kogir, ldayley, leephillips, lest, lettergram, luigi, marcieoum, mazsa, memoryfailure, moonboots, morefranco, nbashaw, ndesaulniers, nonrecursive, nsns, nsp, null_ptr, octopus, oleganza, patrickaljord, pavel_lishin, philip1209, rasengan, rdemmer, remi, rfnslyr, rvivek, salimmadjd, sgrove, shared4you, sillysaurus2, simonreed, spdy, speednoise, stollercyrus, swannodette, teamgb, thehodge, tippytop, tlongren, tomse, tortilla, trevin, twog, ukd1, whit537, willvarfar, wiradikusuma, xijuan, yitchelle, zachinglis, zhs
10 AirbnbNerds, Avalaxy, CaptainZapp, Danieru, JanLaussmann, MattRogish, Maven911, Mitt, SonicSoul, Xcelerate, andreiursan, andrewnez, antman, austinhallock, bdehaaff, bearwithclaws, bevenky, binarybits, blackhole, brown9-2, ceeK, chaostheory, cheeaun, chewxy, chrislloyd, clarky07, cobrausn, cyang08, dazbradbury, dcope, denysonique, dhotson, diminium, dirkk0, dmitri1981, dsowers, dylangs1030, dz0ny, edtechdev, fecak, felipebueno, fmavituna, giorgiofontana, gkuan, gregman, hboon, hippo33, hodgesmr, imkevinxu, jacobwg, jashmenn, jayadevan, jazzychad, jballanc, jf, jkarneges, josh2600, jpatokal, jvns, jwallaceparker, k33l0r, kafkaesque, kinlan, kintamanimatt, krat0sprakhar, kyledrake, lukedeering, maccman, malloc47, marojejian, matt1, maudlinmau5, mdturnerphys, mecredis, milesf, misiti3780, mjfern, molecule, mparramon, mschonfeld, nathanbarry, nileshd, ovechtrick, pavs, pearkes, petethomas, phenylene, philk10, pkrein, r4vik, radley, ramisms, robhawkes, rocky1138, rsobers, sidcool, siong1987, smanuel, stefan_kendall, sw007, swah, swampthing, swanson, sweis, techdog, thingsilearned, timjahn, tylermenezes, ukdm, will_brown, wsieroci, yesplorer, yonasb, zacharyvoasedoes this project have a page?
I accidentally posted that with my other account. Yes - http://hackernewsletter.com
thx. is there an article on the aforementioned data?
(oh, and I submitted a link to your newsletter. maybe we can get it on the front page!)
(oh, and I submitted a link to your newsletter. maybe we can get it on the front page!)
Look through the submission list of each user whose posts make it to the top, or of all of the users submitting to new. You'll see if the stories each user posts gains lots of points; if they don't, it's all comment karma.
Any Karma measure analysis would be biased since Karma is accumulated through both Links and Comments (in contrast with Reddit, which splits both measurements).
"Dominated" is a flawed qualifier since Karma is not a zero-sum game.
"Dominated" is a flawed qualifier since Karma is not a zero-sum game.
There's a book you should read called "The 80/20 Principle". It might help you understand better how this sort of thing naturally happens, and why it is to be expected and nothing wrong about it.
No, they are dominated by the common denonimator and that is a massive flaw of vote-based user input. You will only see politically correct, left winged and in-tone liberal opinions and support everywhere. Mutual backpatting is encouraged and because of these traits, the circle continues.
echo chamber depending on first (maybe most numerous) adopters that self-reinforces by driving away dissenting voices. I agree 100%
I thought it would be easy to find out how karma is distributed among users using the HNsearch API.
Unfortunately, it's not that easy because this API is not optimized for fetching massive amounts of data. What I could do was looking at the 1000 users with the most karma. I separated those into ten chunks and looked at the total sum of Karma per chunk of 100 users. This is the result:
http://www.chartgo.com/share.do?id=6cca3d09aa
(Data: [ (0, 2941707), (1, 1314760), (2, 936226), (3, 745944), (4, 636533), (5, 557441), (6, 493996), (7, 449999), (8, 413963), (9, 384491)] )
So as you can see the 1st chunk is pretty dominant but then it kind of flattens out. So are HN submissions dominated by a small elite? It obviously depends on your definitions, but I guess not.
It would be interesting to have better data available. Did I miss a better way to query the API? Here's the Python script which I used to query the API: https://gist.github.com/johannes-gehrs/77a92284e3509f2ae960