Scaling up a Mastodon server to 128K active users(gist.github.com)
gist.github.com
Scaling up a Mastodon server to 128K active users
https://gist.github.com/Gargron/aa9341a49dc91d5a721019d9e0c9fd11
6 comments
As usual so much rejection & anti-faith in the comments. Belief that progress is impossible, trying not-worthwhile. What a sad corruption of spirit!
Doubly annoying considering the forum. I know that HN is typically pessimistic but it'll never stop confusing me that a forum from YCombinator /rejects/ change and innovation. Irony, is that the word?
Consider how many people here may be invested, financially and ideologically, in maintaining the status quo or suppressing interest in potential competition. For better or worse, the "Hacker" in Hacker News refers as much to capitalism as software.
That's very true actually, and I'll admit to being naive enough to not having considered that.
Such moral wording you've got there.. "faith", "spirit", "Belief"..
Mastodon is doing fairly fine, but it does have its problems. More will be discovered along the way.
Mastodon is doing fairly fine, but it does have its problems. More will be discovered along the way.
> 128K active users
You gotta pump those numbers up, those are rookie numbers in this racket.
You gotta pump those numbers up, those are rookie numbers in this racket.
What are good numbers for a Mastodon server?
1000 at most, because your actual scaling problem is moderation, because humans don't scale.
Total profiles, or actives?
I agree that moderation is a chokepoint. And that it scales two ways:
- Moderating users, which argues for smaller instances.
- Moderating instances, which argues for larger instances.
That is: there are presently about 18,000 Mastodon instances. At 1k profiles each, this means a maximum Fediverse size of 18 million people. Note that 18k is an order of magnitude larger than the peak user limit.
Twitter sees, roughly, 200m MAU (monthly active users). Facebook sees 3 billion.
Mastodon at those scales would break both in terms of instance size and number of instances. With an even balancing, you'd have ~ 15k instances of 15k members, or 30k instances of 30k members. The overhead of moderating instances at that scale would be large, and some intermediary tier to offload that workload would likely come into existence --- effectively a hub-and-spoke hierarchical network.
The alternative would be something like present-day email, in which long-lived early-established mailservers are (mostly) grandfathered in, and increasingly draconian rules against new servers, TLDs, or even national-level (BGP) blocking comes into play.
Note: I've been on Mastodon since 2016. Played in email antispam space for a while as well.
I agree that moderation is a chokepoint. And that it scales two ways:
- Moderating users, which argues for smaller instances.
- Moderating instances, which argues for larger instances.
That is: there are presently about 18,000 Mastodon instances. At 1k profiles each, this means a maximum Fediverse size of 18 million people. Note that 18k is an order of magnitude larger than the peak user limit.
Twitter sees, roughly, 200m MAU (monthly active users). Facebook sees 3 billion.
Mastodon at those scales would break both in terms of instance size and number of instances. With an even balancing, you'd have ~ 15k instances of 15k members, or 30k instances of 30k members. The overhead of moderating instances at that scale would be large, and some intermediary tier to offload that workload would likely come into existence --- effectively a hub-and-spoke hierarchical network.
The alternative would be something like present-day email, in which long-lived early-established mailservers are (mostly) grandfathered in, and increasingly draconian rules against new servers, TLDs, or even national-level (BGP) blocking comes into play.
Note: I've been on Mastodon since 2016. Played in email antispam space for a while as well.
> and it's the one piece in the machinery that only scales vertically (i.e. with more powerful hardware)
PostgreSQL does scale horizontally ijs especially for a read-mostly app like social networking.
PostgreSQL does scale horizontally ijs especially for a read-mostly app like social networking.
Ruby on rails is a bad choice,if you want to scale.
Pleroma integrates into the network people think of as "Mastodon" perfectly well and is reputedly much lighter-weight (though its dependencies list is just as unwieldy).
The fediverse is nice because it limits censorship.
In theory. The censorship i've noticed on the fediverse is INSANE. Entire instances (Usually conservative ones) getting banned just because they get the automatic label of "Racist, Homophobe, etc" even though that is not what happening in most of them. (There are some horrible instances, not denying they don't exist).
I'm in the middle of (and searching recent Mastodon discussions on HN because someone had made some good points about surveillance / privacy aspects that I'm trying to track down) a discussion with my instance admin over a block ("silencing") of another instance as that instance hosts a few accounts of US CISA staff.
The conflict between safe space / security and highly-visible but effectively pointless gestures is ... significant.
I'll also note that I've known my admin for many years (across a few social networks, with many interactions), and generally highly regard her take and approach on matters. This is not a trivial question, though I feel she's come down on the wrong side.
The conflict between safe space / security and highly-visible but effectively pointless gestures is ... significant.
I'll also note that I've known my admin for many years (across a few social networks, with many interactions), and generally highly regard her take and approach on matters. This is not a trivial question, though I feel she's come down on the wrong side.
Limits it to the owners of the instances, who are just as (more) vulnerable to pressure from cancel campaigns.
Remember when Gab tried to join the fediverse? A bunch of instance owners got together and agreed to betray the entire premise of the fediverse by blacklisting Gab.
[bla bla bla, i don't support gab, and all other disclaimers]
Remember when Gab tried to join the fediverse? A bunch of instance owners got together and agreed to betray the entire premise of the fediverse by blacklisting Gab.
[bla bla bla, i don't support gab, and all other disclaimers]
The "entire premise of the fediverse," to the degree that such a thing exists, is the right of instance owners to choose with whom they federate and not. Blacklisting Gab is the fediverse working as intended.
However, Gab and the like are still there, and you're free to run your own instance and federate with them if you want. Maybe not Gab, last I heard they forked Mastodon and ripped out their federation code.
However, Gab and the like are still there, and you're free to run your own instance and federate with them if you want. Maybe not Gab, last I heard they forked Mastodon and ripped out their federation code.
This is why it'll never take off.
What is any normal social media but a cluster of eventually-consistent federated servers controlled by the same entity?
All of these posts just make me think "What other alternatives scale better" instead of this constant "This is why federation sucks".