The realtime magic behind Moot and how we deliver all your instant notifications(moot.it)
moot.it
The realtime magic behind Moot and how we deliver all your instant notifications
https://moot.it/blog/technology/real-timey-wimey-stuff.html
14 comments
Hmm I think you must have hit it while we were doing a deploy. Looks like the www alias points right to S3 instead of our CDN. The CDN would have hidden the bucket reconfiguration. Oops.
Will be updating the alias to point to the CDN. Thanks for catching that.
Will be updating the alias to point to the CDN. Thanks for catching that.
You're welcome!
(less welcome whoever downvoted me :))
(less welcome whoever downvoted me :))
You guys should really change your name.
the last thing any startup needs is 4chan's wrath.
You're right, the ents will be pissed ;)
It's a really good name for forum software - better than Discourse for a few reasons:
- one syllable, one easy phonetic spelling - archaic, so it won't be confused in everyday speech - it doesn't sound like it might do something else. Discourse could be online learning, golf venue rankings, etc.
It's a really good name for forum software - better than Discourse for a few reasons:
- one syllable, one easy phonetic spelling - archaic, so it won't be confused in everyday speech - it doesn't sound like it might do something else. Discourse could be online learning, golf venue rankings, etc.
The "Moot" brand is decidedly won by Chris Poole.
Ignoring the 4chan aspect, Moot is archaic and won't be confused in everyday speech? Is English not your first language?
http://dictionary.reference.com/browse/moot
One definition is of archaic usage, but the others, not so much.
http://dictionary.reference.com/browse/moot
One definition is of archaic usage, but the others, not so much.
It's used in one idiomatic expression, "the point is moot", which is almost always misused to mean 'irrelevant' rather than 'debatable'. The funny thing about idioms is that many are atomic: native speakers don't bother to dissect the individual words, they use the whole phrase because it signals something specific to other fluent speakers. When you learn a second language (French, in my case), these expressions are interesting because they usually do have an outmoded meaning, but if you ask someone they have a very hard time translating or explaining. They're also a giant pain unless you learn through immersion, because they're habitual and not usually systematic in any way.
The "archaic" usage at that link is actually very much alive, it's a technical term used in law school for mock trials [1]. I would argue law students actually understand the meaning of the term better than average, native English speakers, making the technical term less archaic.
1. http://en.wikipedia.org/wiki/Moot_court
The "archaic" usage at that link is actually very much alive, it's a technical term used in law school for mock trials [1]. I would argue law students actually understand the meaning of the term better than average, native English speakers, making the technical term less archaic.
1. http://en.wikipedia.org/wiki/Moot_court
This.
Also the word overwhelmingly is used to describe a meeting to discuss a topic throughout Europe.
It's not an unusual statement in Europe to have a moot or attend a moot. We're primarily a Finnish company so the name for us is meaningful.
Also the word overwhelmingly is used to describe a meeting to discuss a topic throughout Europe.
It's not an unusual statement in Europe to have a moot or attend a moot. We're primarily a Finnish company so the name for us is meaningful.
Totally agreed here. We love the name.
If you want something similar to this and you're on the JVM, you can use open source Atmosphere[1] which handles this flow exactly (websocket with fallback, multiple 'tags', etc).
[1] https://github.com/Atmosphere/atmosphere
[1] https://github.com/Atmosphere/atmosphere
I integrated Atmosphere into a Dropwizard-based project (not on GitHub yet, hoping to get it out there in a week or two -- look for Cyranix/mi5 if interested). Although I did get it to work, it wasn't a smooth experience. There seem to be many ways of accomplishing the same thing which are just different enough to trip you up in subtle ways (at least if you're only a moderately experienced dev like myself), and the documentation is a bit of a mess (scattered, not well translated from French, etc. -- not entirely, but enough to be a pain). Any JavaScript devs interested in this sort of tech could make a much-appreciated contribution to the project by making a client-side script that doesn't depend on jQuery.
[deleted]
"This is then picked up by every JSON-RPC server and they will intersect those two tags, sending the event {hello: 'world'} to any connection that matches."
Do you see this eventually becoming your bottleneck (every rpc server has to handle every event)? If so, what's the plan for reducing the event fan-out?
Do you see this eventually becoming your bottleneck (every rpc server has to handle every event)? If so, what's the plan for reducing the event fan-out?
Yes we do see this as becoming a bottleneck. We are already working on a solution for that. I mention in the "What's Next" section about a bit more advanced routing using ZeroMQ instead of Redis.
The problem is a difficult one to say the least. We're nowhere near the bottleneck now, but we do want to resolve this long before it is close to becoming an issue.
The problem is a difficult one to say the least. We're nowhere near the bottleneck now, but we do want to resolve this long before it is close to becoming an issue.
Could you please elaborate on how using ZeroMQ would solve your problem? I have a surprisingly similar setup and I'm also worried that this is going to be a bottleneck.
ZeroMQ doesn't solve the problem. ZeroMQ is flexible enough that we can build a solution. We're working on distributed routing and registration using ZeroMQ and Redis as it's data store.
The RPC servers register the tags and channels we distribute based on that data. Any ZeroMQ server can process and deliver any event to the end RPC server.
We'll probably do a detailed writeup when this is done. We plan on using this for all the infrastructure wiring when it's done.
The RPC servers register the tags and channels we distribute based on that data. Any ZeroMQ server can process and deliver any event to the end RPC server.
We'll probably do a detailed writeup when this is done. We plan on using this for all the infrastructure wiring when it's done.
Please do a writeup when you've done this - would love to read about this - very relevant to my work as well.
[deleted]
When you registered the domain name and selected branding, did you know moot is a well-known internet handle?
No further imformation/action after that?
It came up again halfway between then and now in comments here; I do not know of any resolution.
https://news.ycombinator.com/item?id=5486786
https://news.ycombinator.com/item?id=5486786
> tens of millions of realtime events daily
During peak hours thousands per second, off hours not many at all.
I followed it the whole way through until I got to the diagram of the way Moot handles channels compared to static channels. What does the diagram illustrate? It seems to be that the Moot version doesn't have central input. But it would all need to be filtered through something central, otherwise it is completely possible to miss a location the payload was intended to be delivered to.
That central location is Redis pub/sub. Multiple publishers publish to Redis and many JSON-RPC instances listen and then process the notifications to distribute.
The diagram is more meant to illustrate that the notifications don't just blanket deliver to all subscribers rather than be an exactly technically accurate.
The diagram is more meant to illustrate that the notifications don't just blanket deliver to all subscribers rather than be an exactly technically accurate.
Realistically, it sounds like the 2nd diagram should have the middle section of the first diagram. As in, a single message/circle, gets "split" (or more accurately dup'd) and somewhat simultaneously delivered to the start of every 'channel'. The way it is now makes the appearance that a message isn't sent down a channel where the logic gate will not eventually deliver it. But that doesn't seem to be the case.
Yeah, yeah, I know: I'm just a graphic designer without a deep knowledge of the technology... But what happens in the diagram is still pretty close to what you describe, to be fair: messages duped and delivered simultaneously. Absolute accuracy would have made for a pretty complex illustration. Point taken anyway, I'll try harder next time!
Btw, did you notice it has blue heads? I thought those were cool.
Btw, did you notice it has blue heads? I thought those were cool.
Why are they blue do they not understand the diagram they are being used in either?
Do you have any plans to open source this for people who wants to install Moot on their own server? As a forum junkie, I'm excited about Moot, but I hope you do understand that to be successful as a platform you have to give the option to self-host :)
Our infrastructure is built as a service. I totally understand that many people want to run their forum on their own network and that's totally ok. There's plenty of products out there servicing that market.
If you were to compare us to blogging companies we're more inline with tumblr than wordpress. We're a service that makes it simple for anyone to add forums and commenting.
Our service obviously isn't for everyone, but we feel like there is a large market for simple to set up and manage forums that can integrate with little to no technical knowledge or expertise.
If you were to compare us to blogging companies we're more inline with tumblr than wordpress. We're a service that makes it simple for anyone to add forums and commenting.
Our service obviously isn't for everyone, but we feel like there is a large market for simple to set up and manage forums that can integrate with little to no technical knowledge or expertise.
Why's that?
I've built something very similar using Vert.x, which is sort of like Node on the JVM. But instead of Redis, we have full on state machines at the network turn publishing on the event bus. It's pretty sweet, and I hope to open source it soon.
Are you going to open-source your json-rpc library? If so, is there a specific timeframe you have in mind? I'd really love to use this technology in my own projects.
Yes we intent to open source all of this. No timeframe in mind. We need to get through our beta roadmap before packaging up the various pieces we plan on open sourcing.
Thank you so much, that's amazing! :)
You may be interested in dnode[0] as well. Protocol documented here[1].
[0] https://github.com/substack/dnode
[1] https://github.com/substack/dnode-protocol/blob/master/doc/p...
[0] https://github.com/substack/dnode
[1] https://github.com/substack/dnode-protocol/blob/master/doc/p...
On Moot: I registered my account because I wanted to integrate Moot on my new website I'm building and when trying to integrate it to a view in Rails:
" (moot logo) startup does not exist."
Isn't this like Disqus where I can have commenting threads based on the page?
`a class="moot" href="#{request.original_url}"> Awesome Startup`
" (moot logo) startup does not exist."
Isn't this like Disqus where I can have commenting threads based on the page?
Yes. You can have commenting threads based on the page. The value on the href attribute takes a form
https\://moot.it/i/[forumname]/[an_unique_path]
You'll get the [forumname] from the setup (https://moot.it/setup) and you need to pick an unique path for each of the commenting area. The paths can be designed to fit your site's structure. For example
/blog/2013/august:my-awesome-entry /blog/2013/june:another-blog-post
You can list all comments from any path such as
/blog/2013
The "/i/" part means SEO indexed content. Our servers automatically generate a search engine optimized version of all the comments that are structured according to the path structure. Here is the SEO content for our forums for example
https://moot.it/i/moot
And here are all comments from our release blog entries
https://moot.it/i/moot/blog/release
The way we handle SEO is one of the differences from Disqus.
Here is more information about the paths:
https://moot.it/docs/embedding.html#paths
https\://moot.it/i/[forumname]/[an_unique_path]
You'll get the [forumname] from the setup (https://moot.it/setup) and you need to pick an unique path for each of the commenting area. The paths can be designed to fit your site's structure. For example
/blog/2013/august:my-awesome-entry /blog/2013/june:another-blog-post
You can list all comments from any path such as
/blog/2013
The "/i/" part means SEO indexed content. Our servers automatically generate a search engine optimized version of all the comments that are structured according to the path structure. Here is the SEO content for our forums for example
https://moot.it/i/moot
And here are all comments from our release blog entries
https://moot.it/i/moot/blog/release
The way we handle SEO is one of the differences from Disqus.
Here is more information about the paths:
https://moot.it/docs/embedding.html#paths
Interesting, but the inability to delete posts and host it myself make it useless for me. I want freedom, damnit.
Reasons for why Moot is a hosted solution: https://moot.it/docs/technology.html#distributed-service
It sounds like what you want is to run a tyranny. Freedom for you, damnit.
I wanted to follow you on Twitter so I get updates about your product, but you don't seem to have one :)
@letsmoot :)
404 Not Found
Code: NoSuchWebsiteConfiguration
Message: The specified bucket does not have a website configuration
BucketName: www.moot.it
RequestId: xxxx
HostId: xxxxx
I didn't know what "Moot" was so I tried www.moot.it