Centrifugo – real-time messaging server migrated from Python to Go(github.com)
github.com
Centrifugo – real-time messaging server migrated from Python to Go
https://github.com/centrifugal/centrifugo
4 comments
Hi!
1) It was very interesting and challenging sometimes. I remember several difficulties during migration: many usages of interface{} values at first stages which then be replaced with more strong types, several possible races which Mr Klaus Post (https://github.com/klauspost) helped to find, he also noticed at some problems in code style and channel buffer overflow problem. But the greatest part of this migration is that I don't need anymore think "Is library I use non-blocking or not?" as Go's built-in concurrency allows to use all libraries written for language. And also after Tornado it's great to have tools to inspect active goroutines. I also personally don't like how many locks I use at moment - I hope to refactor several parts of code to use channels for synchronization.
2) Elixir is great, but unfortunately I had no experience with it - just read several articles. Centrifugo is not massively scalable at moment - it does not support Redis cluster yet - this is an area for investigating. So now it is limited to Redis throughput and suites well for small or medium projects. So the answer here: Erlang is absolutely great for software like this but I have chosen a language in which I could implement all that needed in a reasonable time. Go has also its unique features so at moment I am pretty happy that I started migration.
1) It was very interesting and challenging sometimes. I remember several difficulties during migration: many usages of interface{} values at first stages which then be replaced with more strong types, several possible races which Mr Klaus Post (https://github.com/klauspost) helped to find, he also noticed at some problems in code style and channel buffer overflow problem. But the greatest part of this migration is that I don't need anymore think "Is library I use non-blocking or not?" as Go's built-in concurrency allows to use all libraries written for language. And also after Tornado it's great to have tools to inspect active goroutines. I also personally don't like how many locks I use at moment - I hope to refactor several parts of code to use channels for synchronization.
2) Elixir is great, but unfortunately I had no experience with it - just read several articles. Centrifugo is not massively scalable at moment - it does not support Redis cluster yet - this is an area for investigating. So now it is limited to Redis throughput and suites well for small or medium projects. So the answer here: Erlang is absolutely great for software like this but I have chosen a language in which I could implement all that needed in a reasonable time. Go has also its unique features so at moment I am pretty happy that I started migration.
It would be great if you write of blog post about your experience in migrating Centrifugo from Python to Go.
Thank you very much for your answers. Like d_luaz I would like to suggest to write a blog post about your ups and downs with the migration journey, would be absolutely interesting! Have a nice jump into the coming week!
I haven't looked into this, but I've noticed a lot of things being migrated to Go recently. I assume they also migrate to a more event driven architecture at the same time. Is that true or have I got the wrong end of the stick?
Go doesn't expose events to the user - the programming interface is one go routine (green thread) per connection.
Internally the go scheduler does lots of clever stuff with events but you as the programmer don't have to worry about it.
go routines are very light weight - you can have 10 or 100 thousand easily so this is a very efficient way of running servers with lots of connections.
It is also very easy to program - it is much easier to reason about one thread per connection than to reason about asynchronous event handling.
Internally the go scheduler does lots of clever stuff with events but you as the programmer don't have to worry about it.
go routines are very light weight - you can have 10 or 100 thousand easily so this is a very efficient way of running servers with lots of connections.
It is also very easy to program - it is much easier to reason about one thread per connection than to reason about asynchronous event handling.
THis looks great, I've been looking for something like this for an existing project.
Question, I notice that if I submit test msgs, it all works and displays in the development project area. However, if I open a new browser tab and navigate to the same project, i dont see the history?
I'm looking to use this for deployment purposes. I can have the window open initially but what if a user decides to join our deployment process a few minutes later? he/she won't see what had happened.
Question, I notice that if I submit test msgs, it all works and displays in the development project area. However, if I open a new browser tab and navigate to the same project, i dont see the history?
I'm looking to use this for deployment purposes. I can have the window open initially but what if a user decides to join our deployment process a few minutes later? he/she won't see what had happened.
Thanks! You are looking at administrative web interface, the goal of this interface is just to show messages coming into channels in real-time just to see that Centrifugo works fine and maybe inspect messages being sent. So when you are reloading a page web interface is just start waiting for new messages from scratch. Web interface must be used as just a tool to monitor your Centrifugo installation and is absolutely optional. It allows to send API commands: publish messages, see presence and history information etc
The goal of Centrifugo is just to deliver messages instantly to all connected clients. It has channel history which can be used to show last N messages from channel but it's not related to what you see in web interface. Also note that Centrifugo is not a persistent storage and channel history messages will expire after configurable timeout. So it's possible to implement what you want using channel history but if you take into account 2 things: message history is limited to N messages and message history will expire after M seconds.
The goal of Centrifugo is just to deliver messages instantly to all connected clients. It has channel history which can be used to show last N messages from channel but it's not related to what you see in web interface. Also note that Centrifugo is not a persistent storage and channel history messages will expire after configurable timeout. So it's possible to implement what you want using channel history but if you take into account 2 things: message history is limited to N messages and message history will expire after M seconds.
I also wrote a super-simple version of this, for talking to Raspis and stuff:
https://github.com/skorokithakis/gweet
https://github.com/skorokithakis/gweet
Hello, thanks for sharing, yes - it looks similar in its core idea, will definitely look at code base and hopefully find something interesting to learn from. One of current Centrifugo problems is that lots of features out of the box result in not so trivial client. So as far as I personally don't know Java and Objective-C/Swift there are no clients for mobile apps yet. Hopefully this will change in future with help of open-source community. Or maybe Go 1.5 mobile apps support will allow to create shared library for using from those languages.
Yeah, client support is always the hard part, there are just so many languages to support... Centrifugo seems feature-packed, though, good job!
1) how was your experience of changing the code from Python to Go, did it feel ok or have you had major headaches?
2) Why didn't you choose Elixir, there are already massively scalable messaging servers in Erlang and it looks like the best choice for a project like this?
Thanks!