Running an Akka cluster in an EC2 Autoscaling Group(chrisloy.net)

23 pointsby chrisloyvor 12 Jahren13 comments

Running an Akka cluster in an EC2 Autoscaling Group

http://chrisloy.net/2014/05/11/akka-cluster-ec2-autoscaling.html

3 comments

saryantvor 12 Jahren

Excellent post. We made the mistake of trying to use Chef to deal with seed node discovery in our Akka cluster and that was a terrible idea. Really wish we'd gone with the AWS SDK instead.

chrisloyvor 12 Jahren

Thanks! The SDK has some "quirks" (and isn't especially Scala-friendly) but I reckon it's ultimately pretty powerful. My main gripe is the lack of real-world examples in the docs - hence my desire to write this up.

gatehousevor 12 Jahren

Hi, I didn't see an obvious way to reach you on the blog bit just FYI, it looks weird on iPhone Safari

Edit http://imgur.com/oX0CzuZ http://imgur.com/WsqEVxA

chrisloyvor 12 Jahren

Hi, thanks for the feedback! Yeah I'm aware the look on Chrome on my Android is pretty terrible, which is on my list to get sorted. Looks like it's even worse on iOS! At the moment its basically using the default site template from Jekyll which is apparently very much not mobile first!

Also re: contact, there's a link to my Twitter account but not much more. Cheers for letting me know anyway!

Edit: Grokked media queries; hopefully a little nicer on mobile now.

saryantvor 12 Jahren

One question: how do you deal with crashed JVMs? You can't rejoin a new JVM if it has the same address and port but I can't think of a good way to communicate the new port using the AWS SDK.

chrisloyvor 12 Jahren

The way we have it set up, if the JVM crashes then it will mean that the health check url used by the load balancer will become unresponsive. This would lead to the auto scaling group downing the instance and replacing it with a new one. So, the same node wouldn't ever need to rejoin.

zackangelovor 12 Jahren

Were you using Chef inside of OpsWorks? I'm curious about what problems you're having.

I'd think that hooking the configure lifecycle event on your cluster layer in OpsWorks would work just fine.

saryantvor 12 Jahren

No, just Chef Solo. We only had four nodes at the time.

xwowsersxvor 12 Jahren

Excellent post! How do you ensure that you don't split the cluster - do you have to configure the ASG to add the 2nd node only when the first one is already up? If so, how exactly do you do that?

noelwelshvor 12 Jahren

Nice post. Would love to more about your application of Akka Cluster -- it's far and away the most interesting part of Akka to me, and something we have plans to build some very interesting systems on top of.

chrisloyvor 12 Jahren

I work for a large organisation and am not really in a position to go into details about what we'd be using it for at the moment, sorry.

Suffice it to say that we're pretty early on in the project and so far we're using Akka Cluster for sharing transient state between nodes, without the need for any kind of persistence store being used. My post was in response to the pain I had in getting over the initial hurdle of getting a prototype running.

chaostheoryvor 12 Jahren

Does Akka cluster do load balancing between nodes yet?

saryantvor 12 Jahren

You should turn this post into an Activator template.