Load Balanced Actors with Akka and Scala(jimplush.com)
jimplush.com
Load Balanced Actors with Akka and Scala
http://jimplush.com/blog/article/183/Load-Balanced-Actors-with-Akka-and-Scala
3 comments
Nice example - but it might give the wrong idea to the unfamiliar. Local worker actors like these would typically be thread-pool backed and then the load balancing is free (need not be explicitly coded). Remote workers do of course need to be load balanced.
nice and simple example for something that confounds people like myself. Thanks for putting it together.
http://akka.io/docs/akka/1.1/intro/getting-started-first-sca...
That page also has some key features including the use of PoisonPills and stop hooks to terminate all threads once the tasks are finished and sending messages betweens workers and the master actor that are missing in the OP.
Also, web crawling is an I/O bound task - so why limit the number of worker actors to the number of cores? That approach makes sense only for CPU bound tasks. The approach in the tutorial to accept number of workers as a parameter is better practice imo, so that you can tinker with it to maximise throughput for your particular problem.