High Availability Principle : Concurrency Control(saasinterrupted.com)
saasinterrupted.com
High Availability Principle : Concurrency Control
http://saasinterrupted.com/2010/02/05/high-availability-principle-concurrency-control/
2 comments
It would largely depend on how gracefully you degrade your service to those other 200 users. Take any game server with the concept of queues. They could (depending on the game, I know this isn't universal) take in more players than they allow at a time, but the quality would drop significantly. Instead, they give you a number, and say "You are number X in line. Please wait". While it isn't great, and is very frustrating for players, it is a decent solution.
On the other hand, you have Monopoly Street View, which launched with a massively higher number of players than they expected. The end result was a game that was playable for a very, very small group of people, and no one else even knew what was going on.
In general, you need to be able to balance the two. There should be a point where you have an absolute cutoff, no more users/requests beyond that, but you should also have a sliding scale. Of course a server only processing one request at a time will be faster, but you need to be able to degrade somewhat, while still providing good service.
On the other hand, you have Monopoly Street View, which launched with a massively higher number of players than they expected. The end result was a game that was playable for a very, very small group of people, and no one else even knew what was going on.
In general, you need to be able to balance the two. There should be a point where you have an absolute cutoff, no more users/requests beyond that, but you should also have a sliding scale. Of course a server only processing one request at a time will be faster, but you need to be able to degrade somewhat, while still providing good service.
The challenge is that by the time you reach the 300th request your system is unusable. In the example mentioned the system starts to experience exponential degradation around the 100th request. One wants to avoid reaching that state of the system where the exponential degradation starts .
There IS a better solution to this than just rejecting the request and it has to do with request queuing.
Pardon my ignorance in this matter, but hasn't commoditized, scalable hosting (such as Amazon EC2) largely mitigated the issue of service degradation?
Only if every part of your system is perfectly horizontally scalable. There are very frequently some singleton aspects of a system, perhaps the most common example being a single central data store with a fan out of web servers. The web servers scale by adding more, but it's an architectural change to add a second DB.
Exactly. There is always a "deployed system limit" even in horizontally scaling DB layers as deployed with sharding etc.
What about the 300th request? I guess it's a matter of opinion, but is it better to provide a mediocre (or worse) quality experience to all 300 users, or to provide a good quality experience to 1/3 of the users and deny the other 2/3 requests?