Introducing HTTP Load Balancing and SSD Persistent Disk(googlecloudplatform.blogspot.com)
googlecloudplatform.blogspot.com
Introducing HTTP Load Balancing and SSD Persistent Disk
http://googlecloudplatform.blogspot.com/2014/06/introducing-http-load-balancing-and-ssd-persistent-disk-on-google-cloud-platform.html
17 comments
Google, Amazon, and Azure appear to have shaped their clouds after their internal needs first. This has lead to peculiarities in their pricing and feature sets. Amazon, for example, is fine with scaling out to an extreme degree, and so their storage options, particularly their provisioned IOPS options, are rather expensive. Their network and storage infrastructure is likely the oldest and with the lowest turnover rate, so they're the most expensive to get performant storage on.
Google probably has the highest server turnover rate, and my wager is that they recently switched to new SSDs to reduce failure rates in their datacenters. My guess is that their compute servers are now swimming in excess IOPS, and so they are clearly the cheapest if you want very fast storage. This reminds me of Facebook purchasing massive quantities of SSDs for their own datacenters.
Microsoft has very peculiar needs. Azure grew out of servicing internal Microsoft IT needs first, so it is stuck with some quirks. You don't provision IOPS, you create disks, which you attach to VMs. There are no IOPS tunable options for their PaaS, only IaaS VMs. They use software RAID (Linux or Windows) to turn multiple disks into higher speed storage. Their storage backend can only be described as weird[1], because whatever you think the actual disk layout looks like from your VM's perspective, it doesn't look anything like that on the storage layer. They partition data into chunks (disks) which get written to a distributed, log-structured file system on JBODs. Every 1GB extant is made immutable once full, and they use erasure encoding to distribute the data such that they can sustain 2 failures with only 33% disk overhead. The result is that writes are cheap (almost all writes are sequential) and the backend does a bunch of compute work to make reads fast (with caching of metadata and the erasure encoding, I'm guessing).
[1] - http://snia.org/sites/default/files2/SDC2013/presentations/G...
[2] - http://research.microsoft.com/pubs/179583/LRC12-cheng%20webp...
Google probably has the highest server turnover rate, and my wager is that they recently switched to new SSDs to reduce failure rates in their datacenters. My guess is that their compute servers are now swimming in excess IOPS, and so they are clearly the cheapest if you want very fast storage. This reminds me of Facebook purchasing massive quantities of SSDs for their own datacenters.
Microsoft has very peculiar needs. Azure grew out of servicing internal Microsoft IT needs first, so it is stuck with some quirks. You don't provision IOPS, you create disks, which you attach to VMs. There are no IOPS tunable options for their PaaS, only IaaS VMs. They use software RAID (Linux or Windows) to turn multiple disks into higher speed storage. Their storage backend can only be described as weird[1], because whatever you think the actual disk layout looks like from your VM's perspective, it doesn't look anything like that on the storage layer. They partition data into chunks (disks) which get written to a distributed, log-structured file system on JBODs. Every 1GB extant is made immutable once full, and they use erasure encoding to distribute the data such that they can sustain 2 failures with only 33% disk overhead. The result is that writes are cheap (almost all writes are sequential) and the backend does a bunch of compute work to make reads fast (with caching of metadata and the erasure encoding, I'm guessing).
[1] - http://snia.org/sites/default/files2/SDC2013/presentations/G...
[2] - http://research.microsoft.com/pubs/179583/LRC12-cheng%20webp...
Google's SSD price is $0.325/GB/mo with 30 IOPS (flat rate).
AWS's EBS with provisioned IOPS are SSD backed, and cost $0.125/GB/mo, with $0.10 per IOPS.
To get 30 IOPS on AWS would cost $3.125 - almost 10 times as much as at Google. If you're doing more than 2 IOPS, you're better off with GCP.
To get 30 IOPS on AWS would cost $3.125 - almost 10 times as much as at Google. If you're doing more than 2 IOPS, you're better off with GCP.
Not quite a fair comparison because you are showing the price of a 1GB volume. However, if you had a 50 GB volume, then you would be paying $16.25 (50 * 0.325) per month on GCP, and you would be paying $9.25 ((50 * 0.125 + (30 * 0.10)) on Amazon for 30 IOPS.
I hope I have my math right here. Let me know if I missed something. If it's correct, then it seems like the bigger the volume that you have, the more sense it would make to go with Amazon.
Edit: I might have done the math wrong, it looks like Google gives you 30 IOPS per every GB you pay for, so a 50GB volume on GCP would actually be 1500 IOPS. If that's the case, then it's a very good offering from Google since you don't pay for disk access too!
I hope I have my math right here. Let me know if I missed something. If it's correct, then it seems like the bigger the volume that you have, the more sense it would make to go with Amazon.
Edit: I might have done the math wrong, it looks like Google gives you 30 IOPS per every GB you pay for, so a 50GB volume on GCP would actually be 1500 IOPS. If that's the case, then it's a very good offering from Google since you don't pay for disk access too!
Disclaimer: I work for Google as the product manager for Persistent Disk.
I suspect that what crb is referring to is the ratio of 30 IOPS/GB being the top ratio that could be bought and what that would cost on a per GB basis. I don't think crb was suggesting a 1 GB volume.
As for the specific case you describe - looking for 30 IOPS consistently on a 50 GB volume, with Google, there is no need to go to SSD for that or to pay $16.25/month.
- If you meant 30 random reads, you can hit that with 100GB of Standard PD for $4.00/month (with no IO charges)
- If you meant 30 random writes, it's less as 50 GB gets you 75 random write IOPS for $2.00/month (with no IO charges)
At Google, we've tried to bring consistency of performance even to the lower tier of block storage so no matter what level of IO you need, you can count on seeing a consistent level between volumes and over time.
So, how do you choose between the two PD types? As a very coarse rule of thumb:
- If you are looking for 2 random read IOPS / GB or less, growing Standard PD volumes to meet the IOPS needs is the most cost effective route
- If you are looking for 2-30 IOPS/GB, SSD Persistent Disk gets you the better value
I suspect that what crb is referring to is the ratio of 30 IOPS/GB being the top ratio that could be bought and what that would cost on a per GB basis. I don't think crb was suggesting a 1 GB volume.
As for the specific case you describe - looking for 30 IOPS consistently on a 50 GB volume, with Google, there is no need to go to SSD for that or to pay $16.25/month.
- If you meant 30 random reads, you can hit that with 100GB of Standard PD for $4.00/month (with no IO charges)
- If you meant 30 random writes, it's less as 50 GB gets you 75 random write IOPS for $2.00/month (with no IO charges)
At Google, we've tried to bring consistency of performance even to the lower tier of block storage so no matter what level of IO you need, you can count on seeing a consistent level between volumes and over time.
So, how do you choose between the two PD types? As a very coarse rule of thumb:
- If you are looking for 2 random read IOPS / GB or less, growing Standard PD volumes to meet the IOPS needs is the most cost effective route
- If you are looking for 2-30 IOPS/GB, SSD Persistent Disk gets you the better value
(edit: "flat rate" = cost, not total flat rate. The 30 IOPS are per GB, so it scales with volume size. Sorry if I misled.)
> HTTP Load Balancing changes the game for how developers interact with network resources. HTTP load balancing can easily scale to support more than 1 million requests per second with no “warm up.”
> And while other providers count each and every IOPS and charge extra for them, SSD persistent disk includes IOPS in the base price with no extra charges or fees, making cost completely predictable and easy to model.
Between Google's price cuts, and releasing features directly addressed at Amazon AWS feature/pricing deficiencies, Google is coming out swinging.
Google vs Amazon's ELBs:
https://aws.amazon.com/articles/1636185810492479#pre-warming
"Amazon ELB is able to handle the vast majority of use cases for our customers without requiring "pre-warming" (configuring the load balancer to have the appropriate level of capacity based on expected traffic). In certain scenarios, such as when flash traffic is expected, or in the case where a load test cannot be configured to gradually increase traffic, we recommend that you contact us to have your load balancer "pre-warmed". We will then configure the load balancer to have the appropriate level of capacity based on the traffic that you expect. We will need to know the start and end dates of your tests or expected flash traffic, the expected request rate per second and the total size of the typical request/response that you will be testing."
Google vs Amazon charging for IOPS:
"And while other providers count each and every IOPS and charge extra for them, SSD persistent disk includes IOPS in the base price with no extra charges or fees, making cost completely predictable and easy to model."
> And while other providers count each and every IOPS and charge extra for them, SSD persistent disk includes IOPS in the base price with no extra charges or fees, making cost completely predictable and easy to model.
Between Google's price cuts, and releasing features directly addressed at Amazon AWS feature/pricing deficiencies, Google is coming out swinging.
Google vs Amazon's ELBs:
https://aws.amazon.com/articles/1636185810492479#pre-warming
"Amazon ELB is able to handle the vast majority of use cases for our customers without requiring "pre-warming" (configuring the load balancer to have the appropriate level of capacity based on expected traffic). In certain scenarios, such as when flash traffic is expected, or in the case where a load test cannot be configured to gradually increase traffic, we recommend that you contact us to have your load balancer "pre-warmed". We will then configure the load balancer to have the appropriate level of capacity based on the traffic that you expect. We will need to know the start and end dates of your tests or expected flash traffic, the expected request rate per second and the total size of the typical request/response that you will be testing."
Google vs Amazon charging for IOPS:
"And while other providers count each and every IOPS and charge extra for them, SSD persistent disk includes IOPS in the base price with no extra charges or fees, making cost completely predictable and easy to model."
Cloud platforms are exciting to me because it seems every new feature presents an opportunity to break out a spreadsheet to figure out performance/$.
I'm going to consider 3 cases:
CDN: 1TB of storage, 1000 IOPS required. (a cache or "smart"/ACLed CDN)
DB: 20GB of storage, 5000 IOPS required. (a moderately sized database)
Hybrid: 500GB of storage, 2500 IOPS required. (some composite of the two)
A summary of pricing right now for storage IOPS:
Google: pay for the GB, get a fixed IOPS/GB.
Amazon: pay for the GB, pay for the IOPS.
Microsoft: pay for the GB, get 500 IOPS/disk, with a maximum of 16 disks. (but you can't attach that many disks unless you pay for a larger instance size, too)
Microsoft (w/VM): I've included the cost of the compute instance in this summary. I used an A1 VM for CDN, A4 for DB, and A3 for Hybrid, at $35, $277, and $139 monthly, respectively.
CDN
Google: $325 for 1TB/30,000 IOPS.
Amazon: $225 for precisely 1TB/1,000 IOPS.
Microsoft: $50 (using 2 500GB disks)
Microsoft (w/vm): $85 (an A1 VM is required to use two disks)
DB
Google: $54 for 167GB/5,000 IOPS.
Amazon: $502 for 20GB/5,000 IOPS.
Microsoft: $1 for 20GB/5,000 IOPS (using 10 2GB disks)
Microsoft (w/vm): $278 (an A4 VM is required to use 9-16 disks)
Hybrid
Google: $162.50 for 500GB/15,000 IOPS.
Amazon: $312.50 for 500GB/2,500 IOPS.
Microsoft: $25 for 500GB/2500 IOPS (using 5 100 GB disks).
Microsoft (w/vm): $164 (an A3 VM is required to use 5-8 disks)
EDIT: I forgot to add a disclaimer. I am a student at the University of Northern Iowa. I am not affiliated with Amazon or Google. I am a Microsoft Partner, however. I was not paid or asked to write this post, and have never been directly paid by Microsoft to produce any writing, benchmark, or whatever, etc, etc. IANAL, etc. I have received standard gifts from them from attending conferences.
I'm going to consider 3 cases:
CDN: 1TB of storage, 1000 IOPS required. (a cache or "smart"/ACLed CDN)
DB: 20GB of storage, 5000 IOPS required. (a moderately sized database)
Hybrid: 500GB of storage, 2500 IOPS required. (some composite of the two)
A summary of pricing right now for storage IOPS:
(US East)
$/GB $/IOPS CDN DB Hybrid
Google Cloud Platform 0.325 $0.011 $ 325 $ 54 $ 163
Amazon EC2/EBS 0.125 $0.100 $ 225 $ 502 $ 313
Microsoft Azure 0.050 n/a $ 50 $ 1 $ 25
Microsoft Azure (w/vm) 0.050 n/a $ 85 $ 278 $ 164
Google: pay for the GB, get a fixed IOPS/GB.
Amazon: pay for the GB, pay for the IOPS.
Microsoft: pay for the GB, get 500 IOPS/disk, with a maximum of 16 disks. (but you can't attach that many disks unless you pay for a larger instance size, too)
Microsoft (w/VM): I've included the cost of the compute instance in this summary. I used an A1 VM for CDN, A4 for DB, and A3 for Hybrid, at $35, $277, and $139 monthly, respectively.
CDN
Google: $325 for 1TB/30,000 IOPS.
Amazon: $225 for precisely 1TB/1,000 IOPS.
Microsoft: $50 (using 2 500GB disks)
Microsoft (w/vm): $85 (an A1 VM is required to use two disks)
DB
Google: $54 for 167GB/5,000 IOPS.
Amazon: $502 for 20GB/5,000 IOPS.
Microsoft: $1 for 20GB/5,000 IOPS (using 10 2GB disks)
Microsoft (w/vm): $278 (an A4 VM is required to use 9-16 disks)
Hybrid
Google: $162.50 for 500GB/15,000 IOPS.
Amazon: $312.50 for 500GB/2,500 IOPS.
Microsoft: $25 for 500GB/2500 IOPS (using 5 100 GB disks).
Microsoft (w/vm): $164 (an A3 VM is required to use 5-8 disks)
EDIT: I forgot to add a disclaimer. I am a student at the University of Northern Iowa. I am not affiliated with Amazon or Google. I am a Microsoft Partner, however. I was not paid or asked to write this post, and have never been directly paid by Microsoft to produce any writing, benchmark, or whatever, etc, etc. IANAL, etc. I have received standard gifts from them from attending conferences.
What are the advatages over normal servers. You can get (2x120GB intel ssd (75k iops each)/32gb ram/250 Mbit/s garanteed) servers for around 50$/Month from reputable hosters.
It's a mixed bag really.
I prefer cloud based solutions where load is going to be an issue, but only sporadic load. You can easily spin up a new instance, pay for what you use and not be paying for it the rest of the year too. With the load balances available, this can even be made automatic.
It's also good if you want a dev environment which is exactly the same as production. spin up an environment for a day, then spin it down and only pay for usage.
But there are also downsides - you also want at least 2 servers of anything up at any time in case of a datacenter outage or upgrade. But I guess the availability is the upside of that.
I prefer cloud based solutions where load is going to be an issue, but only sporadic load. You can easily spin up a new instance, pay for what you use and not be paying for it the rest of the year too. With the load balances available, this can even be made automatic.
It's also good if you want a dev environment which is exactly the same as production. spin up an environment for a day, then spin it down and only pay for usage.
But there are also downsides - you also want at least 2 servers of anything up at any time in case of a datacenter outage or upgrade. But I guess the availability is the upside of that.
Have a Google spreadsheet link you could share read-only with this?
Here you go, I formatted it up to include formulas and a variety of configurations.
https://docs.google.com/spreadsheets/d/1pw9dKHE_pKSEPqQNrNqz...
https://docs.google.com/spreadsheets/d/1pw9dKHE_pKSEPqQNrNqz...
Thank you so much!
The combination of this HTTP Load Balancing announcement, and May 23rd's announcement of CoreOS support are super exciting. I've been working on setting up a similar docker-based multi-AZ autoscaling setup on EC2, but getting it up and running has required integrating a lot of separately moving parts. If Google can simplify this process via a few simple gcutil commands and a cloud-config YAML file, it would be a hugely compelling offering.
Exciting developments on the load balancing front. Is anyone able to decipher whether this can be used to load balance external (non-google) services? It looks like the input parameter is simply an external IP, but I haven't looked that closely yet.
It's routing traffic to a pool of Compute Engine instances: https://developers.google.com/compute/docs/load-balancing/ht...
Could this be a huge problem for services like Cloudflare. I mean i would trust Google a lot more to be able to survive a huge DDOS.
Softlayer has a similar product called Global IP where traffic enters their extensive network at the local POP. You can then reroute the IP to any of your servers but it’s not load balanced and is really just another IP. Still a good product which we use to power all our services for automated failover. But Google just one-upped them.
Think AWS elastic IP, but Softlayer allow you to route to any server in any DC. And Google allow you to automatically route to a pool of servers in a location closest to the user.