Hetzner EX 4S Unixbench, dd and htparm results(gist.github.com)
gist.github.com
Hetzner EX 4S Unixbench, dd and htparm results
https://gist.github.com/1723876
9 comments
That's hardly a fair comparison. Last time I checked Hetzner doesn't offer auto scaling, "elastic" load balancing, or any of the countless other features that EC2 offers. Actually, forget auto-scaling: what if you want to bring up another bunch of instances manually to deal with a spike in demand?
The EX 4S looks like a very compelling alternative for many use cases but saying that its 9x cheaper is comparing apples to oranges.
The EX 4S looks like a very compelling alternative for many use cases but saying that its 9x cheaper is comparing apples to oranges.
Maybe not 9x, but how about 3x?
You can rent 3 Hetzner servers to deal with 300% load spikes or improve reliability. And you're still 3 times cheaper then EC2.
You can rent 3 Hetzner servers to deal with 300% load spikes or improve reliability. And you're still 3 times cheaper then EC2.
Of course Elastic Scaling is not possible. Because Hosting a server in a datacenter and having cloud instances are two different stories. If you have need for flexibility you can simply run a loadbalancer having your base hardware hosted for a good price at hetzner (or another datacenter you like) and then spin up some AWS Instances behind the loadbalancer for dealing with spikes.
Or as others have written you can buy 3 boxes at hetzner and scale 300% ;)
Solution?
Or as others have written you can buy 3 boxes at hetzner and scale 300% ;)
Solution?
The problem is that you need to have all the three boxes on the same rack wired up to a gigabit switch. Hetzner can do that for you but you can't add or remove servers easily.
I have found that OVH has a virtual rack that can accomplish this. After years with Hetzner I think it is time to move on
I have found that OVH has a virtual rack that can accomplish this. After years with Hetzner I think it is time to move on
Do you have data for the difference in latency &/or bandwidth between servers on the same rack at Hetzner vs. on different racks?
I was assuming that having the servers in different racks would be an advantage, mainly because the reason I have multiple servers is for data redundancy (I'm using Riak and data is replicated across the cluster.) However, in Riak the servers do talk to each other a fair bit.
The latency and bandwidth between a given server and a web browser out in the world shouldn't vary much from rack to rack, I'm assuming.
I was assuming that having the servers in different racks would be an advantage, mainly because the reason I have multiple servers is for data redundancy (I'm using Riak and data is replicated across the cluster.) However, in Riak the servers do talk to each other a fair bit.
The latency and bandwidth between a given server and a web browser out in the world shouldn't vary much from rack to rack, I'm assuming.
Sometimes i had a 5-10% ping loss between servers in the same Hetzner datacenter, so not being on the same rack can be a problem. Especially if you are using nginx -> backend. Then every request is a TCP connect to the backend server, and when SYN packet is lost, you will have a 3 seconds timeout (at least on FreeBSD TCP stack).
Actually it's a really fair comparison, for the price of the AWS service you can buy 9 hetzner instances. So you don't need to spin anything up, just buy it.
A long term contract is the perfect comparison because it's a similar service, you know you'll be using the capacity so you pay up front to have it 'reserved' and in exchange receive reduced prices. Normally with EC2 you'd buy 1 instance and then spin up 8 more on demand, with hetzner you just buy 9 machines and pay the same price you'd pay for 1 AWS machine. When your load spikes you've got 8 extra machines to handle the load. Voila, "auto scaling".
I don't know very many businesses that need to scale beyond 8X capacity for an afternoon. If you really think you need the EC2 API to add and remove machines on demand just install UEC.
It may be comparing apples to oranges but I can tell you that when I need fruit if apples are a $1/lb and oranges are $9/lb I buy apples.
Personally, I think the XS29 @ $299 per month would have been a much better comparison, with 15 drives you could push through some serious IO which would make it suitable for running a database on.
A long term contract is the perfect comparison because it's a similar service, you know you'll be using the capacity so you pay up front to have it 'reserved' and in exchange receive reduced prices. Normally with EC2 you'd buy 1 instance and then spin up 8 more on demand, with hetzner you just buy 9 machines and pay the same price you'd pay for 1 AWS machine. When your load spikes you've got 8 extra machines to handle the load. Voila, "auto scaling".
I don't know very many businesses that need to scale beyond 8X capacity for an afternoon. If you really think you need the EC2 API to add and remove machines on demand just install UEC.
It may be comparing apples to oranges but I can tell you that when I need fruit if apples are a $1/lb and oranges are $9/lb I buy apples.
Personally, I think the XS29 @ $299 per month would have been a much better comparison, with 15 drives you could push through some serious IO which would make it suitable for running a database on.
You and your sibling posters are all focusing on my point about scaling. Fair enough, maybe the fact that it's so much cheaper means that point is moot.
That still doesn't mean that it's a fair comparison. EC2 has plenty of other features out-of-the-box like the ability to manage security groups, VPNs, elastic IPs, easy access to other AWS offerings like SQS and S3, the ability to treat images and the machines they're running on as independent, multiple locations, etc.etc.
Yes you can have all these things outside of the cloud, and yes you might not need any or all of them. (EDIT: and yes using their services means you're locked-in to some degree.) Still, I maintain my original point which is that you can't really say that Hetzner is X times cheaper based purely on the hardware specs, disregarding the fact that EC2 is much more than just a bunch of virtual machines.
That still doesn't mean that it's a fair comparison. EC2 has plenty of other features out-of-the-box like the ability to manage security groups, VPNs, elastic IPs, easy access to other AWS offerings like SQS and S3, the ability to treat images and the machines they're running on as independent, multiple locations, etc.etc.
Yes you can have all these things outside of the cloud, and yes you might not need any or all of them. (EDIT: and yes using their services means you're locked-in to some degree.) Still, I maintain my original point which is that you can't really say that Hetzner is X times cheaper based purely on the hardware specs, disregarding the fact that EC2 is much more than just a bunch of virtual machines.
I'd agree with that, if you need those services it may make fiscal sense if you have few enough servers.
The only problem with your comparison is that you forgot to include the 149 Euro setup fee for the Hetzner server. I don't think this makes a material difference however over three years.
Some would say this isn't fair because you're not including elastic scaling which AWS supports, however you can't elastically scale with a 3 year commitment. Further its hard to estimate how much scaling you'll need because that's a highly variable figure.
But you can say that you will need to keep some number of servers up and running 24/7 for your service.
I think it would be a more perfect comparison to compare AWS's on-demand price to Hetzner with the built in higher performance of the 4S considered as a hot standby for scaling. (I don't know how long it takes Hetzner to provision extras, though I hear its relatively fast.) Since servers are cheap (at Hetzner, at least) it seems reasonable to have 1 or 2 extra spun up if you really have spiky traffic.
Since I'm planning to get a cluster of Hetzner boxes to support a Riak distributed cluster, and each Hetzner box is over provisioned for my needs (if I get a 4S I think they will be way over-provisioned, leaving a lot of headroom for this "elastic scaling" issue)... I think I'll compare the price of the same number of each boxes. (The AWS box has less performance, but I can elastically scale if I need to, and I'll just give the cost of those short term scale-ups to Amazon for free in this example.) The results may not be a perfect example for whatever others are considering, but it is a reasonable comparison for my purposes. (my service is a big data service, involving a lot of map reduce, requiring a cluster for operational simplicity and reliability as well as the ability to scale to millions of customers. I don't know if we'll get millions of customers, but one of our closest competitors had millions of customers within a few weeks.)
Amortizing the 4@ m2.4xlarge = 4* $600/month = $2,400 (using the $600 for the reserved instance costs from vladd's example.)
4@ Hetzner 4S = $149 Euro setup fee, amortized over 3 years = 4.14 Euro/month Add the 49.58 Euros in hosting = 53.71 euros x 4 servers = 214.87 euros or $282.50 a month.
So, for my cluster, AWS is 8.5 times as expensive.
I think that $282 per month is a lot easier for a scrappy startup to handle than $2,400 as well. This brings "big data" within the realm of undefended mortals.
Many people use AWS for all the extra services that AWS offers. The value of these services is, of course, subjective, as some startups will need them and others won't. But they all are based on a custom Amazon API which means if you build your architecture across many servers at Amazon you've got a fair amount of lock-in.
For me, I'll take the 8.5X reduction in costs, over lock in anyway. If Hetzner starts performing poorly, I can migrate without too much difficulty, and without having to re-architect my service.
Some would say this isn't fair because you're not including elastic scaling which AWS supports, however you can't elastically scale with a 3 year commitment. Further its hard to estimate how much scaling you'll need because that's a highly variable figure.
But you can say that you will need to keep some number of servers up and running 24/7 for your service.
I think it would be a more perfect comparison to compare AWS's on-demand price to Hetzner with the built in higher performance of the 4S considered as a hot standby for scaling. (I don't know how long it takes Hetzner to provision extras, though I hear its relatively fast.) Since servers are cheap (at Hetzner, at least) it seems reasonable to have 1 or 2 extra spun up if you really have spiky traffic.
Since I'm planning to get a cluster of Hetzner boxes to support a Riak distributed cluster, and each Hetzner box is over provisioned for my needs (if I get a 4S I think they will be way over-provisioned, leaving a lot of headroom for this "elastic scaling" issue)... I think I'll compare the price of the same number of each boxes. (The AWS box has less performance, but I can elastically scale if I need to, and I'll just give the cost of those short term scale-ups to Amazon for free in this example.) The results may not be a perfect example for whatever others are considering, but it is a reasonable comparison for my purposes. (my service is a big data service, involving a lot of map reduce, requiring a cluster for operational simplicity and reliability as well as the ability to scale to millions of customers. I don't know if we'll get millions of customers, but one of our closest competitors had millions of customers within a few weeks.)
Amortizing the 4@ m2.4xlarge = 4* $600/month = $2,400 (using the $600 for the reserved instance costs from vladd's example.)
4@ Hetzner 4S = $149 Euro setup fee, amortized over 3 years = 4.14 Euro/month Add the 49.58 Euros in hosting = 53.71 euros x 4 servers = 214.87 euros or $282.50 a month.
So, for my cluster, AWS is 8.5 times as expensive.
I think that $282 per month is a lot easier for a scrappy startup to handle than $2,400 as well. This brings "big data" within the realm of undefended mortals.
Many people use AWS for all the extra services that AWS offers. The value of these services is, of course, subjective, as some startups will need them and others won't. But they all are based on a custom Amazon API which means if you build your architecture across many servers at Amazon you've got a fair amount of lock-in.
For me, I'll take the 8.5X reduction in costs, over lock in anyway. If Hetzner starts performing poorly, I can migrate without too much difficulty, and without having to re-architect my service.
[deleted]
Apples / Oranges.
Quite apart from the ability to buy by the hour, the Hetzner offering really isn't suitable for processing any data you care about - it's not using ECC RAM, and the processor used doesn't support it.
Quite apart from the ability to buy by the hour, the Hetzner offering really isn't suitable for processing any data you care about - it's not using ECC RAM, and the processor used doesn't support it.
Why is this being downvoted? The processor used (i-7 2600) does not support ECC RAM, and this platform does not include it. A server platform without ECC RAM is a pretty big disadvantage for several reasons, to most compelling being the high real-world rate of memory errors in real systems. Memory errors in non-ECC systems can be extremely difficult and expensive to track down, and can cause data corruption or loss if the machine is used in the data pipeline.
In one study (http://www.cs.toronto.edu/%7Ebianca/papers/sigmetrics09.pdf), strong correlations were found between age, usage and error rate. Before using a server without ECC RAM, it's worth doing a quick estimate of the amount of data you intend to move around, and whether it's worth it to you (and, more importantly, your customers) to save a little money going with a non-ECC platform.
In one study (http://www.cs.toronto.edu/%7Ebianca/papers/sigmetrics09.pdf), strong correlations were found between age, usage and error rate. Before using a server without ECC RAM, it's worth doing a quick estimate of the amount of data you intend to move around, and whether it's worth it to you (and, more importantly, your customers) to save a little money going with a non-ECC platform.
What would be the downside to using non ECC RAM in a heavily loaded server? Would it cause processes to crash?
If you are lucky, processes would crash. More likely there would be silent bits of data corruption.
Gotcha, so it would just depend on what bits in RAM were corrupted (data versus process instructions)?
I am curious about a couple of specific use cases that I can think of where this might affect me (and, most likely, would be common points of failure for others):
1. data in the MySQL tables (stored in memory) is corrupted. Would mysql crash? Indicate table corruption, and I could just reload it from disk? Write corrupted data to disk, and permanently trash my data collection?
2. large process that's running data analysis (say a big python process with tons of data in RAM). Would one of my variables (say an int with value 4) turn into another number? Would it become unreadable?
I appreciate the effort to explain this. I know, in theory, why ECC RAM is useful, but I have difficulty visualizing real world scenarios.
I am curious about a couple of specific use cases that I can think of where this might affect me (and, most likely, would be common points of failure for others):
1. data in the MySQL tables (stored in memory) is corrupted. Would mysql crash? Indicate table corruption, and I could just reload it from disk? Write corrupted data to disk, and permanently trash my data collection?
2. large process that's running data analysis (say a big python process with tons of data in RAM). Would one of my variables (say an int with value 4) turn into another number? Would it become unreadable?
I appreciate the effort to explain this. I know, in theory, why ECC RAM is useful, but I have difficulty visualizing real world scenarios.
If it was a C int then it would just change to another value. If it was a Python int then two things can happen: either the bit flip was in the value which causes the value to change, OR the bit flip was in the tag bits which causes Python to interpret the data as something else than an int. The latter would most likely cause your program to crash.
With MySQL any of those things you can happen. If you're lucky then only the cache is corrupted and then you can just reload from disk. If you're unlucky then the data got corrupted on its way to disk and the wrong data will be written to disk. If you are astronomically unlucky then the in memory machine code of MySQL got changed in such a way that it starts overwriting your entire disk with garbage. You should probably be more afraid of meteorites though. And of bugs in either your own or others' code.
ECC RAM reduces the probability of such a bit flip happening. That doesn't mean that they are eliminated entirely. So you have to do these two things in any case:
1. Bit flips can cause processes to misbehave/crash. So you want to have a way to detect and restart misbehaving/crashed processes.
2. Even with ECC RAM you want to do your own error correction for critical data (say a bank transaction log).
Here is an interesting paper that discusses the prevalence of DRAM errors and the effectiveness of ECC RAM:
DRAM Errors in the Wild: A Large-Scale Field Study -- http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf
It would be interesting if somebody did an experiment where they artificially flipped bits of various software's memory to see what happens. I'd expect that in many cases it doesn't do any harm at all.
With MySQL any of those things you can happen. If you're lucky then only the cache is corrupted and then you can just reload from disk. If you're unlucky then the data got corrupted on its way to disk and the wrong data will be written to disk. If you are astronomically unlucky then the in memory machine code of MySQL got changed in such a way that it starts overwriting your entire disk with garbage. You should probably be more afraid of meteorites though. And of bugs in either your own or others' code.
ECC RAM reduces the probability of such a bit flip happening. That doesn't mean that they are eliminated entirely. So you have to do these two things in any case:
1. Bit flips can cause processes to misbehave/crash. So you want to have a way to detect and restart misbehaving/crashed processes.
2. Even with ECC RAM you want to do your own error correction for critical data (say a bank transaction log).
Here is an interesting paper that discusses the prevalence of DRAM errors and the effectiveness of ECC RAM:
DRAM Errors in the Wild: A Large-Scale Field Study -- http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf
It would be interesting if somebody did an experiment where they artificially flipped bits of various software's memory to see what happens. I'd expect that in many cases it doesn't do any harm at all.
I suggest looking into studies of radiation effects upon computer systems. They do a lot of bit-flipping. I was privy to results from a confidential study once, and as one might expect, enough bit flips cause big problems (the study went into more details than that, of course).
The answer to most of your questions is 'maybe', unfortunately. Reasoning about the things that could happen when memory errors occur is very difficult, because they occur outside the mental model of computation that most programmers (and systems administrators) use.
Let's use MySQL as an example. A bit flip in the memory which holds the code may cause it to crash. A bit flip in the 'metadata' could cause the table to become corrupted, potentially recoverably. A bit flip in the data itself could turn 'Travis' into 'Trbvis', which might go undetected depending on where it happened and which storage engine you are using.
The use of memory for OS page caching (less so in databases, which often use O_DIRECT and more so in other programs) means that arbitrary corruption could happen to pieces of disk data your program didn't even touch, if you touch data near them.
Let's use MySQL as an example. A bit flip in the memory which holds the code may cause it to crash. A bit flip in the 'metadata' could cause the table to become corrupted, potentially recoverably. A bit flip in the data itself could turn 'Travis' into 'Trbvis', which might go undetected depending on where it happened and which storage engine you are using.
The use of memory for OS page caching (less so in databases, which often use O_DIRECT and more so in other programs) means that arbitrary corruption could happen to pieces of disk data your program didn't even touch, if you touch data near them.
As someone who runs physical servers for hosting, you really get what you pay for.
I've had non-ECC RAM systems destroy database tables leading to data loss.
Systems with ECC are either able to correct the error (and log it), or throw the alarm bells. Even discounting RAM bit flips, simple bad RAM can destroy your data. Having an ECC aware system (including getting Linux to check the EDAC or have the baseband monitor do so) has saved me many times from failing hardware.
I've had non-ECC RAM systems destroy database tables leading to data loss.
Systems with ECC are either able to correct the error (and log it), or throw the alarm bells. Even discounting RAM bit flips, simple bad RAM can destroy your data. Having an ECC aware system (including getting Linux to check the EDAC or have the baseband monitor do so) has saved me many times from failing hardware.
If you're using an AWS m2.4xlarge (score: 1511) then you pay $2.00 per hour. If you're using this Hetzner (score: 1729) you pay less than $0.09 per hour. You'd have to use very few hours indeed for AWS to be cheaper. That's not even factoring in traffic. You get 10TB per month with Hetzner which would cost you an additional $1200/month (!) with Amazon. AFAICT Amazon doesn't use ECC ram either; at least I don't see it mentioned anywhere. Hetzner does have competitively priced servers with ECC ram.
you could use the ex6:
http://www.hetzner.de/en/hosting/produkte_rootserver/ex6
a little more expensive (69 EUR) and less RAM (16 GB) but using ECC.
http://www.hetzner.de/en/hosting/produkte_rootserver/ex6
a little more expensive (69 EUR) and less RAM (16 GB) but using ECC.
or someone like providerservice.com who have almost the same specs but with no setup fee.
[deleted]
You can get the one with ECC (EX 6) it's just 10euro extra / per month... which is still away cheeper than AWS.
TLDR; the €59/mo EX 4S scores higher (1729) than AWS m2.4xlarge (1511).
That number for AWS is based on running the benchmark in May 2010 [1].
[1] http://blog.cloudharmony.com/2010/05/what-is-ecu-cpu-benchma...
[1] http://blog.cloudharmony.com/2010/05/what-is-ecu-cpu-benchma...
I'm not sure AWS have updated their specs since then (all the specs for an ECU refer to a 1Ghz 2007 Xeon). We also tested our staging c1.xlarge and it scored less than the cloudharmony results indicated a c1.xlarge should.
You might also want to check out some benchmarks I ran a few weeks ago. It compares many types of EC2 instances: http://blog.carlmercier.com/2012/01/05/ec2-is-basically-one-...
Why is this company getting so much attention as of late? I've never heard of them until recently, and considering how crowded the whole VPS/dedicated server market is, what makes them stand out?
They are well known in Germany, they've been one of the biggest, if not the biggest, provider of VPS and colo services over here since the mid 90s.
The recent buzz is the first time I see them mentioned on US-centric sites though, if memory serves.
As for your question, it appears that many people find the price/performance ratio of the EX 4S compelling.
The recent buzz is the first time I see them mentioned on US-centric sites though, if memory serves.
As for your question, it appears that many people find the price/performance ratio of the EX 4S compelling.
I've used them for a few years now. I think they used to be very much German-language focused, but over the past couple of years they have become more and more international. They give good support and the servers I have with them have been rock solid
Because their prices are amazing.
This server is does not come with ECC-memory. How "bad" would it be to run a database server on a non ecc server?
As said other places, they have ECC servers to: http://www.hetzner.de/en/hosting/produkte_rootserver/ex6
I never meant to criticise, I saw that they have ECC-servers as well, I just wondered what the implications would be if running a database on a non ecc server, as I've never really understood how much damage it could cause.
Short answer: ECC will save you from having corrupt data written to your database if one of the DIMMs are corrupt.
Or more importantly, if cosmic rays hit your data in the wrong way.
For those that are curious, the motherboard in these systems is a ASUSTeK P8H67-M PRO. Seems like it's a gaming board - understandable given it's an i7 platform.
Does anyone offer a similar product / price point in the US? I haven't found anything remotely close. Why not?
An Amazon m2.4xlarge instance (High-Memory Quadruple Extra Large Reserved Instance) with Heavy Utilization on a 3-year commitment costs $9'660 for the 3y term and 3 * 365 * 24 * $0.454 = $11'931 in usage fees, which means a $21'591 total, $7'197 per year or $600 per month (EUR 458 per month at the current exchange rate).
458 EUR versus 49-60 EUR is 9x more expensive at full-time usage. (instance type and price data extracted from http://aws.amazon.com/ec2/instance-types/ and http://aws.amazon.com/ec2/pricing/ ).
If you were to consider just the Amazon Heavy Utilization term commitment for 3 years, that's still $9'660 / 3 / 12 = $268 (EUR 205) each month.