Dispelling Backblaze's HDD Reliability Myth(tweaktown.com)
tweaktown.com
Dispelling Backblaze's HDD Reliability Myth
http://www.tweaktown.com/articles/6028/dispelling-backblaze-s-hdd-reliability-myth-the-real-story-covered/index.html
12 comments
I find it particularly galling that "Everybody knows temperature is important" is the rebuttal to a bunch of data saying "we didn't see an effect".
If you've got data, post it. If not, it starts to sound like astroturf.
Fwiw, with my small sample size, I agree with backblaze. Seagates appear to be failing more, and avoid wdc greens.
If you've got data, post it. If not, it starts to sound like astroturf.
Fwiw, with my small sample size, I agree with backblaze. Seagates appear to be failing more, and avoid wdc greens.
> and avoid wdc greens
Anecdotally, at my last job, we had used them on several of our internal file and VM servers (because they were cheap), and they had a nasty habit of falling off the bus overnight, causing the RAID controller to go berserk.
I think they had issues with early death from vibration, and they don't perform well in anything like demanding performance situations.
Though that may just be my faulty memory of he The original piece.
Though that may just be my faulty memory of he The original piece.
I thought the point about which version chassis the drives in to be a valid point.
I believe the backblaze design has 1 pound drive vibrating at 120hz or so supported by a SATA connector. I don't know the SATA specification, but I suspect they aren't designed to support the drives weight. If SATA connectors where designed to support the drive they wouldn't be plastic.
Additionally the SATA Multiplier boards (thin fiberglass with delicate traces) are supporting the weight of 5 drives each.
This would also explain why the enterprise drives (typically heavier, more platters, and with a more conservative design) are less reliable then the consumer (with typically higher density platters).
I believe the backblaze design has 1 pound drive vibrating at 120hz or so supported by a SATA connector. I don't know the SATA specification, but I suspect they aren't designed to support the drives weight. If SATA connectors where designed to support the drive they wouldn't be plastic.
Additionally the SATA Multiplier boards (thin fiberglass with delicate traces) are supporting the weight of 5 drives each.
This would also explain why the enterprise drives (typically heavier, more platters, and with a more conservative design) are less reliable then the consumer (with typically higher density platters).
I love how he keeps coming back to how the drives were sourced through retail channels - wtf does he think consumers get them from? I frankly don't care if one brand lasts longer when handled with white gloves, because I'm not going to be paying someone to pretend to do that for me.
Also, insisting that Backblaze has some ridiculously out-of-this-world "workload"? We're not talking a database, but an operation that is usually done with tapes. According to him, consumers must only need 4TB drives with 1000 hour MTBFs and 20TB of lifetime IOs.
Also, insisting that Backblaze has some ridiculously out-of-this-world "workload"? We're not talking a database, but an operation that is usually done with tapes. According to him, consumers must only need 4TB drives with 1000 hour MTBFs and 20TB of lifetime IOs.
>"Backblaze claims that drive temperature doesn't affect drive life. That is counter to the observations of many others, including drive manufacturers."
Google has a whitepaper[0] on a large deployed sample size of drives which disagree with the author's claims here. I agree that more data from backblaze would be nice, but their claims do seem to have more plausibility than the author suggests. (If he has methodology issues with the Google paper, that would be more interesting)
[0]- http://static.googleusercontent.com/media/research.google.co... (pdf warning)
Google has a whitepaper[0] on a large deployed sample size of drives which disagree with the author's claims here. I agree that more data from backblaze would be nice, but their claims do seem to have more plausibility than the author suggests. (If he has methodology issues with the Google paper, that would be more interesting)
[0]- http://static.googleusercontent.com/media/research.google.co... (pdf warning)
There's a typo in that link.
http://static.googleusercontent.com/media/research.google.co...
I'm more inclined to believe Google's results than anybody elses, the amount of data they have is staggering and the results they got are fairly clear cut. I personally don't bother cooling spinning disks, it just seems like a complete waste of time and effort based on that paper.
http://static.googleusercontent.com/media/research.google.co...
I'm more inclined to believe Google's results than anybody elses, the amount of data they have is staggering and the results they got are fairly clear cut. I personally don't bother cooling spinning disks, it just seems like a complete waste of time and effort based on that paper.
"With varying numbers of drives for each model, it is possible that some bad batches may have made their way into the sample pool, thus further skewing the numbers."
So, to measure drive reliability, the author recommends that you don't count unreliable drives, since it may "skew the numbers"?
So, to measure drive reliability, the author recommends that you don't count unreliable drives, since it may "skew the numbers"?
This Tweaktown article offers nothing in the way of factual evidence to discount any of the hard work Backblaze put into that blog post.
Shame on Tweaktown for poo poo'ing on Backblaze's effort.
Shame on Tweaktown for poo poo'ing on Backblaze's effort.
I'm curious why the author believes that beating the ever loving crap out of drives in harsh conditions and seeing which ones come out, overall, on top creates a myth. Perhaps the Backblaze article is better headlined "which hard drives seem to hold up well under pressure." Consumer environments usually have crappy, unreliable power (power strip chained to power strip plugged into an outlet from the 1960s), random vibrations (especially with kids), and poor air flow conditions (who here has done Family IT and had to carve the dust out of the case stuck behind a non-moving fan?). The only thing a consumer environment lacks from Backblaze's comparison is 24/7 read/write operations.
I would read an article about how bullet-resistant different models of car are, but I wouldn't completely change my purchasing decisions, as I avoid bullets. (Whoa, I didn't even mean to use a car analogy.)
"Consumer environments usually have crappy, unreliable power ..."
I would hope that Backblaze is on reliable, stable, data center power -- though knowing how cost-conscious they are, perhaps not -- so that's one aspect where they wouldn't be able to provide good information.
"Consumer environments usually have crappy, unreliable power ..."
I would hope that Backblaze is on reliable, stable, data center power -- though knowing how cost-conscious they are, perhaps not -- so that's one aspect where they wouldn't be able to provide good information.
You got it backwards. You live in Kabul, so the article on bullet resistance is completely relevant. The point is that Backblaze's deployment conditions are way more similar to a domestic user's than other enterprise deployments.
More similar, yes, but maybe not similar enough. I wonder if it's the pathological vibration killing the Seagates. If so, you don't want to use Seagates in a file server sitting on top of your washing machine, but maybe they're fine for your living room media box.
Edit: Unlike Backblaze, I have no hard numbers, and can only pull suppositions out of my ass.
Edit: Unlike Backblaze, I have no hard numbers, and can only pull suppositions out of my ass.
It's not even clear to me that Backblaze is accessing the drives that much. I would imagine most of their data is written once and never read or deleted.
This article is astonishing. I try hard to give the benefit of the doubt to authors (as opposed to salespeople, speechwriters, and advertisers), and assume that they have good intentions and believe what they are saying. But I can't find any explanation for this article other than paid hit-piece or personal vendetta.
Here's the article's conclusion:
The data from Backblaze should not influence a purchasing decision by any consumer, regardless of what type of drive they are purchasing. The innumerable variables, and lack of documentation, ensures the results are unreliable. Even for the winners, the results aren't good; the failure rates are exponentially higher than those observed in the real-world. One should question whether these companies could survive financially with the massive warranty return rates in real-world scenarios.
And here is some of the logic:
It is the release of the data, in handy charts and graphs that encourage misrepresentation, which brings out the data-storage stickler in me. HDD manufacturers spend billions of dollars in R&D, and their labs are designed to characterize and measure the reliability and endurance of their storage solutions.
This last line converted from to "undecided" to "paid propaganda", as I doubt that vendettas come with such clearly presented "suggested talking points". It's worth reading if just to see such a stunning example of the art form.
Here's the article's conclusion:
The data from Backblaze should not influence a purchasing decision by any consumer, regardless of what type of drive they are purchasing. The innumerable variables, and lack of documentation, ensures the results are unreliable. Even for the winners, the results aren't good; the failure rates are exponentially higher than those observed in the real-world. One should question whether these companies could survive financially with the massive warranty return rates in real-world scenarios.
And here is some of the logic:
It is the release of the data, in handy charts and graphs that encourage misrepresentation, which brings out the data-storage stickler in me. HDD manufacturers spend billions of dollars in R&D, and their labs are designed to characterize and measure the reliability and endurance of their storage solutions.
This last line converted from to "undecided" to "paid propaganda", as I doubt that vendettas come with such clearly presented "suggested talking points". It's worth reading if just to see such a stunning example of the art form.
A third option other than hit-piece or vendetta would be just page views.
A lot of tech writers and bloggers just write counter pieces to popular articles in the hope to get read, spread and shared by whoever dislikes or disagrees with the original article. It's become very common to see articles like "Why [insert yesterdays top HN post] is wrong/will never work/is flawed/..." on HN and reddit.
A lot of tech writers and bloggers just write counter pieces to popular articles in the hope to get read, spread and shared by whoever dislikes or disagrees with the original article. It's become very common to see articles like "Why [insert yesterdays top HN post] is wrong/will never work/is flawed/..." on HN and reddit.
I'm willing to believe that page views are his and his employer's intent, and I'm willing to believe it would be titled as it is, the level of vitriol in the content makes me think there must be something more.
Data-storage stickler, indeed. He has been in the storage industry for barely over a year, with no achievements of note. Before that he was a restaurant manager. He's not just making an appeal to authority, but an appeal to risible authority. Whatever the flaws in Backblaze's data might be, it's still data and that wins over completely empty blathering.
The author of this article (Paul) appears to ignore the law of large numbers - while Paul's arguments against using the figures from Backblaze as measures of drive reliability would hold if Backblaze's analysis was based on a small sample size, because their sample size is relatively large, Paul's criticisms don't hold. Paul's argument is that any analysis of hard-drive reliability should control for factors like vibration and temperature as these vary in all environments - but in fact, because Backblaze has taken large (enough) sample of drives, these variations are already controlled for (ie cancelled out). Sure the consumer is probably never going to experience a failure irrespective of whether the buy a seagate or WD drive - but that's not the point - the point is that the long tail of the distribution, WD and Fujitsu beat Seagate for reliability. That is still valuable information for people, even if they aren't going to be using their drives in the same type of environment as Backblaze.
Of course it isn't a large scale scientific experiment with controls in place for all the variables they are running a business and the data is a by-product (albeit one which they can use to optimise their own business).
But it is the largest scale testing I'm aware of that publishes the manufacturer names. For that alone it is valuable. I also like the fact it is harsher than normal environment, I don't want the most reliable disk if it never gets touched or used or to have to baby it.
But it is the largest scale testing I'm aware of that publishes the manufacturer names. For that alone it is valuable. I also like the fact it is harsher than normal environment, I don't want the most reliable disk if it never gets touched or used or to have to baby it.
That article was just like reading the Backblaze post, if the Backblaze post had been split into three pages and covered in ads.
[deleted]
Backblaze already acknowledged that their usage environment doesn't reflect that of normal consumer, yet the author harps on the point.
The author tries to debunk Backblaze's claim that it found no correlation between drive temperature and failure rate. Yet he presents no evidence that Backblaze's methodolgy was wrong.
Backblaze had large enough sample to have even distribution of drives in various temperature, rack, and load environment. I doubt they put all Seagate drives in the middle shelf of Ver. 1 racks where it was the hottest, but put all Hitachi drives in the bottom of the Ver. 3 racks where it was cooler and more stable.
Unless there is another study that shows failure rate of these drives in normal consumer environment, Seagate's 3-5x failure rate should give people a pause.