2014 MacBook Air: Performance benchmarks(macworld.com)
macworld.com
2014 MacBook Air: Performance benchmarks
http://www.macworld.com/article/2150841/lab-tested-new-2014-macbook-air-benchmarks.html
69 comments
Really, really sloppy benchmarking work here -- only four machines were tested, and every one of them had a drive from a different manufacturer. They weren't even all the same size! It's already well-known that Apple sources SSDs (as well as other parts) from a variety of suppliers, and not all of them perform identically. Concluding that the 2014 model is "slower" than the 2013 models from a sample this small is totally inappropriate, though.
Apple are sourcing parts from several component vendors, with varying performance, and name the resulting product exactly the same. They don't even specify what variants exist. How is somebody doing benchmarks supposed to test all possible configurations if they don't know how many exist, and have no way of determining which configuration they are buying?
Exactly. You could argue that if Apple is happy with this situation where some of its customers get worse products than others, then this is absolutely worth punishing Apple for. And a good way to do this is by publishing benchmark reviews with bad values for the new thing.
They all meet a baseline standard, it's just some people get machines that are above those baselines. Yes, sometimes the "good" machine from a previous generation will outperform the baseline from the next generation in a given benchmark, but that's an inevitability of having machines that outperform the baselines of their generation.
This really depends on where you set the baseline standard.
You may see the worst-performing variants of a given model as defining the baseline, with some being above the baseline.
But it's perfectly legitimate to define the baseline at the best-performing variants of a given model, with some unfortunately being below the baseline.
Personally, I prefer to set expectations high, and define the baseline based on the best that has been achieved so far. The worst-performing variant from 2014 should still be expected to exceed, or at the very worst be equal to, the best-performing variant from 2013.
You may see the worst-performing variants of a given model as defining the baseline, with some being above the baseline.
But it's perfectly legitimate to define the baseline at the best-performing variants of a given model, with some unfortunately being below the baseline.
Personally, I prefer to set expectations high, and define the baseline based on the best that has been achieved so far. The worst-performing variant from 2014 should still be expected to exceed, or at the very worst be equal to, the best-performing variant from 2013.
>Concluding that the 2014 model is "slower" than the 2013 models from a sample this small is totally inappropriate, though.
You only need one sample to make the point that it may be slower, and that's very much contrary to expectations.
You only need one sample to make the point that it may be slower, and that's very much contrary to expectations.
He already explained that the expectations are wrong to begin with.
2013 models had drives from different manufactures and some of those were slower than others.
Here's an example: Say 2013-A-SSD > 2013-B-SSD and we have the same two A and B SSDs in 2014 machines.
Then 2013-A-SSD > 2014-B-SSD.
With just one machine (and without knowing the specifics) you can deduce from the above that "2014 drives are slower than 2013s" when in fact they are the exact same 2 models that 2013 Air's had.
2013 models had drives from different manufactures and some of those were slower than others.
Here's an example: Say 2013-A-SSD > 2013-B-SSD and we have the same two A and B SSDs in 2014 machines.
Then 2013-A-SSD > 2014-B-SSD.
With just one machine (and without knowing the specifics) you can deduce from the above that "2014 drives are slower than 2013s" when in fact they are the exact same 2 models that 2013 Air's had.
Which still means that, if you buy a 2014 model, you might get a machine that's slower than the 2013 model. It's not always the case, but it _can_ be -- which, as a prospective buyer, is infuriating.
>Which still means that, if you buy a 2014 model, you might get a machine that's slower than the 2013 model.
I think what you mean is "You might get a machine that's slower than _a_ 2014 model". Assuming every 2014 model performs at the baseline, unless that baseline is lower than the previous generation's you're still getting better performance than the _official_ 2013 model.
I think what you mean is "You might get a machine that's slower than _a_ 2014 model". Assuming every 2014 model performs at the baseline, unless that baseline is lower than the previous generation's you're still getting better performance than the _official_ 2013 model.
And which model is the "official" one if not the one that I bought in the Apple Store? There is no official and unofficial one, they're all sold under the same name.
If I have a 2013 MBA I expect the SSD when I upgrade to a 2014 one to be at least as fast as the old one.
If I have a 2013 MBA I expect the SSD when I upgrade to a 2014 one to be at least as fast as the old one.
OK, let's put it this way: Apple doesn't guarrantee you a specific speed, and doesn't tell you you'll get a specific SSD model in your Air.
It just tells you you'll get a speedy SSD of X capacity (true), and you're not supposed to obsess over manufacturer and a couple percentage points of performance. Else, you wouldn't have bought the Air, but some custom rig.
It just tells you you'll get a speedy SSD of X capacity (true), and you're not supposed to obsess over manufacturer and a couple percentage points of performance. Else, you wouldn't have bought the Air, but some custom rig.
What expectation? Apple tech specs are very brief for the ssd drives, just size and 'PCIe based', and they have a nebulous comparison to a spinning drive. They also don't tell you what model processor you get, just family and speed.
The expectation is that a 2014 model will perform at least as well, if not better than, a predecessor model from 2013.
That's a perfectly reasonable expectation to hold, as well. One really shouldn't end up worse off (for whatever reason) merely by buying a more recent model, especially when it's a computer that's being purchased.
That's a perfectly reasonable expectation to hold, as well. One really shouldn't end up worse off (for whatever reason) merely by buying a more recent model, especially when it's a computer that's being purchased.
Completely agree. I suppose the benchmark tests were a good start, but far from complete. To back up what you said here is a post from 2013 "not all Mac SSD's are the same"
http://blog.macsales.com/19008-performance-testing-not-all-2...
http://blog.macsales.com/19008-performance-testing-not-all-2...
> already well-known that Apple sources SSDs (as well as other parts) from a variety of suppliers
Ah. Why would they do that? Someone could spend $3000 on MacBook Pro 16GB + SSD highest spec and risk the chance of getting bad parts? I never knew this was a practice. Is this even common in the industry otherwise Apple?
Ah. Why would they do that? Someone could spend $3000 on MacBook Pro 16GB + SSD highest spec and risk the chance of getting bad parts? I never knew this was a practice. Is this even common in the industry otherwise Apple?
It's risk-aversion. Think of it this way: if drive-models are assigned to Macs in a round-robin fashion, then if there are three drive manufacturers and one has shipped a buggy drive, at most 1/3 of customers would be sending their laptops back for repairs. On the other hand, if their only supplier gave them buggy drives, then everyone would be sending their laptops back.
It also lets them have a stronger negotiating position to drop a crap supplier. They can only maintain that negotiating position, though, if they buy drives from all of them; if they only used drives from whoever was the at-the-time "best" supplier, then if that supplier started to suck, all the others would demand a king's ransom for the "privilege" of switching to them.
It also lets them have a stronger negotiating position to drop a crap supplier. They can only maintain that negotiating position, though, if they buy drives from all of them; if they only used drives from whoever was the at-the-time "best" supplier, then if that supplier started to suck, all the others would demand a king's ransom for the "privilege" of switching to them.
Interesting. I could also imagine that the inverse of that point is also true: if one of the suppliers is really good and reliably so, if Apple depends on them more than the others, the cash inflow will help that one supplier more than the others, which that company could put back into R&D to widen that quality gap. And if one supplier is far superior to the others, then the comparability == competition between suppliers lessens, and so does Apple's bargaining power.
Yeah, companies have been burned in the past for relying on too few suppliers of disks. For example, Amstrad.
Apple would probably argue that all the SSDs provide acceptable performance and some of them provide bonus performance beyond what was promised. The specs only mention "PCIe-based flash storage": http://www.apple.com/macbook-air/specs.html
Apple has done this various times in the past, like shipping a 16x DVD drive when they advertised 8x. More problematic was the rMBP where LG LCDs were noticeably lower quality than Samsungs. http://9to5mac.com/2013/03/20/macbook-pro-with-retina-displa...
Apple has done this various times in the past, like shipping a 16x DVD drive when they advertised 8x. More problematic was the rMBP where LG LCDs were noticeably lower quality than Samsungs. http://9to5mac.com/2013/03/20/macbook-pro-with-retina-displa...
Wow, that's unfortunate about the LG vs Samsung screens. How could you tell what screen you have? I'd like to spread awareness about this to my friends.
If their displays are faulty, there will be a noticeable ghosting effect when a window is left in the same place for a long time and then moved somewhere else on the screen. A faded "ghost window" will remain in its place. Another issue is stuck/bright pixels, there may be several bright white pixels that never change.
If your friends aren't seeing noticeable problems though, there's nothing to warn them about. It's not that one display is inferior to the other in terms of color accuracy or brightness or quality. One display (I think it's the LG) has a noticeable ghosting and bright pixel problem in some cases.
If your friends aren't seeing noticeable problems though, there's nothing to warn them about. It's not that one display is inferior to the other in terms of color accuracy or brightness or quality. One display (I think it's the LG) has a noticeable ghosting and bright pixel problem in some cases.
Oh ok. That's not too bad. And it sucks you can't just look up the hardware specs or something.
Use it at your own risk, but I found this test from Marco Arment to be one of the most popular ones to determine if your retinaMacBook Pro has ghosting / burn-in problem.
http://www.marco.org/rmbp-irtest.html
I am not sure which one of two manufacturers display is faulty, but per discussions on Apple's support Communities (https://discussions.apple.com/message/18669644#18669644), rMBPs carrying display panel from LG tends to suffers from this problem more commonly compared to those from Samsung.
EDIT: Just ran this test on a retinaMacBookPro to determine if I have a LG or Samsung Panel and found it to be working.
http://geeksblogs.com/lg-or-samsung-heres-a-little-guide-to-...
http://www.marco.org/rmbp-irtest.html
I am not sure which one of two manufacturers display is faulty, but per discussions on Apple's support Communities (https://discussions.apple.com/message/18669644#18669644), rMBPs carrying display panel from LG tends to suffers from this problem more commonly compared to those from Samsung.
EDIT: Just ran this test on a retinaMacBookPro to determine if I have a LG or Samsung Panel and found it to be working.
http://geeksblogs.com/lg-or-samsung-heres-a-little-guide-to-...
This was pretty easily fixed, though. The technician took one look at it, nodded, and I had it replaced for free while on Christmas + New Year vacation.
Maybe in the US. But in the Philippines, we're out of luck.
EDIT: I mean about the replacement part. Sure, they'll probably replace it after a several back and forth, plus the replacement will take serveral weeks.
EDIT: I mean about the replacement part. Sure, they'll probably replace it after a several back and forth, plus the replacement will take serveral weeks.
> I never knew this was a practice.
All the component manufacturers that supply Apple are also supplying other contracts.
An example is the high-definition screens; 'Retina' Apple laptops use interchangeable screens from Sharp, LG and Samsung as availability permits. There was a run of returns last year due to some issues with the LG screens.
All the component manufacturers that supply Apple are also supplying other contracts.
An example is the high-definition screens; 'Retina' Apple laptops use interchangeable screens from Sharp, LG and Samsung as availability permits. There was a run of returns last year due to some issues with the LG screens.
This is why I've stuck with my 2012 13" unibody. You can get 16GB and gobs of SSD into one for way cheaper.
Is the variance between manufacturers really that dramatic? If so, as a future buyer, how can I make sure I get the fastest among the ostensibly identical SSDs for a given configuration?
Return policy
What you're saying is scary to me. Does this mean I could buy a Macbook Air and get a slower one than if you buy one?
Could anyone offer any tips for evaluating processors in this post-megahertz age? My 2012 Macbook Air (base 11" model) has a 1.7 GHz Core i5 processor, yet I understand the 1.4 GHz (Haswell) Core i5 in the latest machines performs significantly better.
Is the generation of the processor always the dominant factor nowadays? Is there a convenient way to estimate performance without poring over benchmarks? What are people's favourite sites for finding out about these things?
Is the generation of the processor always the dominant factor nowadays? Is there a convenient way to estimate performance without poring over benchmarks? What are people's favourite sites for finding out about these things?
> Is the generation of the processor always the dominant factor nowadays?
No.
> Is there a convenient way to estimate performance without poring over benchmarks?
Unless you are willing to quite a lot of time learning, not really. And unless you've been willing to do that, you haven't been able to conveniently estimate performance for about 14 years now.
> What are people's favourite sites for finding out about these things?
http://www.anandtech.com/bench/ is pretty good, as they standardize all results into a convenient database with easy head-to-head comparisons.
No.
> Is there a convenient way to estimate performance without poring over benchmarks?
Unless you are willing to quite a lot of time learning, not really. And unless you've been willing to do that, you haven't been able to conveniently estimate performance for about 14 years now.
> What are people's favourite sites for finding out about these things?
http://www.anandtech.com/bench/ is pretty good, as they standardize all results into a convenient database with easy head-to-head comparisons.
PassMark: http://www.cpubenchmark.net/
Tomshardware has the good benchmarks for "hacker" usage.
In my opinion Phoronix does the best job about benchmarking, but usually you must wait a little.
...and still only 4Gb of memory.
I think a non-anemic amount of memory is the most important feature lacking in the Air. I often find my 4GB 2011 Air chugging and breaking a sweat swapping to "disk" simply because I've had Firefox open for too long. Would it be that difficult for them to finally double the amount of memory in the next base model to 8GB and make 16GB an available option?
I think this could have something do with 16GB dimms being a new thing and not supported by Intel on the laptop chipsets.
Lenovo has a similar problem in their new models. X240 has only one dimm slot and currently supports max 8GB of memory. While investigating the issue I found a discussion[1] where somebody mentioned that Intel would need to add support for these new dimms in the "memory reference code"[2].
[1] http://forums.lenovo.com/t5/X-Series-ThinkPad-Laptops/16GB-m... [2] http://en.wikipedia.org/wiki/Memory_Reference_Code
Lenovo has a similar problem in their new models. X240 has only one dimm slot and currently supports max 8GB of memory. While investigating the issue I found a discussion[1] where somebody mentioned that Intel would need to add support for these new dimms in the "memory reference code"[2].
[1] http://forums.lenovo.com/t5/X-Series-ThinkPad-Laptops/16GB-m... [2] http://en.wikipedia.org/wiki/Memory_Reference_Code
I solved that problem by not using Firefox. (And this is not intended to be snarky - Firefox was my preference, but under near-identical conditions, Firefox exhibited this problem while Safari did not.)
I agree 4 GB is getting to be a little low for a modern laptop, but I'm curious as to how bad this "chugging" is considering swapping to SSD isn't too bad and is many times faster than any swapping that occurred on those old 5400 RPM drives.
Honestly it's more than enough for many use cases. OSX is quite lean (for a non-minimalist OS) and memory compression saves quite a bit before having to hit swap. Activities such as word processing and typical (i.e not geeky 100+ tabs) surfing around workloads fit very well. My girlfriend has the base 2013 11" (Haswell, 4GB/128GB) as her main machine and it's perfectly snappy. I have an early 2013 base 13" Retina (pre-Haswell, 8GB/256GB), do loads of dev with huge terminal scrollbacks (a real memory eater) and a few throwaway non-GUI VMs (also, shared video memory) and it'll be fine for at least the next four years.
The takeaway is not everyone has the same use case. Also IMHO bloated DOMs and careless coding† are the real pain point.
† It doesn't have to be optimised to the metal but seriously, quite a few people just develop as if we all have ideal Turing machines.
The takeaway is not everyone has the same use case. Also IMHO bloated DOMs and careless coding† are the real pain point.
† It doesn't have to be optimised to the metal but seriously, quite a few people just develop as if we all have ideal Turing machines.
I have 8GB of RAM in my Macbook Air that's over a year old. I just checked, and yes, it's still a $100 upgrade on either model.
I had 6GB of RAM in my main development box until very recently. Even ran a few VMs on it. 8GB is a good amount for most power users, especially considering that the limited CPU will probably be your bottleneck first.
I had 6GB of RAM in my main development box until very recently. Even ran a few VMs on it. 8GB is a good amount for most power users, especially considering that the limited CPU will probably be your bottleneck first.
I will admit I did not read the article.
Flash is trending towards fewer higher capacity chips. In some cases this is resulting in less parallelism and slower response times for individual reads and writes.
Not saying anything about this article and set of hardware, just in general that this has been my observation with new models of SSDs as they are released.
Unfortunately the way most applications are written the response time of SSDs is the primary benefit because they present low concurrency and don't parallelize the procesing of data as it comes off of disk. Still way better than seek times on spinning disk, but far from what could be.
Flash is trending towards fewer higher capacity chips. In some cases this is resulting in less parallelism and slower response times for individual reads and writes.
Not saying anything about this article and set of hardware, just in general that this has been my observation with new models of SSDs as they are released.
Unfortunately the way most applications are written the response time of SSDs is the primary benefit because they present low concurrency and don't parallelize the procesing of data as it comes off of disk. Still way better than seek times on spinning disk, but far from what could be.
The situation with NAND flash has been pretty disappointing, I think; in the quest for more capacity, manufacturers have been sacrificing other desirable aspects of storage, like write endurance and data retention. In particular, 3-bit and 4-bit MLC are becoming the norm, and they are both slower and less reliable than older 2-bit MLC or SLC, while requiring more complex error correction and bad block management. The relatively increasing fragility of flash storage is never mentioned, and many people don't find out until it's too late --- because the typical consumer only focuses on capacity.
I still miss the days when SLC was the norm, and you didn't need complex error-correcting-codes and bad block management. Now SLC is considered "high-end", even in enterprise applications, and becoming rarer to find, while 2-bit MLC, which used to be considered the inferior, consumer-level grade, has also become more difficult to find and a "professional" feature. Considering that endurance and retention decrease exponentially with each additional bit-per-cell while capacity only increases multiplicatively, the tradeoffs don't seem quite so great.
I still miss the days when SLC was the norm, and you didn't need complex error-correcting-codes and bad block management. Now SLC is considered "high-end", even in enterprise applications, and becoming rarer to find, while 2-bit MLC, which used to be considered the inferior, consumer-level grade, has also become more difficult to find and a "professional" feature. Considering that endurance and retention decrease exponentially with each additional bit-per-cell while capacity only increases multiplicatively, the tradeoffs don't seem quite so great.
I'm glad TLC exists; the 840 Evo is fast enough and reliable enough and it's much cheaper than a 2-bit MLC SSD.
I'm not saying it is a bad thing if you understand the tradeoffs and can work within them, but what I'm getting at is that a lot of consumers unfortunately don't.
The extreme endurance torture tests that get reported often don't tell the whole story either - the more flash cells are cycled, the "leakier" they become and retention goes down significantly. Figures I've seen for SLC are 10 years retention after 100K P/E cycles, earlier-generation MLC 5 years after 10K P/E cycles, newer MLC is 5 years @1.5~3K, TLC is 1 year @ <1K. Of course retention tests don't make for as interesting news articles as endurance ones since they're almost like watching paint dry, but IMHO they are just as if not more important, and manufacturers should provide warnings that flash-based devices are intrinsically unstable and their retention ability is measured in years. "Bit rot" is a reality with NAND flash. I only hope that people who think they've "backed up" data onto used SSDs, memory cards, and other forms of high-capacity flash don't find that much of it has literally self-erased and disappeared after only a few years.
The extreme endurance torture tests that get reported often don't tell the whole story either - the more flash cells are cycled, the "leakier" they become and retention goes down significantly. Figures I've seen for SLC are 10 years retention after 100K P/E cycles, earlier-generation MLC 5 years after 10K P/E cycles, newer MLC is 5 years @1.5~3K, TLC is 1 year @ <1K. Of course retention tests don't make for as interesting news articles as endurance ones since they're almost like watching paint dry, but IMHO they are just as if not more important, and manufacturers should provide warnings that flash-based devices are intrinsically unstable and their retention ability is measured in years. "Bit rot" is a reality with NAND flash. I only hope that people who think they've "backed up" data onto used SSDs, memory cards, and other forms of high-capacity flash don't find that much of it has literally self-erased and disappeared after only a few years.
Unfortunately the way most applications are written the response time of SSDs is the primary benefit because they present low concurrency and don't parallelize the procesing of data as it comes off of disk.
One of the blessings of an education in hardware; you learn to think in a distributed, parallel model.
People are really bad at that until they learn/train otherwise. Our brains are parallel at low levels, but conscious thought seems to be mostly serial.
One of the blessings of an education in hardware; you learn to think in a distributed, parallel model.
People are really bad at that until they learn/train otherwise. Our brains are parallel at low levels, but conscious thought seems to be mostly serial.
Crap - I just bought a new Macbook Air the first day it was available. Oh well, I guess I'll never really notice...
Seeing as the 2013 MacBook Air had 2X as fast an SSD as the 2012 model, which in turn was 2X faster than the 2011 model and he 2011 model feels super snappy, yeah I think you're ok.
I have the 2010 model...probably time to upgrade...
Hopefully you'll benefit from a better build. My 2013 MacBook Air has a nasty issue where trackpad doesn't work properly if the machine is tilted even 15 degrees. The adapter has failed once already and the second one stopped charging last week.
Whatever you do, get the 3 year care is all I can say.
Whatever you do, get the 3 year care is all I can say.
In the Netherlands, there are several shops that are offering 3 year guarantees on all laptops sold. Including Macbooks.
Considering the terrible customersupport that I've received multiple times from Apple, I bought my latest Macs at one of those stores.
Considering the terrible customersupport that I've received multiple times from Apple, I bought my latest Macs at one of those stores.
I'm in America and I've never had anything but incredibly good customer support from Apple. Interesting.
When did you buy it? Did the 1 year warranty run out?
I got it 8 months ago but noticed the problem into the fifth month. Intend to have it sent in for repair when I can make time to part with it.
I bought 2 Macbook airs together, both have the same issue.
I bought 2 Macbook airs together, both have the same issue.
According to the replies above, you can run benchmark by yourself, maybe some Macbook Air still have fast SSDs.
mixing up machines order and what is in bold for sensationalism much?
The 2014 drives were faster than the 2013 in many of the test cases, specially 2nd page.
The 2014 drives were faster than the 2013 in many of the test cases, specially 2nd page.
And it's still faster than PCs. :)
Even though your statement is not necessarily true, I bought a Vaio T11 almost two years ago and I was able to upgrade the SSD (the laptop has the footprint of the 11" MacBook air) and I have a HDD (in addition to the SSD), that I can upgrade very easily, I also had a free SoDIMM slot and was able to double the ram from 4GB to 8GB. That's what I like about PCs. Here with the mac you only have one drive that you cannot change or upgrade and the ram is soldered.
It has nothing to do with them being Macs vs. PCs, it just depends on who they buy the SSDs from.
my brain just exploded
Apple makes some nice hardware but they are not alone in making good stuff. They have some stiff competition.
For example my Sony Vaio Pro is significantly faster than any of the numbers mentioned in the article. It gets nearly 1GB/sec in read speeds: http://i.imgur.com/24GdPiz.png
The drive is partitioned so ignore the odd size.
For example my Sony Vaio Pro is significantly faster than any of the numbers mentioned in the article. It gets nearly 1GB/sec in read speeds: http://i.imgur.com/24GdPiz.png
The drive is partitioned so ignore the odd size.
They won't get much stiff competition from Sony:
http://www.cnet.com/news/sony-agrees-to-sell-pc-business-rev...
http://www.cnet.com/news/sony-agrees-to-sell-pc-business-rev...
Going forward, no, but they did in the past and others will step in to fill those shoes. I hear ASUS makes some good stuff currently.
The thing is, you don't need 1GB/sec for an 11-13" laptop.
This comment is so bad it was featured on @shit_hn_says: https://twitter.com/shit_hn_says/status/462524610844442626
Why? I boot VMs and 1 GB/sec makes it happen in 5 seconds