What Every Data Programmer Needs to Know About Disks(oscon.com)
oscon.com
What Every Data Programmer Needs to Know About Disks
http://www.oscon.com/oscon2011/public/schedule/detail/20137
5 comments
Thank you for that link. This is a far more interesting presentation than the OT. (Though it does assume some prior knowledge, like what fsync is, etc.)
Found the video if anyone else is interested (same talk, but from 2007) : http://mirror.linux.org.au/pub/linux.conf.au/2007/video/talk...
This talk totally failed to properly discuss the issues of the various kinds of fsync, instead (it seems to me) just making a selection of vague comments. That is a shame, as really properly understanding file integrity is a really important issue, which is very easy to get very wrong.
When ext4 came out, I remember a large number of apps losing data, as the metadata and 'normal data' caches can get out of order, meaning that:
Write file B. Rename B -> A.
Can result (after a crash) in A being empty, as the rename can occur before the contents of B are actually written.
When ext4 came out, I remember a large number of apps losing data, as the metadata and 'normal data' caches can get out of order, meaning that:
Write file B. Rename B -> A.
Can result (after a crash) in A being empty, as the rename can occur before the contents of B are actually written.
Agreed. These slides are like "Disks are not memory 101". I'd hope the talk went into more details.
After an hour or so reading semi-related material I have an understanding that goes beyond these basics :-/
After an hour or so reading semi-related material I have an understanding that goes beyond these basics :-/
Converted the Presentation PPTX to PDF for accessibility: http://dl.dropbox.com/u/5137/pdf-link/What%20Every%20Data%20...
Slide 6: "[Disk] throughput: 768 MB/sec on SATA 3" is misleading, consumer-grade rotating disks barely fill SATA 1 with ~70 MB/sec. SSD are faster in raw transfer, and if you have a RAID you are not limited by a single SATA channel any more.
Slide 13: "Laugh at MongoDB" is pretty stupid, mongo makes certain trade-offs that are not for everyone, but they are still meaningful.
The presentation doesn't explore the RAID options, which is a shame.
Slide 13: "Laugh at MongoDB" is pretty stupid, mongo makes certain trade-offs that are not for everyone, but they are still meaningful.
The presentation doesn't explore the RAID options, which is a shame.
He's referring to the bus speed of SATA 3 as an upper bound -- but like you mentioned, if you throw RAID and such into the mix, I'm not so sure either that it's a very useful number.
http://www.oscon.com/oscon2008/public/schedule/detail/3172
Get the presentation here: http://flamingspork.com/talks/2007/06/eat_my_data.odp