Optical Disc Data Rot: Burned CDs going bad(howtogeek.com)
howtogeek.com
Optical Disc Data Rot: Burned CDs going bad
https://www.howtogeek.com/682807/the-cds-you-burned-are-going-bad-heres-what-you-need-to-do/
20 comments
Ha ha. I was doing that pre-Napster (I think it was 97 or 98). I would duplicate my schoolmates CDs if they provided a second blank disc for myself, so I could keep a copy if I liked it!
I used to have a closet full of 1.44mb disks... Some games took about 50 of them.
When I record to DVDs, I always add error correction using dvdisaster.[1]
In addition, I sometimes have par2 files[2] on the DVD as well, though it's probably better to just have additional dvdisaster ECC instead.
It's a good idea to periodically go through all your old backups and transfer them to new media, or at least make sure the old media still works and start recovery on them right away if they don't.
[1] - http://dvdisaster.net/
[2] - https://github.com/Parchive/par2cmdline
In addition, I sometimes have par2 files[2] on the DVD as well, though it's probably better to just have additional dvdisaster ECC instead.
It's a good idea to periodically go through all your old backups and transfer them to new media, or at least make sure the old media still works and start recovery on them right away if they don't.
[1] - http://dvdisaster.net/
[2] - https://github.com/Parchive/par2cmdline
I had pretty good luck using a program that could be programmed to extract all readable sectors first and then go over and over damaged areas to try and recover the data. I can’t recall the name of it but it had a picture of the developers cat on the GUI. (If it is one of the ones you mentioned already I apologize) Bad link on dvd disaster.
I have had good use of FEC for archival purposes, plenty of testing but I haven't had to use it in anger.
This isn't the same tool I used, but it is derived from it, https://github.com/randombit/fecpp
This isn't the same tool I used, but it is derived from it, https://github.com/randombit/fecpp
I've been meaning to try M-DISCs [0] after I found out that my BD drive supports burning them. Apparently, they last longer than normal discs because the drives burn "a permanent hole in the material, rather than changing the color of a dye".
I don't use optical discs much anymore, but I still like the idea of archiving things like family pictures on read-only media once in a while.
[0] https://en.wikipedia.org/wiki/M-DISC
I don't use optical discs much anymore, but I still like the idea of archiving things like family pictures on read-only media once in a while.
[0] https://en.wikipedia.org/wiki/M-DISC
M-DISC is mostly irrelevant these days unless you want to use DVD for some reason.
The biggest benefit of M-DISC for DVDs was that it didn't use an organic dye, as those dyes tend to decay over time. The dominant BD-R recording mechanism (HTL) uses an inorganic recording medium so isn't affected by this problem.
There are differences in how BD-Rs are manufactured that can lead to big differences in longevity [0] but there are regular retail BD-Rs that outperform M-DISC in accelerated aging tests [1]. There are also DVD variants that outperform M-DISC [2].
Also, M-DISC hasn't produced any evidence for their BD-R discs. For their DVDs there were a few scientific studies they could point to that demonstrated their longevity, for BD-R they have no such evidence.
[0]: https://www.lne.fr/sites/default/files/inline-files/etude-qu...
[1]: https://www.lne.fr/sites/default/files/inline-files/etude-qu...
[2]: https://www.lne.fr/sites/default/files/inline-files/syylex-g...
The biggest benefit of M-DISC for DVDs was that it didn't use an organic dye, as those dyes tend to decay over time. The dominant BD-R recording mechanism (HTL) uses an inorganic recording medium so isn't affected by this problem.
There are differences in how BD-Rs are manufactured that can lead to big differences in longevity [0] but there are regular retail BD-Rs that outperform M-DISC in accelerated aging tests [1]. There are also DVD variants that outperform M-DISC [2].
Also, M-DISC hasn't produced any evidence for their BD-R discs. For their DVDs there were a few scientific studies they could point to that demonstrated their longevity, for BD-R they have no such evidence.
[0]: https://www.lne.fr/sites/default/files/inline-files/etude-qu...
[1]: https://www.lne.fr/sites/default/files/inline-files/etude-qu...
[2]: https://www.lne.fr/sites/default/files/inline-files/syylex-g...
So basically I want to buy Panasonic or Sony HTL BDs for archiving and store them somewhere dark and dry?
Unfortunately that data is eight years old and I'm not even sure if Panasonic and Sony even make BD-Rs anymore. Seeing a lot of out of stock Japanese imports on Amazon.de for both of them. I can really only find Verbatim BD-Rs and they didn't really perform that well in the comparisons.
I've actually been interested in BD archiving for a bit but haven't gotten into it yet because of the initial costs. I have about 1.1 TB of data that I'd like to archive for the long-term.
Unfortunately that data is eight years old and I'm not even sure if Panasonic and Sony even make BD-Rs anymore. Seeing a lot of out of stock Japanese imports on Amazon.de for both of them. I can really only find Verbatim BD-Rs and they didn't really perform that well in the comparisons.
I've actually been interested in BD archiving for a bit but haven't gotten into it yet because of the initial costs. I have about 1.1 TB of data that I'd like to archive for the long-term.
They're both definitely making them. If you got to a retail store in Japan, they'll have a HUGE section filled with BD-Rs and if you search Amazon JP you can see that tons of them are in stock.
You can always import them yourself. Tenso and Buyee both work great.
I think it's just that the western market doesn't care for them much, so you don't see anybody bothering to import them for you.
You can always import them yourself. Tenso and Buyee both work great.
I think it's just that the western market doesn't care for them much, so you don't see anybody bothering to import them for you.
x50 HTL BD-R (indicated as "non-organic") for video by SONY is sold on 3355 JPY. Possibly Amazon.co.jp supports international shipping.
https://www.amazon.co.jp/dp/B01H8A8HH2/
https://www.amazon.co.jp/dp/B01H8A8HH2/
Their inorganic properties of the data layer are also another big point leading to their longevity. Most other burnable media have organic data layers which essentially rust and break down in the presence of oxygen.
M-DISC is surprisingly cheap. Amazon is selling an M-DISC-capable USB drive for $22 and 4.7GB M-DISC DVD-R for $1.68 each.
The costs for getting started with BD ones are quite steep in my mind though. It's about 100 euro to get an external BD burner with M-DISC support and then like 6 euro per 25 GB disc or 20 euro for 100 GB disc. So if you were to archive 100 GB now, you'd be paying like 120 euro. 1000 GB and it's 300 euro.
I have friends who I remember always insisting on burning their CDs and DVDs at the maximum speed the discs allowed for, and they all had problems with their discs dying with time, some very soon, some after a year or two. No matter what brand. Conversely, I never burned any of my CDs or DVDs at more than half their advertised maximum speed, and so far I've yet to run into a single disc dying from age. Some of these discs are now 20 years old, and they still read/play fine.
My theory - and this is just a theory as I don't really know what's going on - is that the dyes in CD-Rs and DVD-Rs after laser exposure slowly regress back towards the original transparent state, and that very "fast" dyes combined with a very short exposure time ("up to 52x speed!") emphasizes this problem, while giving the dyes a longer exposure time prevents it by simply burning the dye opaque beyond "recovery".
My theory - and this is just a theory as I don't really know what's going on - is that the dyes in CD-Rs and DVD-Rs after laser exposure slowly regress back towards the original transparent state, and that very "fast" dyes combined with a very short exposure time ("up to 52x speed!") emphasizes this problem, while giving the dyes a longer exposure time prevents it by simply burning the dye opaque beyond "recovery".
At the beginning of current year I dumped my data which I have written to CDs in years 1997-99. The CD where dark green or blue, one or two were golden. All this disks were written with 1x or 2x speed. Then I had access to the Yamaha CDR102 burner which was SCSI. Burning disks then was a fragile process involving a lot of mumbling prayers.
I dumped to harddrive about 100 disks... it was all MP3 and AVI files. One disk had a hole in its reflecting layer the size of cross-section of a matchstick, nevertheless I could read whole disk.
I don't think I will throw away these disks. I will just put them in my basement. Who knows how long that 2.5 inch harddrive will be in working conditions :)
I remember in 1998 ordering my Yamaha COD and waiting out in the cold Minnesota snow to get it from the UPS driver, then paying 1$/disk. Total cost per GB amortized over the life of the drive was like 2$ since a newer faster model came out 3 months later. Now I get pissed if storage is more than about 0.04$/gb. Times have changed.
Guessing these were the golden ones?
https://www.amazon.com/Verbatim-DVD-R-4-7GB-UltraLife-Archiv...
https://www.amazon.com/Verbatim-DVD-R-4-7GB-UltraLife-Archiv...
> The CD where dark green or blue, one or two were golden
For some reason this image just brought back a whole load of memories for me. Some of the first of these I encountered were previews of my Dad's music he would send me on those ever so dark green discs.
For some reason this image just brought back a whole load of memories for me. Some of the first of these I encountered were previews of my Dad's music he would send me on those ever so dark green discs.
A thought occurred to me on a recovery method while reading this article: If you had a disc that was marginal, where it might or might not read correctly depending on the phase of the moon, would it be possible to do something like take multiple raw images of the disc, perhaps using multiple different drives, and collectively use the images so captured to isolate out the bad parts and splice together a valid image file?
Or am I failing to grok the degree to which It's Complicated?
Or am I failing to grok the degree to which It's Complicated?
Yeah, apparently there's software that does that to get a rip that's identical bit for bit.
http://www.exactaudiocopy.de/en/index.php/overview/basic-tec...
http://www.accuraterip.com/
http://www.exactaudiocopy.de/en/index.php/overview/basic-tec...
http://www.accuraterip.com/
For audio CDs, there's also CueTools, which can repair slightly damaged rips (I assume it stores some parity information). I have a CD where that appears to have either two slightly different pressings or a bad batch, and one has an audible click at the end of one of the tracks. CueTools detects this error, and lets me fix it.
From memory here, GNU ddrescue is what you're looking for:
https://www.gnu.org/software/ddrescue/
I used this a few months back to get some data off old floppies, along with some suggested commands from https://www.archiveteam.org/index.php?title=Talk:Rescuing_Fl...
Very neat piece of software.
Very neat piece of software.
That's what Exact Audio Copy does when you rip a scratched CD, except on the frame-level.
By coincidence, I'd been looking into the problem of bitrot and what options exist. Hard drives are a starting point and last longer than removable discs. If going for optical, M-DISC seems to be by far the best option.
https://uncentered.saigonist.com/decentralize-your-data-part... https://uncentered.saigonist.com/decentralize-your-data-part...
https://uncentered.saigonist.com/decentralize-your-data-part... https://uncentered.saigonist.com/decentralize-your-data-part...
I still have a few spindles of CD-R's probably topping out at 15 years old now - managed to burn copies of linux for some old machines earlier on in the year though with no errors.
Really doubt I will end up using much more of them at this rate though besides using them as "retro" coasters.
Really doubt I will end up using much more of them at this rate though besides using them as "retro" coasters.
Can anyone recommend encoding settings for archiving DVD-Video discs created on a recorder?
I've taken dd images, but can't find a reasonable "archival" h265 profile that doesn't come out larger than the original.
I've taken dd images, but can't find a reasonable "archival" h265 profile that doesn't come out larger than the original.
As you've discovered, you can't transcode highly-compressed (meaning, encoded for distribution) digital media without either a quality or file size tradeoff. So, try to avoid that.
If you want to merge related VOB files into one video file, that can be done losslessly with tools like tsMuxer[1].
[1] https://github.com/justdan96/tsMuxer
If you want to merge related VOB files into one video file, that can be done losslessly with tools like tsMuxer[1].
[1] https://github.com/justdan96/tsMuxer
I've been meaning to write up a page on how to do this, but basically:
- Take an ISO rip and never delete it since any additional encoding is a new derivative work of your data.
- Run it through MakeMKV to get a Matroska file for each title
- Assuming a video source (e.g. recorded TV shows or camcorder footage) use QTGMC to deinterlace the video from 60 fields per second to 60 frames per second. This will approximate the original look on today's screens by simulating the slight delay in the dimming of a CRT's phosphors after the electron beam excites them.
- Crop the video if any black bars are present. Many DVDs have 8px of dead area on the left and right edges because they are rooted in the D-1 4:2:2 standard and often have a program area resolution of 702 or 704x480: https://en.wikipedia.org/wiki/D-1_(Sony)
- Convert the chroma from SD colorspace to HD colorspace to avoid them looking washed-out on modern displays: https://en.wikipedia.org/wiki/Rec._601
- Decide how to handle the aspect ratio. People use "anamorphic" to mean "widescreen DVD" but that's kinda a misnomer because all DVDs are anamorphic: its 720x480 pixel resolution is 3:2, halfway between 16:9 and 4:3. Each title sets a flag telling the display to stretch it ±18.6%, depending. You can leave it like this, setting the same DAR flag, but I've come to prefer resizing it myself to a square-pixel resolution since I'm usually cropping it anyway and VapourSynth will do a way better job than most displays can do on the fly. I also dislike the idea of throwing away horizontal pixel data by resizing 4:3 video down from 720x480 to 640x480, so some times I stretch it vertically instead, to 720x540 (or 720x544 with 4px black along the bottom if you want to follow the ATSC 3.0 HEVC recommendations).
- Encode the video with x265. I have a personal "SD" tuning that I've been working on and would love to share once I can get some example clips together :)
- Encode the probably-AC3 audio with FDK-AAC. I find it handles voices and other things better than faac thanks to its low-pass step, but for very high-bitrate sources (like concert DVDs) I disable that by using VBR-5: http://wiki.hydrogenaud.io/index.php?title=Fraunhofer_FDK_AA...
- Mux your new streams together into a new MKV, and take a moment to fill in all the metadata fields the container provides. You can even attach a JPEG named "cover.jpg" and have it display as the icon in Explorer!
- Take an ISO rip and never delete it since any additional encoding is a new derivative work of your data.
- Run it through MakeMKV to get a Matroska file for each title
- Assuming a video source (e.g. recorded TV shows or camcorder footage) use QTGMC to deinterlace the video from 60 fields per second to 60 frames per second. This will approximate the original look on today's screens by simulating the slight delay in the dimming of a CRT's phosphors after the electron beam excites them.
- Crop the video if any black bars are present. Many DVDs have 8px of dead area on the left and right edges because they are rooted in the D-1 4:2:2 standard and often have a program area resolution of 702 or 704x480: https://en.wikipedia.org/wiki/D-1_(Sony)
- Convert the chroma from SD colorspace to HD colorspace to avoid them looking washed-out on modern displays: https://en.wikipedia.org/wiki/Rec._601
- Decide how to handle the aspect ratio. People use "anamorphic" to mean "widescreen DVD" but that's kinda a misnomer because all DVDs are anamorphic: its 720x480 pixel resolution is 3:2, halfway between 16:9 and 4:3. Each title sets a flag telling the display to stretch it ±18.6%, depending. You can leave it like this, setting the same DAR flag, but I've come to prefer resizing it myself to a square-pixel resolution since I'm usually cropping it anyway and VapourSynth will do a way better job than most displays can do on the fly. I also dislike the idea of throwing away horizontal pixel data by resizing 4:3 video down from 720x480 to 640x480, so some times I stretch it vertically instead, to 720x540 (or 720x544 with 4px black along the bottom if you want to follow the ATSC 3.0 HEVC recommendations).
- Encode the video with x265. I have a personal "SD" tuning that I've been working on and would love to share once I can get some example clips together :)
- Encode the probably-AC3 audio with FDK-AAC. I find it handles voices and other things better than faac thanks to its low-pass step, but for very high-bitrate sources (like concert DVDs) I disable that by using VBR-5: http://wiki.hydrogenaud.io/index.php?title=Fraunhofer_FDK_AA...
- Mux your new streams together into a new MKV, and take a moment to fill in all the metadata fields the container provides. You can even attach a JPEG named "cover.jpg" and have it display as the icon in Explorer!
Thanks for the writeup! Would BD be much different? Do you recommend converting x264 to x265 for Blu-Ray sources?
BDs usually aren't interlaced or overscanned, so the hard part is done for you :)
I do usually re-encode from from AVC to HEVC but with a more usual HD tuning. Not mine, but check out https://docs.google.com/spreadsheets/d/1u9fKNsRcLgc-1oVV5GhB...
I do usually re-encode from from AVC to HEVC but with a more usual HD tuning. Not mine, but check out https://docs.google.com/spreadsheets/d/1u9fKNsRcLgc-1oVV5GhB...
I noticed that some CDs/DVDs are already going bad after very few years in the early 2000s, so I put everything on harddisks, recovered broken stuff with ddrescue and was done with writable optical media.
> Sometimes, CD- and DVD-Rs were burned at high speeds with high error rates that were normally fixed on the fly by error-correcting code. This can make it more difficult for modern optical disc drives to read them, so try them on an older drive, if possible.
Does that mean drives were first made with relatively robust ECC, and they just stopped using it for later drives? What would be the motivation to drop that?
Does that mean drives were first made with relatively robust ECC, and they just stopped using it for later drives? What would be the motivation to drop that?
Is there good Linux software for cataloging offline media and identifying duplicates? On Windows/Mac/iOS there is the venerable (since 2000) http://abemeda.com.
In my past, I've purchased some high quality CD-Rs used by studios. Not sure if they made a difference though. I didn't see any definitive information with a quick search.
I've long since abandoned using optical as a form of stroage...
I've long since abandoned using optical as a form of stroage...
I always burned at the slowest possible speed, with all of the error-checking if it was available, on the best media I could find. As a strategy, it largely paid off. More expensive, more time-consuming, less frustrating later.
last year I spent a while ripping all my burned disks to redundant harddrives. I had to use isobuster on a couple but having multiple redundant backups helps me sleep better at night.
[deleted]
vinyl records are having their revenge.
Isn't that trading one failure mode for another? While vinyl records won't rot over time, they do wear down, and thus the sound quality deteriorates each time you play it.
In theory, yes. But in practice it’s not at any rate that most people should worry about. Plus some records are more durable than others.
One could extend the vinyl argument and say there’s no such thing as a storage medium without a failure mode. Cassettes have problems, HDDs can fail, even solid state systems only have a limited life. The best we can do is backup.
One could extend the vinyl argument and say there’s no such thing as a storage medium without a failure mode. Cassettes have problems, HDDs can fail, even solid state systems only have a limited life. The best we can do is backup.
> The best we can do is backup.
No. The best you can do is use mathematics to compensate. In essence represent the data as a set of equations and generate extra ones that will be useful in case you lose one. They have been doing this for decades in telecom/storage.
No. The best you can do is use mathematics to compensate. In essence represent the data as a set of equations and generate extra ones that will be useful in case you lose one. They have been doing this for decades in telecom/storage.
And as I already said, storage will always be subject to fail.
You can have 10 TB of parity for a 1 MB file but if that data isn’t distributed then you’re still vulnerable.
So the problem is analog vs digital nor the lack of mathematics in vinyl; it’s that ultimately any form of storage media requires a physical device and physical devices are bound to the problems of the physical world.
This, by the way, is also why Amazon has multiple data centres in any given region. Why backups should always be stored off site, and why DR solutions are based in different physical premises.
You can have 10 TB of parity for a 1 MB file but if that data isn’t distributed then you’re still vulnerable.
So the problem is analog vs digital nor the lack of mathematics in vinyl; it’s that ultimately any form of storage media requires a physical device and physical devices are bound to the problems of the physical world.
This, by the way, is also why Amazon has multiple data centres in any given region. Why backups should always be stored off site, and why DR solutions are based in different physical premises.
Sounds like time to build an optical pickup head for vinyl -- no wear or static, but you get the "vinyl" experience.
Clearly you've never left a prized record in a hot car/direct sun :( Suspect you're just /s though
I dont know how my cheap burnt CD's from the early 2000's are still fine but some of my pressed CD's are suffering from Disc rot!
How common is Disc Rot for pressed CD's?
I recently bought roughly 100 CD-ROMs (~$650) from eBay of mid to late 90's Macintosh games and 3 of them were rotting (all from the outer edge, the reflectivity was lost in these spots) but still mostly readable. They were from roughly 20 different sellers in the US and Europe.
I regularly buy 2nd hand audio CDs as old as the 80’s and never had a bad one
Factory-made CDs are duplicated by a physical process. Most bad CDs were made using an opto-chemical process.
The chemical processes are pretty much reversible, leading to dead CDs. The physical processes are much more robust, as you'd have to deform the actual base plastic, like cooking vinyl LPs in the sun.
The chemical processes are pretty much reversible, leading to dead CDs. The physical processes are much more robust, as you'd have to deform the actual base plastic, like cooking vinyl LPs in the sun.
Probably survivorship bias as the only old CDs for sale today are in good shape
-Still only anecdata, but I've been buying lots of music on CD since the late eighties, the oldest I know of was pressed in late 1983 sometime. (Which is only a year or so after the introduction of the CD anyway)
In sum, some 3,800 CDs. I ripped them all a couple of years ago - I had problems with a handful, but EAC eventually got them all to disk. I examined the troublesome ones, but no (visual) signs of disc rot.
In sum, some 3,800 CDs. I ripped them all a couple of years ago - I had problems with a handful, but EAC eventually got them all to disk. I examined the troublesome ones, but no (visual) signs of disc rot.
Much more common is top/label-side damage of the reflective substrate. The bottom is protected but the top isn't!
Not CDs, but one brand/manufacturer of HD-DVDs is notorious for rot (I've heard suggestions they used bad glue, but I haven't confirmed this). Of the 20 or so I have (all Warner Brothers IIRC), at least half are unreadable (usually the first 20-50% is readable then it errors out).
They did replace 300 with a Blu-Ray copy (this was after HD-DVD was obviously on its way out), but the rest didn't fail until much later or were bought used for literally pennies.
For pressed CDs, I can't ever remember getting one that was unreadable and not obviously scratched (though I did have a game CD shatter into hundreds of pieces in the drive).
They did replace 300 with a Blu-Ray copy (this was after HD-DVD was obviously on its way out), but the rest didn't fail until much later or were bought used for literally pennies.
For pressed CDs, I can't ever remember getting one that was unreadable and not obviously scratched (though I did have a game CD shatter into hundreds of pieces in the drive).
It depends on the factory, the discs and the storage. I don't often hear of audio CDs rotting but equally I don't often hear of people still using audio CDs in any large quantities. However I do hear of people buying retro games for old CD systems like the Sega Saturn and how many of those games are suffering from rot.
I'm lucky that I've not suffered any rot yet but it's only a matter of time for me because I don't store them properly either.
I'm lucky that I've not suffered any rot yet but it's only a matter of time for me because I don't store them properly either.
I’m surprised there’s nothing about storage conditions.
Is it the plastic or the metals breaking down? If it’s plastic, humidity probably has a bigger impact.
Lower temps are almost always better. Freezer is good. Hard to say if an oxygen absorber is helpful.
Or could sparge your container with nitrogen.
Is it the plastic or the metals breaking down? If it’s plastic, humidity probably has a bigger impact.
Lower temps are almost always better. Freezer is good. Hard to say if an oxygen absorber is helpful.
Or could sparge your container with nitrogen.
I still have some CDs from the early/mid 1980's that play, and I think this is because of two reasons: 1) Not many people owned players back then, so pressing plants weren't under much pressure from high demand, so pressing quality was higher. 2) I stored them in their jewel cases oriented vertically on the shelf, supported by the plastic fingers of the case - so no sagging from being stored horizontally. And this was indoors, away from sunlight.
So far as home-burned CDs, they haven't lasted as well. Mostly because I didn't take as good care of them - I didn't have $18 or more invested in them (typical new music CD price in the 80's & 90's) so they were just tossed in a drawer.
So far as home-burned CDs, they haven't lasted as well. Mostly because I didn't take as good care of them - I didn't have $18 or more invested in them (typical new music CD price in the 80's & 90's) so they were just tossed in a drawer.
Generally it's the photosensitive layer between the plastic and the reflective foil that goes bad.
You can have the reflective layer delaminate, but CD-R bit rot usually has no observable physical changes.
You can have the reflective layer delaminate, but CD-R bit rot usually has no observable physical changes.
There was also a special type of blank disc, the re-writable blank disc which was something like 10 bucks a pop at future shop.
I had a enterprising career in school at one point burning shared music mp3s that I pulled from Napster and selling them to friends.