Amazon S3: Versioning Beta Design(doc.s3.amazonaws.com)
doc.s3.amazonaws.com
Amazon S3: Versioning Beta Design
http://doc.s3.amazonaws.com/betadesign/Versioning.html
5 comments
potential to use this instead of a dropbox account? the code could be substantially easier to write.
That was my absolute first thought.. While I love dropbox, the team, and enjoy reading about their immense success, the security nerd in me demands that everything I put my data on is "mine" for more or less the definition that while I might not own the servers or disks, I have a great level of access to them.
I wrote about that in regards to dropbox in 2008: http://paulstamatiou.com/thoughts-on-dropbox
"Use my own S3 account – I would pay a lot to have the ability to store my data in my own S3 account. I realize part of Dropbox’s business plan likely revolves around charging for storage, but I propose they offer a high-end account where I just pay $30/month for the use of their API to power file revisions, the web interface and all of that, all-the-while being stored on my own S3 account."
Of course that point is now moot if I can get the file versions natively with S3.
I wrote about that in regards to dropbox in 2008: http://paulstamatiou.com/thoughts-on-dropbox
"Use my own S3 account – I would pay a lot to have the ability to store my data in my own S3 account. I realize part of Dropbox’s business plan likely revolves around charging for storage, but I propose they offer a high-end account where I just pay $30/month for the use of their API to power file revisions, the web interface and all of that, all-the-while being stored on my own S3 account."
Of course that point is now moot if I can get the file versions natively with S3.
What open source tools are you using to backup to S3 from linux? I imagine the good ones will add in support for S3 versioning.
s3cmd is a command line tool interfacing to S3, makes scripting your backups downright simple. http://s3tools.org/s3cmd
Another is the Boto Python library (more or less a wrapper for all AWS stuff). http://code.google.com/p/boto/
Hopefully both of these will get version control support real soon.
Another is the Boto Python library (more or less a wrapper for all AWS stuff). http://code.google.com/p/boto/
Hopefully both of these will get version control support real soon.
I use duplicity: http://www.nongnu.org/duplicity/
How is this different than simply adding a version suffix to each file name?
By appending version suffixes by your self, all your clients/apps must be made aware of the current version suffixe and coordinate it (say you have multiple files with different version), can get real messy. And with that method, there is no way to mimic deletes either.
I think the main goal of S3 is to provide a dropbox like feature. You can simply PUT/GET/DELETE/COPY just like before and not worry about version control... and s3 just transparently handles it for you
I think the main goal of S3 is to provide a dropbox like feature. You can simply PUT/GET/DELETE/COPY just like before and not worry about version control... and s3 just transparently handles it for you
Does it do deduplication, or do I have to pay for too much storage when a large file changes just a little each day?
This is why I use JungleDisk.
This is why I use JungleDisk.
I'm wondering the same thing too - how did they build this ?
ZFS (after pool ver. 21 I suppose) has deduplication - are they building on something similar ?
ZFS (after pool ver. 21 I suppose) has deduplication - are they building on something similar ?