How does everybody backup?
30 comments
I use the latest beta version of the online backup software I'm writing (www.tarsnap.com), and then copy said beta onto a USB key which I carry around with me.
Yes, I like eating dogfood. :-)
Yes, I like eating dogfood. :-)
Under Linux we use Rsync. Its great and only the changed portions of changed files are updated. Under Windows I use Viono Backup (www.vionobackup.com). Its in beta at the moment, but this is secure and encrypted and works similar to Rsync
dropbox (http://getdropbox.com , our yc startup) -- keeps a folder synced across pcs and to the web (with continuous syncing, an rsync-like protocol and versioning) for windows/os x. basically we had the same problem and tried the same rsync+cron hacks and wanted to solve the problem right.
we'll be kicking out news.yc beta invites soon :)
we'll be kicking out news.yc beta invites soon :)
1.Check to make sure there are no small children or cute animals behind the vehicle
2.Start car and shift into reverse
3.While looking backwards, slowly release the brake
2.Start car and shift into reverse
3.While looking backwards, slowly release the brake
All source material (.py, .psd etc) and documentations go in to local LAN subversion, Subversion is svnsync'd to a remote repository every 15 minutes.
All email mirrored from off site to local IMAP server once a day.
SuperDuper on macs to firewire drive once a week.
rsync and cron on FreeBSD to remote FreeBSD boxes once a day
No PC's so no problem there (for ever) ;)
All email mirrored from off site to local IMAP server once a day.
SuperDuper on macs to firewire drive once a week.
rsync and cron on FreeBSD to remote FreeBSD boxes once a day
No PC's so no problem there (for ever) ;)
What do you mean by no PCs?
server wise i use a very simple script to dump an archive to S3. http://paulstamatiou.com/2007/07/29/how-to-bulletproof-serve...
but for personal computer files I manually put important files in an S3 bucket or two. not the most efficient but it works. I'm considering writing some rsync to S3 type of thing. regardless i love having my stuff online (and safe). my MBP doesn't have much other than essential apps and a few files i'm working on at the moment. most other things are online.
the only exception being media which i sync to an external drive manually every few months.
but for personal computer files I manually put important files in an S3 bucket or two. not the most efficient but it works. I'm considering writing some rsync to S3 type of thing. regardless i love having my stuff online (and safe). my MBP doesn't have much other than essential apps and a few files i'm working on at the moment. most other things are online.
the only exception being media which i sync to an external drive manually every few months.
You may be interested in Duplicity, which can do encrypted backups to S3: http://duplicity.nongnu.org/ Duplicity uses the rsync algorithm for data transfer as well.
For those creating archives, you may be interested in xar: http://code.google.com/p/xar/ It preserves all the metadata such as SELinux information, ACLs, EAs, etc. On Mac OS X, it preserves all that, the resource fork and finder metadata as well. Two options are particularly relevant to backups: --link-same makes identical file data into hardlinks, reducing space consumption both within the archive, and on disk when extracted. --coalesce-heap when files with identical data are encountered, the data is only stored once in the archive, reducing the size of the archive.
For those creating archives, you may be interested in xar: http://code.google.com/p/xar/ It preserves all the metadata such as SELinux information, ACLs, EAs, etc. On Mac OS X, it preserves all that, the resource fork and finder metadata as well. Two options are particularly relevant to backups: --link-same makes identical file data into hardlinks, reducing space consumption both within the archive, and on disk when extracted. --coalesce-heap when files with identical data are encountered, the data is only stored once in the archive, reducing the size of the archive.
I keep any sort of file I really care about in a personal svn repository. Mostly this is either code, PDF's, Word Docs, Excel, text files, and so on - Not too much video or anything like that.
I then use DatedBackup (http://datedbackup.rubyforge.org/), to backup the repos, and a few servers, which is basically a wrapper around rsync. It more or less mimicks the functionality of a simplified time machine without the gui - keeping snapshots at certain intervals, and deleting them when the drive gets full. It uses a technique similar to rsnapshot, so extra space doesn't get wasted by every snapshot.
I then use DatedBackup (http://datedbackup.rubyforge.org/), to backup the repos, and a few servers, which is basically a wrapper around rsync. It more or less mimicks the functionality of a simplified time machine without the gui - keeping snapshots at certain intervals, and deleting them when the drive gets full. It uses a technique similar to rsnapshot, so extra space doesn't get wasted by every snapshot.
Thanks guy for your comments. Helpful and interesting.
I found a piece about backup, that is a must read: http://www.jwz.org/doc/backups.html
I especially like this one: "" Put one of these drives in its enclosure on your desk. Name it something clever like "Backup". If you are using a Mac, the command you use to back up is this:
sudo rsync -vaxE --delete --ignore-errors / /Volumes/Backup/
If you're using Linux, it's something a lot like that. If you're using Windows, go fuck yourself. ""
Maybe this is a good end for this thread ;o)
I found a piece about backup, that is a must read: http://www.jwz.org/doc/backups.html
I especially like this one: "" Put one of these drives in its enclosure on your desk. Name it something clever like "Backup". If you are using a Mac, the command you use to back up is this:
sudo rsync -vaxE --delete --ignore-errors / /Volumes/Backup/
If you're using Linux, it's something a lot like that. If you're using Windows, go fuck yourself. ""
Maybe this is a good end for this thread ;o)
I use Time Machine.
I use Time Machine, but intend to use SuperDuper as well once the Leopard compatibility issues are solved. (It's taking a long time, but that appears to be because the SuperDuper author is a really careful guy. That's what you want in a backup vendor.)
For Linux I just use rsync. Restoring a failed Linux system often requires fiddling with Knoppix as well; get a nice recent Knoppix disc burned and ready to go in an emergency.
Don't forget: have more than one backup, and have an offsite backup.
I never really tested my Windows system backup solutions; I decided to abandon Windows instead. For backing up Windows files I used to use SyncBack: http://www.2brightsparks.com/downloads.html -- it was pretty good.
I need to work on doing better online backups. A lot of my work is web stuff, though, and that lives on other servers anyway.
For Linux I just use rsync. Restoring a failed Linux system often requires fiddling with Knoppix as well; get a nice recent Knoppix disc burned and ready to go in an emergency.
Don't forget: have more than one backup, and have an offsite backup.
I never really tested my Windows system backup solutions; I decided to abandon Windows instead. For backing up Windows files I used to use SyncBack: http://www.2brightsparks.com/downloads.html -- it was pretty good.
I need to work on doing better online backups. A lot of my work is web stuff, though, and that lives on other servers anyway.
I too used SuperDuper! with Tiger but as soon as I got my hands on Leopard, I started using TM. Why? Well, it's automatic. Problem with SD! is that I would forget to plug my external HD sometimes and the backup would not go through. You can enable to backup TM to a HD attached to AirportExtreme (with a small hack) so that's my makeshift TimeCapsule and works well because I don't have to remember to do anything.
Mac backup: Last day of the month I run Carbon Copy Cloner to backup everything to a 500GB firewire external drive. (firewire, so that I can boot OSX from that image, if need be.)
PC Backup: I dumped my PCs 18 months ago when I bought my Intel-chip Mac. Now both my "PCs" (XP development machine and Win2k3 Web Server) are running as virtual machines on my Mac, so my Windows install is actually just one big file used by Parallels which is backed up as part of the Mac backup.
--Jack
PC Backup: I dumped my PCs 18 months ago when I bought my Intel-chip Mac. Now both my "PCs" (XP development machine and Win2k3 Web Server) are running as virtual machines on my Mac, so my Windows install is actually just one big file used by Parallels which is backed up as part of the Mac backup.
--Jack
yes, thats exactly my home setup. Macs save the world.
All my backups are made to a NAS.
On Mac OS X I use Silverkeeper (I don't have leopard).
On Windows I use SyncBack [1].
All my development work is on a remote SVN server which is backed to the NAS.
The NAS is backed up to DVDs which are shipped to my parents house.
Any recommendations for online remote storage?
[1] Syncback tutorial http://lifehacker.com/software/geek-to-live/geek-to-live-aut...
On Mac OS X I use Silverkeeper (I don't have leopard).
On Windows I use SyncBack [1].
All my development work is on a remote SVN server which is backed to the NAS.
The NAS is backed up to DVDs which are shipped to my parents house.
Any recommendations for online remote storage?
[1] Syncback tutorial http://lifehacker.com/software/geek-to-live/geek-to-live-aut...
cron + rsync
Most days nothing's really changed in the folders I've selected for backup; when things do change it's usually just a new directory of pictures, maybe an mp3, and some new source code; the whole backup takes maybe a minute to run in that case.
Most days nothing's really changed in the folders I've selected for backup; when things do change it's usually just a new directory of pictures, maybe an mp3, and some new source code; the whole backup takes maybe a minute to run in that case.
Boooring. I'll do it tomorrow.
Interesting question. I consider S3 with a front-end like http://www.jungledisk.com/
Does anybody have any experience with that?
Does anybody have any experience with that?
Jungledisk is great. Its easy to use, cheap and runs on Windows, Linux (includes GUI and CLI) and Mac. You can treat is as a network drive or setup automated backups. The protocol is open so you can use tools like S3 Firefox Organizer to retrieve your data outside of the Jungledisk front-end.
Another big plus is that you can run an unlimited number of instances across any number of machines after paying a one time fee of $25.
Another big plus is that you can run an unlimited number of instances across any number of machines after paying a one time fee of $25.
I'll also chime in for Jungledisk. I use it on 2 machines at home and a few at work (Windows Vista, Server 2003). The cost and security is really nice.
Jungledisk Plus gives you the option to view files online - so it's a backup + online viewing option (since it uses Amazon S3 and is already online).
Jungledisk Plus gives you the option to view files online - so it's a backup + online viewing option (since it uses Amazon S3 and is already online).
That looks really neat. I had wondered about using S3 for backups. I'll definitely have to try it out.
It looks like the "Plus" service's block-level access and transfer resume would pay for itself pretty quickly with a reasonable volume of data ($1/month vs 0.10 or 0.18 per GB up/down), besides being worth it for $1 anyway.
It looks like the "Plus" service's block-level access and transfer resume would pay for itself pretty quickly with a reasonable volume of data ($1/month vs 0.10 or 0.18 per GB up/down), besides being worth it for $1 anyway.
I second the Rsync recommendation. I back up our servers every 15 minutes with it. In the event of a catastrophe, we will have lost 15 minutes worth of data, at most.
Have a look at rsnapshot (rsnapshot.org), a perl wrapper script over rsync. Supports *nix and incremental backups that roll over.
Yes, rsnapshot is excelent. The documentation is good too.
We also use gpg (gnupg.org) to encrypt our critical databases and code repository and send them to Amazon S3 every night. I started doing this after the last earthquake here in the bay area (which was centered near Freemont, where our colo is). I have tattooed the key to unencrypt them on my inner thigh.
We also use gpg (gnupg.org) to encrypt our critical databases and code repository and send them to Amazon S3 every night. I started doing this after the last earthquake here in the bay area (which was centered near Freemont, where our colo is). I have tattooed the key to unencrypt them on my inner thigh.
I've been using git for backups the past few weeks. Works incredibly well.
I only backup code and documents though. For other things it might not be such a great solution.
I only backup code and documents though. For other things it might not be such a great solution.
SVN for code. Time Machine for OS X. Typical DB saves for the web, along with server image backups at Slicehost.
I've been trying out wuala - http://wua.la/en/home.html
I usually put the car into reverse, turn my head to look out the rear windshield then apply a little gas.
Personal stuff I copy to a USB drive in the house.
For business stuff it's tar + ssh to a geographically distant location.
For business stuff it's tar + ssh to a geographically distant location.
I just copy my /bin directory onto an 8GB thumb drive. Rotate once a week into fireproof box.
Why would you do this? Isn't /home where your data lives?
On my system, anything that changes is in /bin. For a catastrophe, I would reinstall the OS and environment and then restore data from the thumb drive.
Interesting. Could you elaborate, please?
OK. I bet I'm the only one on this board using this technology...
http://www-306.ibm.com/software/data/u2/universe/
EVERYTHING is stored as data on the u2 database. All javascript, css, html, php, and mysql is generated from this data through custom builds. No need to backup anything except the source data and the custom programs used to generate everything else. And all of that is within the IBM/u2/bin directory.
Have a disaster? Restore the environment, restore the u2 database & code, and rebuild everything else. I designed it for the development leverage. The ease of disaster recovery is just a bonus.
http://www-306.ibm.com/software/data/u2/universe/
EVERYTHING is stored as data on the u2 database. All javascript, css, html, php, and mysql is generated from this data through custom builds. No need to backup anything except the source data and the custom programs used to generate everything else. And all of that is within the IBM/u2/bin directory.
Have a disaster? Restore the environment, restore the u2 database & code, and rebuild everything else. I designed it for the development leverage. The ease of disaster recovery is just a bonus.
You've picked well :-)
I use Mozy for off-site backup and the free SyncBack to sync to an extenral USB harddrive.
rdiff-backup locally; duplicity onto an rsync.net system.
I use rdiff also. It is an rsync mirroring system
http://www.nongnu.org/rdiff-backup/
I use SpiderOak (BTW: I'm a programmer for SpiderOak)
How about cheap online storage to rsync to?
tar and scp to my server. For versioning I use cp -r.
Does this make me a bad person?
Does this make me a bad person?
I like Ghost on Windows.
Time Machine to a drobo.
2 USB keys (kept in separated places) plus external disk (connected only for copying).
Should suffice...
Should suffice...
Since OS X Apple always came with decent freeware solutions that could be used with external firewire drives for example. Time Capsule is a logical (and great) successor.
But what about linux and windows? At work I use linux and we backup with ibms tivoli software. This is crap of course, because replay of a crashed or exchanged hd requires installation of a linux system first in order to access the backup. After that the data has to be moved bit by bit and apps have to be re-installed one by one.
As long as I used windows (until 2000), it was even worse. (sidenote: the degree with which an os is called professional is reverse to its backup capabilities. This strikes me since ever!).
How does everybody backup their macs, linux boxes or windows systems?