Ask HN: Online backup without a computer?
28 comments
Why not just use FTP?
Set up your camera to save the photos to in different folders based on date, then just get some web space, go to ftp://yoursite/ and drag&drop the photos.
It's low tech, but it's almost guaranteed to work from anywhere, and you're less likely to get kicked out for "hacking" when you don't have a cygwin terminal open.
Set up your camera to save the photos to in different folders based on date, then just get some web space, go to ftp://yoursite/ and drag&drop the photos.
It's low tech, but it's almost guaranteed to work from anywhere, and you're less likely to get kicked out for "hacking" when you don't have a cygwin terminal open.
I'd rather have something that can work out what I've already uploaded (and ideally with partial file resume) so that I don't have to keep track of everything myself. I probably won't always have good enough upload bandwidth to be able to always catch up in one session, so just organising by date isn't ideal. But it's not too much hassle to just keep Uploaded/NotUploaded folders locally and move things when done, so this is probably the fallback position.
What's a good SCP/FTP-over-SSH client for Windows? Filezilla seems to work OK, but is a little clunky (on the Mac, anyhow. I don't have a windows machine to hand to test it on)
What's a good SCP/FTP-over-SSH client for Windows? Filezilla seems to work OK, but is a little clunky (on the Mac, anyhow. I don't have a windows machine to hand to test it on)
The best SCP/FTP-over-SSH client for Windows I've encountered is WinSCP (http://winscp.net/eng/index.php).
It comes in a standalone install-nothing flavour and lets you choose, when copying a group of files from A to B, to only copy files that have changed.
I'm not aware of what algorithm is used to determine when a file has changed. I've always assumed this is based on file modification time so this might not be clever enough to spot a partially-uploaded file and either re-copy or start from where it left off.
You can certainly pause file transfers and subsequently resume but only during a session.
It comes in a standalone install-nothing flavour and lets you choose, when copying a group of files from A to B, to only copy files that have changed.
I'm not aware of what algorithm is used to determine when a file has changed. I've always assumed this is based on file modification time so this might not be clever enough to spot a partially-uploaded file and either re-copy or start from where it left off.
You can certainly pause file transfers and subsequently resume but only during a session.
Consider using Dropbox (A YC startup).
https://www.getdropbox.com/
You can get 2gb for free, or pay for more if you need it. If you need more than you're able to pay for, you could have a home PC syncing the stuff to another location and removing the original to free up the space.
https://www.getdropbox.com/
You can get 2gb for free, or pay for more if you need it. If you need more than you're able to pay for, you could have a home PC syncing the stuff to another location and removing the original to free up the space.
I've played with Dropbox before, so it's an option, although I'd prefer something with finer granularity of pricing. I'll probably need more than 2GB, but unlikely to need more than 10, so jumping straight to 50 is quite a leap.
I also got the impression that Dropbox doesn't cope so well with being run from a different computer every day, and would want to sync all my existing stored files down onto the computer I happen to be using that day, which isn't so good. And AIUI I'd need to make double-double-sure I'd uninstalled it from each when I was done, and notify Dropbox that this machine is no longer active etc. But I admit I haven't dug too deeply into all that yet.
I also got the impression that Dropbox doesn't cope so well with being run from a different computer every day, and would want to sync all my existing stored files down onto the computer I happen to be using that day, which isn't so good. And AIUI I'd need to make double-double-sure I'd uninstalled it from each when I was done, and notify Dropbox that this machine is no longer active etc. But I admit I haven't dug too deeply into all that yet.
Couldn't you have the dropbox folder as a folder on your removable drive, removing the need for syncing every time?
So have your home PC move files from the Dropbox folder somewhere else every day. That way when you go to upload your next set of files, you've got a fresh 2GB of unused space. Since the folder would be empty, there'd be nothing to sync back down to the new PC.
Just buy more SD cards. They are cheap and you can pack a lot of them into the space taken by HDD. And you won't have to look for a PC with internet.
Well, my little Buffalo mini-station (http://www.amazon.co.uk/dp/B000UEFQ1O) is pretty small, and has a lot of other stuff on it, so I'll be carrying it anyway, but as per my final paragraph the option of just loading up on a few more SD cards is one I've considered. I'd just like to minimise the loss in a disaster scenario (such as everything I'm carrying getting stolen). Hardware can be replaced, photos can't.
A 4GB card at retail, 5 months ago, was $7. How many pictures do you think you'll be taking? 16GB should be plenty of space.
Interesting question. I really should have worked that out already :)
The current plan involves being in about 20 countries over the course of just under 3 months. So let's say 90 days x 50-100 photos per day x 2-3MB per picture. That gives somewhere in the region of 16GB right there; with backup, twice that. And that's probably on the low side.
Hmmm. I'm going to need more space (and more upload bandwidth) than I thought!
The current plan involves being in about 20 countries over the course of just under 3 months. So let's say 90 days x 50-100 photos per day x 2-3MB per picture. That gives somewhere in the region of 16GB right there; with backup, twice that. And that's probably on the low side.
Hmmm. I'm going to need more space (and more upload bandwidth) than I thought!
I've never set up Cygwin on a USB drive, and then tried to transport it between computers, but that's the first thing I'd try.
Then you could write a simple script to do an rsync-over-ssh to your server, and just run it directly from the USB drive.
It would probably be best to test it on at least both Vista and XP systems, to make sure it works.
I'd use ssh keys, so that you don't have to worry about keystroke logging. You could also make a user for it on your server side, and lock it down to just allow the rsync in.
Look into the '--link-dest' option to rsync, this allows you to really easily do incremental backups, each time you backup your new backup directory has all the files in it, but any files that haven't changed are hardlinked to the previous version, so you don't take up much disk space. Really, really cool.
Then you could write a simple script to do an rsync-over-ssh to your server, and just run it directly from the USB drive.
It would probably be best to test it on at least both Vista and XP systems, to make sure it works.
I'd use ssh keys, so that you don't have to worry about keystroke logging. You could also make a user for it on your server side, and lock it down to just allow the rsync in.
Look into the '--link-dest' option to rsync, this allows you to really easily do incremental backups, each time you backup your new backup directory has all the files in it, but any files that haven't changed are hardlinked to the previous version, so you don't take up much disk space. Really, really cool.
I think the Cygwin approach might be a little too much, but thanks for the --link-dest pointer: I wasn't aware of that before, and it'll be handy for other things I'm doing.
If it's just photos, you could possibly use Flickr Pro? It might be tedious downloading them all later if you lose them, but it's a fairly obvious online home for photos, especially if you want the world to stumble across them. Just another option, really.
Heh. I already have a Flickr Pro account that I use actively, and hadn't thought of that :)
I wouldn't want to pollute my normal stream with the 95% of dross that I'll doubtless take, but setting the default visibility to private could work well.
ISTR that flickr does some processing of the images as you upload them though, and don't really like the idea of losing the originals, but perhaps that only impacts the lower res versions, and the original always stays intact. Must investigate further...
I wouldn't want to pollute my normal stream with the 95% of dross that I'll doubtless take, but setting the default visibility to private could work well.
ISTR that flickr does some processing of the images as you upload them though, and don't really like the idea of losing the originals, but perhaps that only impacts the lower res versions, and the original always stays intact. Must investigate further...
If you got Flickr Pro the originals always stay intact. If you want (don't think you can considering general upload speeds in internet cafe's) you can even upload the RAW's and Flickr will keep them available for your at all times.
FileZilla on the USB hard drive (http://filezilla-project.org/)
It doesn't need to be installed, you can just run it from the drive, and it can transfer via FTP or SSH to a cheap slicehost or dreamhost. I even think it supports syncing directories, so you can just resume where you left off.
It doesn't need to be installed, you can just run it from the drive, and it can transfer via FTP or SSH to a cheap slicehost or dreamhost. I even think it supports syncing directories, so you can just resume where you left off.
http://www.eye.fi/ and a flickr.com account. The card connects to any open wi-fi access point and just uploads everything to flickr.com or other online photo site. When you get back, go through all of your photos and delete the ones you don't need. Better than lugging around a USB harddrive.
I'm assuming that you are assuming (it gets tedious) that that computer you use to dump your data with to the HD will have network access.
Have a look at this (I just took a peek at it this morning, it's built by another HN poster):
https://beta.tarsnap.com/
Have a look at this (I just took a peek at it this morning, it's built by another HN poster):
https://beta.tarsnap.com/
> will have good network access
You will need a reasonably fast Internet access to use the online backup. Depending on where you will be traveling, you may run into upstream bandwidth being severely restricted.
You will need a reasonably fast Internet access to use the online backup. Depending on where you will be traveling, you may run into upstream bandwidth being severely restricted.
Yeah, that's one of the unknown quantities in all this, and it's one of the reasons why I'd like something that can remember what's been uploaded already (and ideally handles partial file transfers etc). Then I can just leave it running everytime I'm online and have it just pick up where it left off next time. Then when the connection isn't great I can at least hopefully get _something_ uploaded, and when I hit good connectivity everything will catch up.
You can't install tarsnap on library computers / internet cafes, and so on.
true, but you could bring a usb stick with ubuntu NBR on it and use that instead. It's my favourite way of taking a computer with me when I don't want to bring a laptop.
All my tools and stuff exactly where I expect it and no chance of having my email or ssh stuff compromised because of some nifty keyboard sniffing virus in an internet cafe. Usually the owners don't mind the reboot.
All my tools and stuff exactly where I expect it and no chance of having my email or ssh stuff compromised because of some nifty keyboard sniffing virus in an internet cafe. Usually the owners don't mind the reboot.
I'll be spending most of the trip in countries where I don't speak the language, though, so trying to explain what I'm doing to them could be interesting, especially if there are network settings to configure etc.
I've done this in Colombia with good success and believe me my spanish is non-existent.
As long as you pay was usually the attitude.
The more advanced internet cafe's (usually with a card based system where you buy a pre-paid card with some time on it) are more difficult.
As long as you pay was usually the attitude.
The more advanced internet cafe's (usually with a card based system where you buy a pre-paid card with some time on it) are more difficult.
I would put them on S3 via any of the FTP clients that support it. Just keep the installers on the hard drive.
I can, of course, carry software on the HD that could be installed temporarily on computers I'm using in order to manage this, but I'd like to minimise the amount of hoops I'll need to jump through at each location (i.e. installing Cygwin everywhere to run rsync isn't really an option!)
I have my own server that I can upload to, but I'm also open to software that stores data on S3 or wherever (this will hopefully be write once, read never!)
Suggestions?
(Alternatives also welcome, e.g. "Don't bother with online backup: just carry a couple of extra SD cards and backup to those", but online backup has the added benefit of being more resistant to everything getting stolen / destroyed when I fall into a lake / whatever)