Ice Box Pro: The best of Dropbox and Amazon Glacier combined(iceboxpro.com)
iceboxpro.com
Ice Box Pro: The best of Dropbox and Amazon Glacier combined
http://www.iceboxpro.com/
21 comments
Your edit/addendum would have been way cooler if it read:
"Looks like it was just patched. Still managed to get a few dozen AWS keys though. I sent emails to the affected parties."
EDIT: effected-> affected (I'm not sure what's up with the snarky comment. Do most people think its inappropriate/snarky to urge proper notification?)
"Looks like it was just patched. Still managed to get a few dozen AWS keys though. I sent emails to the affected parties."
EDIT: effected-> affected (I'm not sure what's up with the snarky comment. Do most people think its inappropriate/snarky to urge proper notification?)
That's a moot point, surely? Everyone who used the app should receive an email from the people behind it very, very quickly - they've all been compromised.
The concern isn't just that RKearney has the keys - it is that anyone could have the keys for anyone on the site. Sending an email to the people whose keys he snagged would help them - but the people whose keys he didn't are still vulnerable too.
The concern isn't just that RKearney has the keys - it is that anyone could have the keys for anyone on the site. Sending an email to the people whose keys he snagged would help them - but the people whose keys he didn't are still vulnerable too.
I imagine you were a little unsure of the timeline of things when you commented. Please keep in mind that I wrote this comment before the "never give your info" story and before the website developer commented here on HN. With that in mind I am not sure what point is a moot point?
Do I think the developer should email all of the users? Yes, which is why in my response to the developer's comment about destroying the IAM keys I said "You should think about sending an email to all of your users..."
Do I think that the right thing for rkearney to do is send emails to the people whose information he has? Yes. Is it the best possible scenario? No, but it is better than no notification at all, which at the time was a possibility.
Do I think the developer should email all of the users? Yes, which is why in my response to the developer's comment about destroying the IAM keys I said "You should think about sending an email to all of your users..."
Do I think that the right thing for rkearney to do is send emails to the people whose information he has? Yes. Is it the best possible scenario? No, but it is better than no notification at all, which at the time was a possibility.
sorry about that. like i said before, we accidentally "launched" today. be nice?
if anyone's concerned about your AWS key, just destroy your IAM user and create a new one. that's what it was designed for.
if anyone's concerned about your AWS key, just destroy your IAM user and create a new one. that's what it was designed for.
You should think about sending an email to all of your users. I realize its a tough call pointing out a problem so early but it might also be a good way to garner user's trust.
good idea.
actually, we'll just wipe them and force new ones.
actually, we'll just wipe them and force new ones.
"Whoops, we disclosed everybody's AWS credentials. I know! Rather than tell our users, I'll wipe the database and remove all evidence of it ever happening."
Legally you must disclose any sort of security breach to your users: http://en.wikipedia.org/wiki/Security_breach_notification_la...
But you still should let your users know that their old keys have been compromised.
Dude... this guy clearly doesn't understand security. I'm glad you're trying to help him out... but your time will be better spent elsewhere.
Well that's not a very helpful attitude. People should be instructive rather than discourage someone from pursuing a reasonable project.
It's not helpful, it's realistic. My exact wording was "your time will be better spent elsewhere."
We each have 24 hours in a day. I spend 8-9 of mine sleeping, 9-10 at work. That doesn't leave much leftover for me.
If zacharyvoase wants to spend his precious free time educating people who haven't done any due diligence to learn how to build a secure web app, that's his prerogative. But I don't see it as a good use of his time - there's already tons of resources out there that will do a better job than zachary. Is HN supposed to be a newbie education destination?
We each have 24 hours in a day. I spend 8-9 of mine sleeping, 9-10 at work. That doesn't leave much leftover for me.
If zacharyvoase wants to spend his precious free time educating people who haven't done any due diligence to learn how to build a secure web app, that's his prerogative. But I don't see it as a good use of his time - there's already tons of resources out there that will do a better job than zachary. Is HN supposed to be a newbie education destination?
in case you have issues with your AWS keys. RKearny's email:
[email protected]
https://secure.gravatar.com/avatar/f7d7b021fb488fe6a67ddb286...
[email protected]
https://secure.gravatar.com/avatar/f7d7b021fb488fe6a67ddb286...
That's your response to someone (admittedly, poorly) discovering and reporting a security vulnerability in your application? Telling him to "be nice" then dropping his e-mail and face as some kind of stick-waving, threatening gesture?
Congratulations on demonstrating to me and countless others why I shouldn't use any product that you EVER touch. You don't get a pass because you're just two nerds. You have a form with a submit button -- that's where your responsibility as a founder and custodianship of user data begins. Day 1, you're already a liability.
I realize this is a pretty direct attack but I'm appalled and staggered by your behavior in this thread. You launched a service on the public Internet. There is no grace period, there is no "friendly fire"; you fucked up and you disclosed AWS credentials. Not users' favorite colors. AWS KEYS. Tied to credit cards, running servers, S3 backups, God knows what. You don't get to tell people to be nice to you when you're acting as the steward of AWS credentials; you protect them and act like you care when someone tells you that you fucked up doing so.
Your behavior here is just foreboding for the future, and you need to realize that before launching your next endeavor (this one is probably done, after that little mess).
Congratulations on demonstrating to me and countless others why I shouldn't use any product that you EVER touch. You don't get a pass because you're just two nerds. You have a form with a submit button -- that's where your responsibility as a founder and custodianship of user data begins. Day 1, you're already a liability.
I realize this is a pretty direct attack but I'm appalled and staggered by your behavior in this thread. You launched a service on the public Internet. There is no grace period, there is no "friendly fire"; you fucked up and you disclosed AWS credentials. Not users' favorite colors. AWS KEYS. Tied to credit cards, running servers, S3 backups, God knows what. You don't get to tell people to be nice to you when you're acting as the steward of AWS credentials; you protect them and act like you care when someone tells you that you fucked up doing so.
Your behavior here is just foreboding for the future, and you need to realize that before launching your next endeavor (this one is probably done, after that little mess).
Yes. This the entire way. Fuck everything about this situation, christ. Thank god I didn't sign up.
This wasn't some exotic exploit either. Public, numbered (1,2,3...) accounts, all of them editable - it's almost funny. Can you imagine what other security problems exist in the code.
EDIT: Guys, don't downvote smeagle's comments. If anything, we should be upvoting them as much as possible so that others can see this blatant disregard for their users
---------
You are quickly making the case for having one of the worst responses I've ever seen, to a huge security flaw. Trying to wipe things under the rug when your users information is clearly exposed is an easy way to destroy trust.
I seriously can't fathom the ineptitude it takes to direct people to someones email like that over your glaring mistake.
---------
You are quickly making the case for having one of the worst responses I've ever seen, to a huge security flaw. Trying to wipe things under the rug when your users information is clearly exposed is an easy way to destroy trust.
I seriously can't fathom the ineptitude it takes to direct people to someones email like that over your glaring mistake.
Classless. Come on Khang - everyone here wants to root for their fellow entrepreneurs, creators, and (self-proclaimed) "nerds".
RKearny pointed out a very real, very important issue that will help you make your service better, and help you deliver even more value for your users. And he did it for free! You should be thanking him and asking him for more feedback, not deflecting responsibility like this.
RKearny pointed out a very real, very important issue that will help you make your service better, and help you deliver even more value for your users. And he did it for free! You should be thanking him and asking him for more feedback, not deflecting responsibility like this.
ryan's info is public. he put it on our feedback forum. i wanted to make sure everyone was aware of his public info since we (as well as others) were very concerned with his course of action and questionable statement "Still managed to get a few dozen AWS keys though."
i'm not sure why thanking him is in order... ?
5 people emailed me privately about the security issue. we fixed it promptly, and followed up with instructions to everyone exposed (~20) on how to protect their credentials. i haven't yet heard a complaint from our actual users.
i'm not sure why thanking him is in order... ?
5 people emailed me privately about the security issue. we fixed it promptly, and followed up with instructions to everyone exposed (~20) on how to protect their credentials. i haven't yet heard a complaint from our actual users.
Great example of why building an app != a startup. You should email all users, and post a public apology on your site or blog.
you and i know that saying "~20" is a random number since you had nothing in place to track it. i'd love to hear how you know it's 20. seriously. tell us.
Could be http server access logs but if I made glaring mistakes like that I surely would not trust my own server logs anymore.
Stop digging yourself into to a hole and just apologise to Ryan and your users like you should have done to begin with.
[deleted]
[email protected]
https://si0.twimg.com/profile_images/2550813276/qnltv3bylbd6...
I can search for email addresses too! Don't direct users to me because you failed to secure your web application.
It's nice to know someone who works at a company that handles credit card and bank payments would just post someones email address and photo. Granted this is all public since I posted on the Uservoice post, it was still unnecessary.
I can search for email addresses too! Don't direct users to me because you failed to secure your web application.
It's nice to know someone who works at a company that handles credit card and bank payments would just post someones email address and photo. Granted this is all public since I posted on the Uservoice post, it was still unnecessary.
WePay is in no way related to IceBox. they're terrific guys; don't trash on their reputation.
i left WePay ages ago. also, our info is publicly displayed here:
http://www.iceboxpro.com/about/team
we're not a company. we're two nerds.
i left WePay ages ago. also, our info is publicly displayed here:
http://www.iceboxpro.com/about/team
we're not a company. we're two nerds.
If you were a company, you'd have insulation against lawsuits. Two nerds mean you and your family's assets are at risk; launching an app with such a spectacular security hole seriously puts the two of you and your families in danger.
> we're not a company. we're two nerds.
Oh, so if you get sued for mishandling personal data or PII it'll be your personal responsibility rather than a company's?
Oh, so if you get sued for mishandling personal data or PII it'll be your personal responsibility rather than a company's?
The only thing you are right about is that WePay is terrific and I am glad you are not associated with them. How difficult it is to owe up to your mistakes instead of shooting the messenger?
You are clearly not very good at being nerds.
then you may wish to remove that email from your profile.
[deleted]
@smeagol. First of all you fucked up. So stop acting so high and mighty and apologize to rkearney and everyone else who even thought about signing up for your product. Secondly, if this was just meant for you and your friends, don't go public with it and post it on HN. I highly recommend another hobby in a different field because clearly this one doesn't agree with you. Lastly, please stop calling yourself a nerd.
I think you misspelled www.iceboxamateur.com then?
Would you mind posting a list of any other projects you guys have worked on or are currently working on so I know what to avoid like the plague?
The best part of this whole catastrophe is that this app isn't just embarrassing for smeagol, but it is playing out HN, YC's main advertising venue, and it undermines the whole concept of these MVP summer vacation startup companies, showing that 2 guys a garage can't launch a minimally provisioned website.
That's a dick move. You should own up to your mistake.
Many of these keys aren't IAM keys, or they were made with full access to the users AWS account.
Luckily you can rotate keys even for the root level accounts.
Update:
I misunderstood, I'm sorry. I wasn't trying to attack but trying to show my concern because I thought he saved some id/keys for himself. Please ignore my comment below.
--
"Still managed to get a few dozen AWS keys though"
Good for you! What a nice person you are. Please abuse more small projects like this. Even if they say they say it was "was only meant for friends to test out".
Oh I see, you just found a security hole and trying to get some reputation? Cute. Please do it by abusing the small power you found and hurting innocent users. That's really, really nice of you.
"Still managed to get a few dozen AWS keys though"
Wow. Just wow.
You sir, just ruined my night. Thank you.
Ps. I am really concerned about your company and its users. If you can do something like this, I wonder what else you could do (or doing) at your current company. I hope, I'm assuming wrong.
edit: "the" » "your". last paragraph.
--
"Still managed to get a few dozen AWS keys though"
Good for you! What a nice person you are. Please abuse more small projects like this. Even if they say they say it was "was only meant for friends to test out".
Oh I see, you just found a security hole and trying to get some reputation? Cute. Please do it by abusing the small power you found and hurting innocent users. That's really, really nice of you.
"Still managed to get a few dozen AWS keys though"
Wow. Just wow.
You sir, just ruined my night. Thank you.
Ps. I am really concerned about your company and its users. If you can do something like this, I wonder what else you could do (or doing) at your current company. I hope, I'm assuming wrong.
edit: "the" » "your". last paragraph.
Perhaps you're misunderstanding my course of action.
1. I didn't disclose how to do it, merely that it was possible.
2. By "get" I in no way mean harvested. I just manually incremented the ID in the URL by hand in my web browser to see how many users could be affected.
3. Since I never saved any of the information (just viewed the pages) I no longer have it since the flaw was patched.
Nothing malicious was done.
1. I didn't disclose how to do it, merely that it was possible.
2. By "get" I in no way mean harvested. I just manually incremented the ID in the URL by hand in my web browser to see how many users could be affected.
3. Since I never saved any of the information (just viewed the pages) I no longer have it since the flaw was patched.
Nothing malicious was done.
Sorry, I think I misunderstood your comment. I thought you saved some info to yourself.
I'll update my comment above.
I'll update my comment above.
It's massively important that people bring security issues into the open. Talking about these things creates pressure on developers to build secure web apps. Not talking about them because people like you get 'upset' about "hurting 2 nerds' inspiration" means we get poor information security policies all over the Internet.
As it stands we don't know whether hostile agents (silently) got copies of the AWS keys of every user on the site. That should be incredibly concerning for you, but apparently it's not.
As it stands we don't know whether hostile agents (silently) got copies of the AWS keys of every user on the site. That should be incredibly concerning for you, but apparently it's not.
Thank you, yes I'm aware of these. I was trying to focus on the fact that the person who found this security hole used it to get some of this data to himself. But apparently I wasn't clear enough.
I'm still trying to improve my English. Next time I'll try to be more clear. Thank you.
I'm still trying to improve my English. Next time I'll try to be more clear. Thank you.
I'm sorry man, but this kind of mistake is unacceptable even for a prototype. This is basic authentication and access control stuff. You'd at least expect this to be implemented if you're giving them your email, needless to say amazon token.
The problem with this is that people become more skeptical in general about new products/companies, which is bad for the community as a whole.
The issue here is how do you deal with this kind of thing? Assuming it is in fact bad for others who are trying to get people to test their prototypes, how do we help avoid basic pitfalls like this one? This could be a service...
The problem with this is that people become more skeptical in general about new products/companies, which is bad for the community as a whole.
The issue here is how do you deal with this kind of thing? Assuming it is in fact bad for others who are trying to get people to test their prototypes, how do we help avoid basic pitfalls like this one? This could be a service...
Why are you upset with him, at least he informed people. You should be more upset with an insecure app. A less talkative person would just keep it and use the AWS accounts for bitmining.
Please read my comment again :)
You can find a security hole and report it. That's nice and good behavior. But when you say "I have compromised the data with me", then you are being evil.
I signed up to the app because I liked the idea and wanted to support them. I knew that their app was in alpha stage yet.
I'm upset about his behavior. I'm upset that his behavior might hurt 2 nerds' inspiration.
I'm not upset about he got my keys.
But you are right. His behavior is still better than not saying anything.
You can find a security hole and report it. That's nice and good behavior. But when you say "I have compromised the data with me", then you are being evil.
I signed up to the app because I liked the idea and wanted to support them. I knew that their app was in alpha stage yet.
I'm upset about his behavior. I'm upset that his behavior might hurt 2 nerds' inspiration.
I'm not upset about he got my keys.
But you are right. His behavior is still better than not saying anything.
> I'm upset about his behavior. I'm upset that his behavior might hurt 2 nerds' inspiration.
I'm hoping they're inspired to write secure code from line one.
I'm hoping they're inspired to write secure code from line one.
If anything, he did you a favor. You're probably going to create new keys now, whereas you might have written it off as a hypothetical vulnerability otherwise.
You're so right! Writing secure web apps is for pussies.
This sounds great, but since this is using your own AWS account, there is a very serious poison pill; Glacier only lets you retrieve 0.17% of your data for free per day. Beyond that, you get charged based on your peak hour of retrieval.
How much? $7.20 per gigabyte for your highest hour (minus a negligible free allowance if you're using it in this manner).
i.e. the cost to restore m gigabytes over n hours is:
$7.20 * m / n
I'm sure Ice Box Pro will have warnings in place, so nobody will get a $350 bill by accident by restoring a 50 gig account in an hour.
But for disaster recovery, the time will come to decide between a large bill or a very slow retrieval.
Pricing details are here (though quite difficult to follow): http://aws.amazon.com/glacier/faqs/#How_much_data_can_I_retr...
How much? $7.20 per gigabyte for your highest hour (minus a negligible free allowance if you're using it in this manner).
i.e. the cost to restore m gigabytes over n hours is:
$7.20 * m / n
I'm sure Ice Box Pro will have warnings in place, so nobody will get a $350 bill by accident by restoring a 50 gig account in an hour.
But for disaster recovery, the time will come to decide between a large bill or a very slow retrieval.
Pricing details are here (though quite difficult to follow): http://aws.amazon.com/glacier/faqs/#How_much_data_can_I_retr...
That seems too costly for even large businesses that don't care. When you lose data, you want it back fast. In this case, the faster you take it back the more it costs.
I wonder why they don't use S3. Is it not reliable enough? The charging infrastructure at least is simpler to figure out.
I wonder why they don't use S3. Is it not reliable enough? The charging infrastructure at least is simpler to figure out.
There is a huge market for data that needs to be retained indefinitely, or for some long period of time. Oftentimes this is related to compliance. The frequency that the data is accessed is incredibly small -- or even zero.
For example, in several states, all records relating to a minor in state custody, an adoption, or receiving certain nbenefits must be kept for 26 years after the minor turns 18. Today, states are either storing this data on tape, or paying some government contractor to do it for them -- at an expense several times that of Glacier.
Another example is litigation holds. One former employer was forced to hold around a petabyte of data 7 years for a complex civil suit, because... A judge said so. In that case, the high cost of retrieval may be a benefit, because the plaintiff would be footing the bill.
For example, in several states, all records relating to a minor in state custody, an adoption, or receiving certain nbenefits must be kept for 26 years after the minor turns 18. Today, states are either storing this data on tape, or paying some government contractor to do it for them -- at an expense several times that of Glacier.
Another example is litigation holds. One former employer was forced to hold around a petabyte of data 7 years for a complex civil suit, because... A judge said so. In that case, the high cost of retrieval may be a benefit, because the plaintiff would be footing the bill.
Sure, it's too costly for large businesses, but it's not too costly for me, I just want to restore my 100 GB photo/video collection if all other backups fail, I don't care if it takes two days as long as it's all there.
If they built something to warn you when you hit the 0.17% (or whatever the magic number is) it would be great, or just retrieve according to the limit for you.
To retrieve all your files at the free rate would take 588 days. Glacier is not a product geared towards consumers, and the free tier is basically negligible for this use case.
You're right that an upfront retrieval speed vs cost interface is the correct way to handle it, preferably on first use.
You're right that an upfront retrieval speed vs cost interface is the correct way to handle it, preferably on first use.
I'm interested in the service as a consumer. I keep backups of my important files in my apartment, so I consider data loss a very rare but catastrophic event. Glacier is a reasonably priced insurance for my important files.
This is exactly my use case, and I've written glacier-cli to integrate with git-annex in order to fulfil it.
Exactly, I am considering this for backup in case of any sort of catastrophic event. Fire, robbery, etc. Events where a bit of money for data retrieval will really be the least of my worries that day (Might it even be possible to get homeowners/renters insurance to cover retrieval costs? Something to consider maybe.)
Exactly, me too. I have multiple backups of my image and video files in my home (~250 Go, growing fast), and I rotate the disks in a vault in the basement. It's very very unlikely that I will lose any file, but it can happen if my whole house burns down or gets robbed (vault included). Glacier is relevant in that case for me as a consumer.
For family photos and unedited family videos I could probably actually wait that long in the even of a catestrophic loss of all my local copies.
I'd prefer the speed/cost interface to be at download time. Give the opportunity to reorder the download sequence to prioritise some parts and probably transfer to S3 so the local computer doesn't need to be running throughout the whole data trickle.
That's a useful service that I can't be bothered to build at the moment.
I'd prefer the speed/cost interface to be at download time. Give the opportunity to reorder the download sequence to prioritise some parts and probably transfer to S3 so the local computer doesn't need to be running throughout the whole data trickle.
That's a useful service that I can't be bothered to build at the moment.
Wow didn't realize it was that slow. I think you are right, it is for companies who don't mind paying if they have a data loss incident. It does support AWS import/export so one can fedex them harddrives if someone needs the data fast.
Also, if you are a company that does hourly backups and keeps them for a year, you can grab back 14 of them per day, which is plenty. That seems to be the canonical use case.
AWS free tier has always been designed as a "temprary free sample" not as a "free for small users" like AppEngine.
I haven't looked at the details but is it cheaper to add a massive amount of dummy data for the duration you need to download all your real content?
No - there is a 3 month minimum for data. You have to add 20 times as much data as you have to download for free over a month, 600 for a day, or 14400 for an hour. Multiplied by 3c a gig.
For an open source alternative (and my tool of choice) check out git-annex with s3[1]. joehy has a todo for git-annex and glacier[2] and someone has already submitted the beginings of a patch. When i first heard of Glacier I was excited about the possibilty of using it for a backend to git-annex, but then I read cpercival's discussion of glacier[3,4].
[1] http://git-annex.branchable.com/tips/using_Amazon_S3/
[2] http://git-annex.branchable.com/todo/special_remote_for_amaz...
[3] http://www.daemonology.net/blog/2012-09-04-why-tarsnap-doesn...
[4] http://www.daemonology.net/blog/2012-09-04-thoughts-on-glaci...
EDIT: Removed comment about competition.
[1] http://git-annex.branchable.com/tips/using_Amazon_S3/
[2] http://git-annex.branchable.com/todo/special_remote_for_amaz...
[3] http://www.daemonology.net/blog/2012-09-04-why-tarsnap-doesn...
[4] http://www.daemonology.net/blog/2012-09-04-thoughts-on-glaci...
EDIT: Removed comment about competition.
I've written glacier integration with git-annex and it's fully functional. Details in [2]. The Glacier retrieval pricing uncertainty just means that I store less in Glacier - but for critical data where the paid retrieval cost is worth it, I happily "git annex copy --to=glacier".
This isn't a Dropbox competitor, it uses Dropbox. It's just an addon.
If I understand correctly, Icebox is not a direct competitor to Dropbox. If anything, it enhances the Dropbox experience, so you could argue that it could help Dropbox get more users/customers.
hey guys,
i built this with a friend and didn't expect it to get posted to HN already. we just got it approved and i accidentally hit our Like button before we were even ready. my friends immediately commented on the story so we just said, what the hey :-)
in any case, it just "launched" today, was only meant for friends to test out, and yes i agree we need to write more copy.
to address some concerns:
- this is really only meant for developers at the moment; i.e. those who actually know their way around AWS. if and when we decide to take payments and provide a service, you can expect a lot more documentation and support. we have no desire to dupe normal folks. we built this for ourselves and really only expected friends to try it out first.
- you don't need to go to your AWS console to get your files. once a file archives, you can click a button to download it back to your Dropbox
- putting in warnings about the quota is a great idea! we'll do that ASAP.
keep the feedback coming, we really appreciate it and wanna build something useful for everyone.
i built this with a friend and didn't expect it to get posted to HN already. we just got it approved and i accidentally hit our Like button before we were even ready. my friends immediately commented on the story so we just said, what the hey :-)
in any case, it just "launched" today, was only meant for friends to test out, and yes i agree we need to write more copy.
to address some concerns:
- this is really only meant for developers at the moment; i.e. those who actually know their way around AWS. if and when we decide to take payments and provide a service, you can expect a lot more documentation and support. we have no desire to dupe normal folks. we built this for ourselves and really only expected friends to try it out first.
- you don't need to go to your AWS console to get your files. once a file archives, you can click a button to download it back to your Dropbox
- putting in warnings about the quota is a great idea! we'll do that ASAP.
keep the feedback coming, we really appreciate it and wanna build something useful for everyone.
Perhaps you could offer a service where everyone saves their files under your corporate account. When 1 person needs to restore, it will mostly fit within the free download % of the total. So if 10 people store 1 TB each, you can download 500GB/month for free. It's like insurance. We all pay a small amount to cover when one person has a problem.
we hope to get to that very soon!
most likely, we'll just keep a small data center. something to get around the bandwidth limitation and availability. we would definitely need to charge then in order to cover those costs.
most likely, we'll just keep a small data center. something to get around the bandwidth limitation and availability. we would definitely need to charge then in order to cover those costs.
What's not very clear to me is if my Glacier files are a subset of my Dropbox files, or a separate datastore.
I only have 2GB of Dropbox space because I never upgraded. But I have about 100GB of archive data I'd probably throw into Amazon Glacier if it were convenient.
Do I need to open a 100GB Dropbox account before using IceBox?
I only have 2GB of Dropbox space because I never upgraded. But I have about 100GB of archive data I'd probably throw into Amazon Glacier if it were convenient.
Do I need to open a 100GB Dropbox account before using IceBox?
What's not very clear to me is whether they have any idea what they're doing when it comes to security, or how to respond properly to a legitimate security hole. It's difficult to trust someone who builds a service like this and leaves securing personal keys as "something we'll get around to."
I'd like to see someone provide a service in a similar vein to this but just for photos.
What I'm thinking is a service that uses Glacier to store my photo library and some type of front end service (like dropbox) that keeps a low res version of that photo along with some meta data about it.
My reasoning for this is that we all build up pretty significant photo libraries (mines already over 60GB) and I'm always trying to make sure I have them backed up. I currently use a paid plan at Dropbox so I can put them all up there but it's kind of a waste since I hardly ever pull many of them down again. Every once in awhile I browse through them looking for certain pictures that I might need to get a copy of (which is why I'd need the low res copies easy to access/browse) and then be able to choose which ones to pull down from glacier. The other good thing about a service like that is the need is not typically immediate.
Maybe I'll look into building this since it's something I'd love to have for myself!
What I'm thinking is a service that uses Glacier to store my photo library and some type of front end service (like dropbox) that keeps a low res version of that photo along with some meta data about it.
My reasoning for this is that we all build up pretty significant photo libraries (mines already over 60GB) and I'm always trying to make sure I have them backed up. I currently use a paid plan at Dropbox so I can put them all up there but it's kind of a waste since I hardly ever pull many of them down again. Every once in awhile I browse through them looking for certain pictures that I might need to get a copy of (which is why I'd need the low res copies easy to access/browse) and then be able to choose which ones to pull down from glacier. The other good thing about a service like that is the need is not typically immediate.
Maybe I'll look into building this since it's something I'd love to have for myself!
I had this _exact_ idea when Glacier came out, and even registered a domain with a splash page. Still thinking about building it but lacking time. I was looking at specifically storing RAW files for photographers and generating low-res versions to show in some sort of library.
Had exactly the same idea myself. Whoever makes it work first is probably onto a winner (I probably won't pursue it myself due to lack of time, but I'd love to use such a service).
Have you heard of https://openphoto.me/ ? It is open source and might be extendable into what you want.
That does sound like a good idea! Sounds perfect for S3, can just store the low-res/thumbnails in S3 for browsing via a web page, then retrieve an occasional photo if necessary through glacier.
Have a look at The OpenPhoto Project.
Hosted version: https://openphoto.me
Project page: http://theopenphotoproject.org
Source: https://github.com/photo
Hosted version: https://openphoto.me
Project page: http://theopenphotoproject.org
Source: https://github.com/photo
soon... :-)
akmiller, can we talk offline? i'd love to hear more about your ideas. (we actually have photos in mind.)
just email [email protected]
akmiller, can we talk offline? i'd love to hear more about your ideas. (we actually have photos in mind.)
just email [email protected]
I use Backblaze--$50/year--primarily for photos. My photo drive sits at around 900GB currently. It's entirely in Backblaze, in case it ever explodes and local backups fail.
I let it handle other drives, too, although I exclude my media/music/download/etc drive for simplicity.
I let it handle other drives, too, although I exclude my media/music/download/etc drive for simplicity.
Is it encrypted on the client side? That's pretty much my only requirement now, since I have separated my two use cases (backups and syncing). I use encfs on Dropbox for encrypted syncing, and it works great, but I need backups encrypted on the client, and I haven't found anything great and cheap yet.
What do you mean by "backups encrypted on the client?"
I would guess he means that the data is encrypted before ever being sent to backblaze. The answer is yes according to http://www.backblaze.com/backup-encryption.html
Rather, it's an optional yes.
Rather, it's an optional yes.
Yep, that's what I meant, thanks.
EDIT: Actually, it looks like they can decrypt your data to restore it on their end, so it's a no.
EDIT: Actually, it looks like they can decrypt your data to restore it on their end, so it's a no.
[deleted]
Actually it is a yes, they cannot decrypt your data as they do not keep your password around. You have to enter it every time you want to restore a file, and it is not saved on their side only used to get the files then thrown away again.
Of course, if you have so sensitive stuff that you cannot accept it ever being on another server in clear text then it still a no on that question.
Of course, if you have so sensitive stuff that you cannot accept it ever being on another server in clear text then it still a no on that question.
[deleted]
I believe Tarsnap is your only viable option, if you want that enforce that degree of paranoia: http://www.tarsnap.com/
Bitcasa is also in the same realm, although I think Bitcasa has some cooler de-dupe technology.
Another good option for photo backup is to sign up for a flickr pro account. $45 for 2 years unlimited storage.
yes, I think photos / videos that you create are the remaining challenge, haven't found a great solution for backing it all up at a low cost.
Have a look at The OpenPhoto Project
Hosted version: https://openphoto.me
Project page: http://theopenphotoproject.org
Source: https://github.com/photo
Hosted version: https://openphoto.me
Project page: http://theopenphotoproject.org
Source: https://github.com/photo
I have actually been toying with building out something like this as well. My wife is a photographer and building something around dropbox and glacier only made sense.
A year ahead of you :). Pull requests!
Hosted version: https://openphoto.me
Project page: http://theopenphotoproject.org
Source: https://github.com/photo
Hosted version: https://openphoto.me
Project page: http://theopenphotoproject.org
Source: https://github.com/photo
I'd be interested in seeing a service like this but for professional photographers. A friend of a friend is a wedding photographer. The work flow for something like this is to have multiple shooters each taking ~25GB worth of photos for one shoot. Then they process the worthy photos, making 3 large files per photo (RAW, .psd, and a final .png). During the turnaround time between wedding and delivery, it would be ideal to keep a copy of each of these files somewhere, but at least keeping the RAW somewhere safe is a minimum requirement. We were trying to figure out a feasible backup solution for the volume of data that such a shoot would require. The time to upload this data is longer than the delivery time for the photos, given the current available upstream bandwidth.
We came to the conclusion that putting files on BluRay disks or hard drives and keeping them physically separate was the best solution for the cost. The problem is that disks take a long time to burn, and external hard drives aren't the best archival medium (not to mention they don't always travel well).
Has anyone here solved the problem of data backups where bandwidth limits make pure online/cloud storage infeasible?
We came to the conclusion that putting files on BluRay disks or hard drives and keeping them physically separate was the best solution for the cost. The problem is that disks take a long time to burn, and external hard drives aren't the best archival medium (not to mention they don't always travel well).
Has anyone here solved the problem of data backups where bandwidth limits make pure online/cloud storage infeasible?
Have a look at The OpenPhoto Project. It's an OpenPhoto photo platform you can sign up to use or install yourself. Works with S3, Dropbox, Box.net, local filesystem or anything else you want to write a little code for.
Hosted version: https://openphoto.me
Project page: http://theopenphotoproject.org
Source: https://github.com/photo
Hosted version: https://openphoto.me
Project page: http://theopenphotoproject.org
Source: https://github.com/photo
Interesting, but we run into the same issues as before. We are either storing locally (backups are good, but local storage doesn't have quite as much redundancy as a data center), or taking infeasible amounts of time to upload to a remote backup.
Ideally, we would be able to ship BluRays to a data center (USPS has terrible latency, but great bandwidth!), where they would load them into their servers. If anything happend to the local copies, we could either have them ship us back a hard drive or BluRays, or download the lost files (downstream is much faster for us, and because we are theoretically only downloading a subset of what we backed up, direct downloads are feasible).
The issue I run in to in my thought experiments is that a data center doesn't necessarily want to physically handle data shipped to them or create recovery media to ship back.
Actually, all that is really needed is a secure place to archive the BluRays. A way to view and download backed up files is a nice-to-have, but not required.
Ideally, we would be able to ship BluRays to a data center (USPS has terrible latency, but great bandwidth!), where they would load them into their servers. If anything happend to the local copies, we could either have them ship us back a hard drive or BluRays, or download the lost files (downstream is much faster for us, and because we are theoretically only downloading a subset of what we backed up, direct downloads are feasible).
The issue I run in to in my thought experiments is that a data center doesn't necessarily want to physically handle data shipped to them or create recovery media to ship back.
Actually, all that is really needed is a secure place to archive the BluRays. A way to view and download backed up files is a nice-to-have, but not required.
Backblaze should provide an option for you to ship them a hdd or blu ray disk. They can store those however you want but if/when you need it you can restore files over the Internet or they can ship you physical media.
Probably not cost effective for them but that's what you're looking for, I think.
Probably not cost effective for them but that's what you're looking for, I think.
I know site like carbonate let you mail them a hard drive, but they're aimed at the full computer restore demographic.
> Ideally, we would be able to ship BluRays to a data cente
Amazon will accept delivery of hard disks and load the stored media in to S3/Glacier:
http://aws.amazon.com/importexport/
Amazon will accept delivery of hard disks and load the stored media in to S3/Glacier:
http://aws.amazon.com/importexport/
Pull requests welcome :)
The OpenPhoto Project is what you're looking for. Open source, installable and offered as a hosted service.
Hosted version: https://openphoto.me
Project page: http://theopenphotoproject.org
Source: https://github.com/photo
The OpenPhoto Project is what you're looking for. Open source, installable and offered as a hosted service.
Hosted version: https://openphoto.me
Project page: http://theopenphotoproject.org
Source: https://github.com/photo
Anything that uses Dropbox as the middleman scares me a bit. Dropbox scares me a bit. It silently ignores a lot of meta data , restores files with dates that aren't what the original was ( last I checked ) and has corrupted a few Mac OS X sparse bundles beyond repair.
There's a great app called Arc that handles all of Mac OS X meta data perfectly, works with the relative ease of Dropbox, and backs up to S3 at great savings. De-duplication and other "smarts" are all there.
It's sort of like a Time Machine that you can point to S3.
There's a great app called Arc that handles all of Mac OS X meta data perfectly, works with the relative ease of Dropbox, and backs up to S3 at great savings. De-duplication and other "smarts" are all there.
It's sort of like a Time Machine that you can point to S3.
I love the idea, but besides the retrieval fees involved that you're not disclosing, I wonder if people will grasp the fact that the files they are putting into the icebox folder are not reachable with opening the same folder and looking into it—instead you need to go to a web site to find them. This is rather disorienting for a non-technical user as the way of putting files in gives a false sense of immediacy to a future retrieval availability.
The security problem that was found (now fixed) is less of an issue than the responses they gave afterwards: http://news.ycombinator.com/item?id=4619652
Wow, I've been working on almost the exact same thing!
Except I'm charging a yearly fee instead of using the users own AWS account, this makes it easier for users without an AWS account and also means you don't have to worry about the crazy Glacier fee structure.
Try it out if you want, 1GB accounts are free at the moment:
https://www.tidy.io/
Try it out if you want, 1GB accounts are free at the moment:
https://www.tidy.io/
Some feedback about the landing page:
It doesn't show any pricing. I don't want to sign up for something just to see the pricing.
It doesn't show any pricing. I don't want to sign up for something just to see the pricing.
Thanks for the feedback!
The landing page not being finished was why I'd been waiting to launch, wish I'd got it done sooner now!
EDIT: Until I sort out the landing page here's the current pricing: 50GB is $25/year, 100GB is $40/year and 500GB is $189
The landing page not being finished was why I'd been waiting to launch, wish I'd got it done sooner now!
EDIT: Until I sort out the landing page here's the current pricing: 50GB is $25/year, 100GB is $40/year and 500GB is $189
[deleted]
There's this nefarious sense of: Give me your data so I can sell it back to you!
Don't give your data to this site until they fix this:
http://blog.ryankearney.com/2012/10/never-give-your-informat...
http://blog.ryankearney.com/2012/10/never-give-your-informat...
And even when it's fixed after the way they responded, don't trust them with your data or anything at all.
It's already fixed.
This is a little OT, but I was thinking about putting together a weekend (read: week) project that would offer a cloud service (details aren't important) but require a user to add an AWS key. The idea is that it would use the free micro instance Amazon gives a user.
The benefit I see is that customers would have total visibility into their own data, and, of course, it's very cheap to run a system like that.
I'd love to hear any opinions anybody might have about it.
The benefit I see is that customers would have total visibility into their own data, and, of course, it's very cheap to run a system like that.
I'd love to hear any opinions anybody might have about it.
That's a subset of what we're trying to do with The OpenPhoto Project. http://theopenphotoproject.org
Trying it now (800mb is uploading).
This tool is great for me because the data I'm backing up is something I'll need to access maybe twice a year, or less. But I still need to have this data backed up somewhere.
I love how it's as simple as Dropbox. Set it up once, the rest is just drag and drop.
This tool is great for me because the data I'm backing up is something I'll need to access maybe twice a year, or less. But I still need to have this data backed up somewhere.
I love how it's as simple as Dropbox. Set it up once, the rest is just drag and drop.
My Dropbox isn’t big enough for the file I would want to send to Glacier.
Does Amazon provide a fairly decent interface? Or, failing that, are there clients that can work with AWS Glacier already, the way some FTP clients can speak to AWS S3?
Does Amazon provide a fairly decent interface? Or, failing that, are there clients that can work with AWS Glacier already, the way some FTP clients can speak to AWS S3?
I actually thought about it some time ago and even started working on a prototype, got dropjar.com for that, but after seeing how low companies like backblaze sell unlimited backup for ... I figured it's a hard sell.
Good luck with icebox.
Good luck with icebox.
[deleted]
Great Idea, although to realy archive terabytes with a relatively small dropbox account you'd have to do it in 10-100 steps.. Recovering it via dropbox would be even more challenging.
I'm not sure you can use competitors' logos like that.
They're not competitiors, they're augmenting Dropbox.
From the Dropbox brading guide ( https://www.dropbox.com/developers/reference/branding ):
> you can use the Dropbox logo in marketing material to show your app's Dropbox integration, but it should never be modified and should always refer to our service. If you are using the logo on a website, it should link to https://www.dropbox.com.
So they're allowed to use it, but it must link to the Dropbox website. It doesn't so it's actually violating the guidelines.
From the Dropbox brading guide ( https://www.dropbox.com/developers/reference/branding ):
> you can use the Dropbox logo in marketing material to show your app's Dropbox integration, but it should never be modified and should always refer to our service. If you are using the logo on a website, it should link to https://www.dropbox.com.
So they're allowed to use it, but it must link to the Dropbox website. It doesn't so it's actually violating the guidelines.
all Dropbox apps require approval. someone from Dropbox actually signed up, reviewed IceBox, and approved it.
we have no intentions of competing with Dropbox. (like that's even possible.) if we get a complaint, we're more than happy to comply.
we love both Dropbox and Amazon Glacier so we decided to pair them. i'm almost certain Dropbox will eventually add a similar feature soon--we may just be beating them to the punch.
we have no intentions of competing with Dropbox. (like that's even possible.) if we get a complaint, we're more than happy to comply.
we love both Dropbox and Amazon Glacier so we decided to pair them. i'm almost certain Dropbox will eventually add a similar feature soon--we may just be beating them to the punch.
You might want to make that logo a link to Dropbox since it's required in their guidelines.
https://www.dropbox.com/developers/reference/branding
https://www.dropbox.com/developers/reference/branding
Is the data stored on Glacier encrypted?
They're encrypted, but nothing's secure if it leaves your computer in plaintext.
The games on Stripe's CTF were more secure than this site...
EDIT: Looks like it was just patched. Still managed to get a few dozen AWS keys though.