How Dropbox Knows When You’re Sharing Copyrighted Stuff Without Actually Looking(techcrunch.com)
techcrunch.com
How Dropbox Knows When You’re Sharing Copyrighted Stuff Without Actually Looking
http://techcrunch.com/2014/03/30/how-dropbox-knows-when-youre-sharing-copyrighted-stuff-without-actually-looking-at-your-stuff
12 comments
How Turtl has no idea when you’re sharing copyrighted stuff: http://turtlapp.tumblr.com/post/81222024691/how-turtl-has-no...
No new info here. Dropbox de-duplicates using hashes, so they would use them against a blacklist. The main problem with Dropbox still remains: They can read your files if they want to and you have only their word that they don't.
For now I'm going with Bittorrent sync [1], at least until an equivalent service with an open source comes along. (Would be nice if they open sourced theirs, though.)
[1] http://www.bittorrent.com/sync
For now I'm going with Bittorrent sync [1], at least until an equivalent service with an open source comes along. (Would be nice if they open sourced theirs, though.)
[1] http://www.bittorrent.com/sync
I couldn't get BitTorrent sync working without using their tracker (even restricting it to LAN-only it couldn't find my phone on the same network). The application gives very little debugging info, which doesn't help me trust it either.
I'll be limiting my use of it and hope that ClearSkies goes well.
I'll be limiting my use of it and hope that ClearSkies goes well.
Not open source, and I haven't tried it, but I've heard ads for http://www.filetransporter.com which may be an interesting solution for you.
They provide discovery, syncing & NAT traversal to a hard drive that you own.
They provide discovery, syncing & NAT traversal to a hard drive that you own.
Clearskies is the one to keep an eye on.
https://github.com/jewel/clearskies
https://github.com/jewel/clearskies
Syncthing seems farther along. Been planning to test both out:
https://github.com/calmh/syncthing
git-annex with the assistant was supposed to be a dropbox replacement too but I haven't had much luck with it.
https://github.com/calmh/syncthing
git-annex with the assistant was supposed to be a dropbox replacement too but I haven't had much luck with it.
I don't use git-annex because it doesn't work on homefolders under OSX, making it sort of useless
It has a bunch of limitations. It doesn't support hardlinks, it doesn't track folders so empty folders don't show up/disappear, it doesn't sync file permissions, etc.
The developer is awesome but he seems to be going at this from "how can I make git more userfriendly" instead of "how can I implement a dropbox that just happens to use git". clearskies/btsync/syncthing won't inherit all the great git stuff (great versioning mostly) but they seem to be more on track to give a good user experience.
The developer is awesome but he seems to be going at this from "how can I make git more userfriendly" instead of "how can I implement a dropbox that just happens to use git". clearskies/btsync/syncthing won't inherit all the great git stuff (great versioning mostly) but they seem to be more on track to give a good user experience.
Also, SpiderOak.
I use SpiderOak, but they can still read your files, they hold your keys.
Something like tarsnap is the only paranoid-proof choice. It's not even distributed as a binary file.
https://www.tarsnap.com/
Something like tarsnap is the only paranoid-proof choice. It's not even distributed as a binary file.
https://www.tarsnap.com/
They cannot read your files, only your password can be used to decrypt them, which is not even saved on their server.
https://spideroak.com/zero-knowledge/
https://spideroak.com/engineering_matters
https://spideroak.com/zero-knowledge/
https://spideroak.com/engineering_matters
... or so they say. I do happen to trust them (and use them), but they absolutely could read your files if they want to: all they have to do is release an updated version of the sync app that does save your password when you first sign in. Or they could start saving passwords when people sign in via their website.
Now, I don't believe that they'd do this maliciously (rogue employees aside), but there's always the possibility that a government agency could secretly require them to actively try to capture a particular user's password the next time they sign in.
Now, I don't believe that they'd do this maliciously (rogue employees aside), but there's always the possibility that a government agency could secretly require them to actively try to capture a particular user's password the next time they sign in.
If you enter your password into a system that they control at any time, they have potential access to your files and can be ordered by the government to change their code so that it does collect your password in order to enable decryption of your files.
and can be ordered by the government to change their code so that it does collect your password in order to enable decryption of your files.
And they can be ordered to do so secretly with no notice, neither to company management nor to you nor to any judicial body that could review the decision.
That's the lesson of Lavabit.
And they can be ordered to do so secretly with no notice, neither to company management nor to you nor to any judicial body that could review the decision.
That's the lesson of Lavabit.
I like the concept of SpiderOak and have used them before. However, their software was way too buggy and would stop syncing / working randomly for no apparent reason and their support was unable to help.
How do they compute a hash of a file without actually looking at the file? Magic?
'looking' here means having a person look at the file or possibly having a system run analysis* of the file. Neither happens.
Since you seem to be talking about having their systems touch any bytes of the file, I have a more important question: How would you expect to have a file synced with dropbox and shared via dropbox without dropbox touching the bytes? Magic?
* you know what I mean by analysis, think gmail
Since you seem to be talking about having their systems touch any bytes of the file, I have a more important question: How would you expect to have a file synced with dropbox and shared via dropbox without dropbox touching the bytes? Magic?
* you know what I mean by analysis, think gmail
> 'looking' here means having a person look at the file or possibly having a system run analysis* of the file. Neither happens.
I know that, and you know that, but I showed this article to my wife (who is a casual computer user), and while she understood the difference when explained, she said that as a non-techie, she interpreted the title differently. If you hash my data, and take action based on that hash, then you have looked at the data, however cursory or algorithmic it may have been.
FWIW, I had this same discussion on reddit, where it devolved instantly into name-calling. HN is much more civil. Thanks for that.
I know that, and you know that, but I showed this article to my wife (who is a casual computer user), and while she understood the difference when explained, she said that as a non-techie, she interpreted the title differently. If you hash my data, and take action based on that hash, then you have looked at the data, however cursory or algorithmic it may have been.
FWIW, I had this same discussion on reddit, where it devolved instantly into name-calling. HN is much more civil. Thanks for that.
Did she realize that this was a file already uploaded to Dropbox? That's a very important distinction in my mind. Dropbox has already calculated hashes and fed the data to the servers at that point. They don't don't do any 'looking' beyond the data-shoveling process.
It's the DMCA sharing check that avoids 'looking' even in the most pedantic sense, not the overall use of Dropbox.
It's the DMCA sharing check that avoids 'looking' even in the most pedantic sense, not the overall use of Dropbox.
Simple. Truecrypt (or whatever other encrypted filesystem you prefer) volume on Dropbox. Files in partition.
Dropbox has no idea what is on the volume.
(In an ideal world, you'd have a filesystem set up that was a separate encrypted file for each file in the virtual filesystem, with an additional encrypted file that contained the virtual directory info. But this is simpler.)
Dropbox has no idea what is on the volume.
(In an ideal world, you'd have a filesystem set up that was a separate encrypted file for each file in the virtual filesystem, with an additional encrypted file that contained the virtual directory info. But this is simpler.)
Well sure if you use another program to encrypt the file then Dropbox can only see the encrypted bytes, but that's not what mbrutsch was objecting to. A particular truecrypt volume could still have a takedown related to it.
Not automatically via the method used here (hash blacklist)
When I say 'particular' I mean someone notices the truecrypt volume being shared either semi-publicly or multiple times and files a DMCA for the volume itself. Then you have to alter it or make a new one.
How would the file get to the truecrypt volume? Wouldn't it be significantly more expensive to run the app on encrypted hdd?
[deleted]
Simple! By redefining "not looking" to mean "totally looking, but in a way that we've decided doesn't count."
It's also a method dropbox can use to save storage space. Inevitably, two or more users will put the same file in their DB. By computing and comparing hashes they can simply just store one copy of the file and point all users to that single instance of the file (they may do an exact match check too once the hashes collide since hashes can have unintended collisions as well).
I have no idea how much efficiency this buys them, but the same mechanism can be used against a blacklist of known copyright or illegal materials. In this case, having a hash that matches something in the copyrighted materials list AND sharing it triggers this action.
Just as easily, putting some child porn in your DB could also have them check against illegal material hashes and trigger an automated notice to some law enforcement agency or something similar.
This can even be used for National Security without violating the classified contents of the documents. Say another Snowden-type grabs a bunch of classified documents. The agency could give DB a list of hashes with instructions to "call us" if a user uploads a bunch of leaked documents to their DB. They could even do this blind, just send DB a hash of every classified document or piece of data they ever produce and catch leakers before anybody in the agency even knew something was leaked internally.
I have no idea how much efficiency this buys them, but the same mechanism can be used against a blacklist of known copyright or illegal materials. In this case, having a hash that matches something in the copyrighted materials list AND sharing it triggers this action.
Just as easily, putting some child porn in your DB could also have them check against illegal material hashes and trigger an automated notice to some law enforcement agency or something similar.
This can even be used for National Security without violating the classified contents of the documents. Say another Snowden-type grabs a bunch of classified documents. The agency could give DB a list of hashes with instructions to "call us" if a user uploads a bunch of leaked documents to their DB. They could even do this blind, just send DB a hash of every classified document or piece of data they ever produce and catch leakers before anybody in the agency even knew something was leaked internally.
Say a company, with its 1000 1TB/user accounts, has ~600GB/user in real life usage, since there's going to be a lot of cross over they could save upwards of .5PB.
The trouble with using hashes is that they have no context. Suppose I own the copyright on some file and issue a DMCA takedown against somebody else sharing it with the world. In that context the takedown is totally legitimate. But now the hash is on the list, so what happens next time when the file is being shared in some totally different context? What happens when the copyright holder wants to share the file? What happens when some other third party wants to share the file with a small number of people in a way which is unambiguously fair use?
You end up over-blocking, unless you include some option to bypass the block, in which case why even bother? It's not like you could inform the copyright holder when someone shares the file without invading users' privacy and opening up the system to abuse against political speech.
The whole thing is a mess because the fundamental problem is that the cost of most infringement on the internet is less than the cost of evaluating whether that use is infringing, so you end up with a high error rate no matter what you do. It breaks the founding assumption that copyright is not in tension with the First Amendment and makes you choose which you want to sacrifice for the other.
You end up over-blocking, unless you include some option to bypass the block, in which case why even bother? It's not like you could inform the copyright holder when someone shares the file without invading users' privacy and opening up the system to abuse against political speech.
The whole thing is a mess because the fundamental problem is that the cost of most infringement on the internet is less than the cost of evaluating whether that use is infringing, so you end up with a high error rate no matter what you do. It breaks the founding assumption that copyright is not in tension with the First Amendment and makes you choose which you want to sacrifice for the other.
This is a good time to note that ClearSkies [1], the open source BTSync alternative, is having a fundraiser: https://www.bountysource.com/teams/clearskies/fundraiser
1: https://github.com/jewel/clearskies
1: https://github.com/jewel/clearskies
[deleted]
What a bunch of doublespeak bullshit. "We're not looking at your stuff, we're only computing a hash of your stuff and comparing it to a blacklist!" Right up there with "we're not recording your phone calls, we're only recording your phone call metadata."
I wish more posts had great summaries like that at the start.
[deleted]
doh
EDIT: Sorry, the moral is probably avoid copyright violations. The advice above is more like the practical workaround. It's nonetheless useful as a way to avoid overreach from rights holders in cases of false positives or suppression of fair use.