Secure Messaging Scorecard(eff.org)
eff.org
Secure Messaging Scorecard
https://www.eff.org/secure-messaging-scorecard
58 comments
I just want to mention a highly underrated/undervalued program in this arena: Murmur/Mumble.
It's open source, easily self-hostable, and both chat and VOIP are encrypted, vis TLS and OCB-AES respectively. The downside is that it is not end-to-end, and a compromised server would compromise communications, which is why I like to self host on something I've hardened myself as opposed to buying the service from somewhere else.
Also, it can also be pushed over TOR if you want...
I highly encourage anyone who uses teamspeak, ventrillo, etc, to switch to mumble. I use it to keep up with friends and family from all over the world.
It's open source, easily self-hostable, and both chat and VOIP are encrypted, vis TLS and OCB-AES respectively. The downside is that it is not end-to-end, and a compromised server would compromise communications, which is why I like to self host on something I've hardened myself as opposed to buying the service from somewhere else.
Also, it can also be pushed over TOR if you want...
I highly encourage anyone who uses teamspeak, ventrillo, etc, to switch to mumble. I use it to keep up with friends and family from all over the world.
I wonder why they didn't include telegram?
It's there if you select "All Tools" instead of "Featured Tools" in the dropdown.
Thanks!
(Why isn't Telegram featured?)
(Why isn't Telegram featured?)
Telegram? They buried PGP.
A blog post on the protocol built by Telegram: http://unhandledexpression.com/2013/12/17/telegram-stand-bac...
sigh
sigh
Side rant: I really hate that people conflate expertise in math with expertise in crypto. Dozens of completely useless cryptosystems are published every year by talented mathematicians.
It's a good question. Would be nice if the page could've details what makes the criteria for being featured.
Google Hangouts, iMessage, fb Chat "Audited"? Wish they could point readers to those audits..
Skype "encrypted so the provider can't read it" even though MS is a known NSA PRISM partner? Recent disclosures quite strongly suggest otherwise.
Also weird that they didn't sort these options by ranking. it took me a while to realize that they weren't displaying everything ranked by default (or the "winners"). to see that, choose 'all tools' from the drop-down.
Skype "encrypted so the provider can't read it" even though MS is a known NSA PRISM partner? Recent disclosures quite strongly suggest otherwise.
Also weird that they didn't sort these options by ranking. it took me a while to realize that they weren't displaying everything ranked by default (or the "winners"). to see that, choose 'all tools' from the drop-down.
Confirm: Skype doesn't deserve that second checkbox. It is now centralised, is a specific real-time PRISM source, including full chat logs, and does indeed read the messages going through it.
Try posting a URL on there and watching an anti-malware scanner ping it sometime. I don't know if they're still doing that, but they definitely were.
Try posting a URL on there and watching an anti-malware scanner ping it sometime. I don't know if they're still doing that, but they definitely were.
It deserves it as much as iMessage, doesn't it?
Which means neither deserves it.
I have not analysed iMessage, but I don't believe it takes positive steps to deter reverse-engineering/auditing like Skype.
?
My understanding of iMessage is end to end encryption. Where the endpoints are the separate certificates on each of your devices.
My understanding of iMessage is end to end encryption. Where the endpoints are the separate certificates on each of your devices.
AFAIK, iMessage encrypts the message on your device. But Apple sends the list of keys to encrypt to. If that list is accurate and only includes the public keys of the target user, fine. But you've no way to verify this. I don't see how Skype is any worse.
Yeah, there are several apps that check every box, but aren't displayed in the 'featured' list.
CryptoCat, the very same client that had an abysmal MitM attack exposed in which the author said "don't worry, it's on a prototype debug version" when infact it was on the deployed real-world version?
Yeah, nah.
https://news.ycombinator.com/item?id=7518761
EDIT: I get that they're choosing more "user-friendly" and "freely available" applications for featured, however this is really only going to get seen by people who don't mind a bit of fiddling around. ChatSecure + Orbot I see as the most reliable on there, yes you'd stick out like a sore thumb (why are you using Tor?) however privacy-wise, you'll be just fine.
Yeah, nah.
https://news.ycombinator.com/item?id=7518761
EDIT: I get that they're choosing more "user-friendly" and "freely available" applications for featured, however this is really only going to get seen by people who don't mind a bit of fiddling around. ChatSecure + Orbot I see as the most reliable on there, yes you'd stick out like a sore thumb (why are you using Tor?) however privacy-wise, you'll be just fine.
What's more, Cryptocat is prominently featured with a perfect score ("score" being their word) above the fold, with TextSecure at the bottom of the list. Alphabetization would be a credible explanation for this if EFF hadn't already made the editorial decision to hide many many more applications behind a "show all applications" dropdown.
I think they may have changed this, as it now defaults to showing all applications and then featured applications is select-able. It's also alphabetized. I doubt there was some nefarious reasoning, but I could be wrong.
What was the logic behind what they chose to feature? They buried PGP, the most important secure messaging tool on the Internet, but had Cryptocat at the top of the list with a perfect score.
I don't like words like "nefarious".
I don't like words like "nefarious".
You are correct, however how they've displayed the page still doesn't sit right with me. There are a lot of unrealistic points being made with it's current form.
Sorry to the EFF guys reading this, I understand what you wanted to do however the execution wasn't perfect so we're nitpicking. News.YC crowd is a finicky bunch.
Sorry to the EFF guys reading this, I understand what you wanted to do however the execution wasn't perfect so we're nitpicking. News.YC crowd is a finicky bunch.
For the record, we aren't endorsing CryptoCat or any of the other tools that got 7/7: ChatSecure, TextSecure, Signal/RedPhone, SilentText, SilentPhone, or the ones that are close like Pidgin+OTR, Subrosa, Surespot, Telegram, Threema or iMessage.
Getting those scores is a sign that those projects are taking the right approach. Lots of codebases have horrific bugs, including OpenSSL, older versions of the SSL and TLS protocol itself. We believe that focusing the community on the task of moving the best projects forward is more constructive than
Testing the tools that are scoring highest for usability, and doing deeper examinations of their designs and codebases, is going to be a future component of this campaign.
Getting those scores is a sign that those projects are taking the right approach. Lots of codebases have horrific bugs, including OpenSSL, older versions of the SSL and TLS protocol itself. We believe that focusing the community on the task of moving the best projects forward is more constructive than
Testing the tools that are scoring highest for usability, and doing deeper examinations of their designs and codebases, is going to be a future component of this campaign.
That sounds like a good start - but I still think the way you're presenting this data is somewhat misleading. I agree that many codebases have horrific bugs, and ultimately exploits will eventually come to light in existing applications.. However instead of listing many different available applications, why not just list what works for "right now"?
For instance: "Encrypted so the provider can’t read it?" on Skype's messaging just isn't true. If a subpoena was issued to Microsoft for conversation data, it'd be available.
I think it'd be better to stand up what does things right, and for everything else say why you're not listing them as effective. Approach is all well and good, but to an anonymous journalist/source being pursued a groups - approach over execution could be the difference between freedom and imprisonment (or worse).
For instance: "Encrypted so the provider can’t read it?" on Skype's messaging just isn't true. If a subpoena was issued to Microsoft for conversation data, it'd be available.
I think it'd be better to stand up what does things right, and for everything else say why you're not listing them as effective. Approach is all well and good, but to an anonymous journalist/source being pursued a groups - approach over execution could be the difference between freedom and imprisonment (or worse).
Sorry, I tried to reply here yesterday but was rate-limited out of the conversation :/
Our aim with this project is to not give advice about what works "right now", because we aren't convinced there are any secure messaging options right now, especially when the usability dimensions of security are taken seriously.
Instead, what we're trying to do is articulate the things that both large companies and open source projects need to be doing to move in the right direction.
Since this is phase 1 of a multi-part campaign, we're going to take a closer look at the usability and further security properties of tools that are doing well on the Scorecard in subsequent phases.
On Skype, before launch it wasn't clear clear to us whether the NSA's reported Skype intercept capability came from breaking or having Microsoft backdoor the crypto (which would mean they loose the second checkmark) or by having Microsoft hand out a false public key for the other party (which is possible due to the lack of a check mark in the third column). We have an ongoing conversation with Microsoft about this and are reviewing Skype's ratings at the moment.
Our aim with this project is to not give advice about what works "right now", because we aren't convinced there are any secure messaging options right now, especially when the usability dimensions of security are taken seriously.
Instead, what we're trying to do is articulate the things that both large companies and open source projects need to be doing to move in the right direction.
Since this is phase 1 of a multi-part campaign, we're going to take a closer look at the usability and further security properties of tools that are doing well on the Scorecard in subsequent phases.
On Skype, before launch it wasn't clear clear to us whether the NSA's reported Skype intercept capability came from breaking or having Microsoft backdoor the crypto (which would mean they loose the second checkmark) or by having Microsoft hand out a false public key for the other party (which is possible due to the lack of a check mark in the third column). We have an ongoing conversation with Microsoft about this and are reviewing Skype's ratings at the moment.
Oh I see, so the intended outcome is to raise awareness with the general public and shed light on issues with existing software they use in hopes that the powers at be take notice?
You may not intend to endorse CryptoCat, but it sure looks like it.
You should provide a simplified rating or some end user friendly explaination that is hard to ignore. It's confusing and misleading to say something is secure but there's no way to e.g. verify keys.
You should provide a simplified rating or some end user friendly explaination that is hard to ignore. It's confusing and misleading to say something is secure but there's no way to e.g. verify keys.
To be fair to the article (if not to Cryptocat), the EFF page doesn't give an analysis of the quality of the encryption. The code has been audited, but it doesn't mean the results were satisfactory.
That said, the fact that the page gives bright green happy checkmarks across the board for Cryptocat does become misleading for people who may not know a ton about it.
That said, the fact that the page gives bright green happy checkmarks across the board for Cryptocat does become misleading for people who may not know a ton about it.
So it's like a Mitch Hedburg joke? "Yeah this program has been audited by the world's best cryptographers. They said not to use it. " The EFF is doing everyone a disservice by taking such a stance.
Reading about the history of CryptoCat doesn't convince me that it was written by people who knew what they were doing.
Look at the RSA key size chronology: 768, 512, 600, 1280, 1024, 1048, ...
http://tobtu.com/decryptocat.php
How good these are from a security point of view seems (roughly) inversely proportional to their popularity.
Which begs the question, in what direction is the causation?
Are less secure apps more popular because they're a better user experience? Are popular apps less likely to be secure because they're owned by large companies less inclined to give up precious access to information? A bit of both? Or something else entirely?
Answering this question is the first step to making truly secure communication actually USED in day to day life.
I think it's a question of focus. The more secure apps are a product of focus on security. The less secure ones are a product of focus on marketing and user experience.
That's why we view this Scorecard as the opening in a campaign to evaluate both the usability and security of messaging tools. Traditional cipherpunk designs are unusable by humans, and a lot more work is going to be needed to build things that can have hundreds of millions of users and be secure at the same time.
Thanks to the EFF for doing this!
The design of this page is so bad! ahaha
Anyway, helpful tip: you can chose "All Tools" instead of "Featured tools" on the top left of the beautiful table
Anyway, helpful tip: you can chose "All Tools" instead of "Featured tools" on the top left of the beautiful table
This is definitely a moment of weakness for EFF's credibility. They need experts to review these before going public. Many of these are demonstrably false.
We do have experts reviewing them. Which items are false?
Last I knew it was strongly suspected that Skype could look at your messages. imessage doesn't pass the mud puddle test indicating apple can look at your messages. Facetime should probably considered suspect but I don't know of any articles that demonstrate how the key exchange is handled. BlackBerry also modified their messaging app to be able to give info to LE. Telegram doesn't have open source server code and uses home rolled crypto. Was telegram even properly audited?
I would also recommend categories for what metadata is exposed; if messages are encrypted at rest on your device; cross platform ubiquity.
You should include bitmessage, and i2p-bote.
I am glad that this is only the first step but I do think that you shouldn't have done it alphabetically but rather by score and usability.
I would also recommend categories for what metadata is exposed; if messages are encrypted at rest on your device; cross platform ubiquity.
You should include bitmessage, and i2p-bote.
I am glad that this is only the first step but I do think that you shouldn't have done it alphabetically but rather by score and usability.
It wasn't clear to us whether the NSA intercepts Skype by breaking the crypto, or by compelling injection of false public keys in order to perform a man in the middle attack. In the latter case it's the third checkmark (lack of ability to verify keys) that's their users' undoing. We're talking to Microsoft about that at the moment, and may revise that entry.
There's a weird case around iMessage and any tool that is provided by an OS vendor. I think we need to add a note about this, but in those cases that company could inject malware or a backdoor either in the messaging system or somewhere else in the OS. Since we're trying to tackle one hard problem at a time (secure messaging but not secure operating systems and software distribution) there should be an extra caveat about offerings from OS vendors.
The only tool that gives strong metadata protection right now is Pond, and we aren't listing unusable tools that aren't out of beta yet. We considered but haven't yet included bitmessage for the same reason.
There's a weird case around iMessage and any tool that is provided by an OS vendor. I think we need to add a note about this, but in those cases that company could inject malware or a backdoor either in the messaging system or somewhere else in the OS. Since we're trying to tackle one hard problem at a time (secure messaging but not secure operating systems and software distribution) there should be an extra caveat about offerings from OS vendors.
The only tool that gives strong metadata protection right now is Pond, and we aren't listing unusable tools that aren't out of beta yet. We considered but haven't yet included bitmessage for the same reason.
question asked in good faith: does it really matter to a given user exactly how adversaries are successfully attacking Skype? shouldn't some of the things that we've already seen disclosed--e.g. that NSA gained significant, at-scale capabilities against Skype right after it was acquired by Microsoft--be enough to invalidate essentially any crypto-related promises the company may assert, or even those that an audit might support? if not, i think it's at least worth making a distinction between products with known backdoors and products without them. today's TAO attack is tomorrow's phd thesis, etc.
There's an article where Skype parses URLs out of the chats and visits them to "check for malware". Certainly they can see them.
I disagree that the only tool that gives strong metadata protection is pond. i2p-bote, bitmessage, and chatsecure over orbot at least advertise themselves as giving metadata protection by design. Thought I didn't realize bitmessage was in beta still.
The iMessage issue isn't related to iMessage being developed by the same company that handles the OS. It is that Apple holds the decryption keys for your messages; this is security by policy not security by design. http://blog.cryptographyengineering.com/2013/06/can-apple-re...
The iMessage issue isn't related to iMessage being developed by the same company that handles the OS. It is that Apple holds the decryption keys for your messages; this is security by policy not security by design. http://blog.cryptographyengineering.com/2013/06/can-apple-re...
IMessage absolutely does pass the mud puddle test.
What fails the mud puddle test is data stored in iCloud backups.
If you don't use iCloud backups, say goodbye to your I messages because they are only stored on the device.
What fails the mud puddle test is data stored in iCloud backups.
If you don't use iCloud backups, say goodbye to your I messages because they are only stored on the device.
They really need a wiki with talk pages for each of these applications...
EDIT: As in, I'd like to know what the details on each analysis is. And I'd also like this to be more like a living document.
EDIT: As in, I'd like to know what the details on each analysis is. And I'd also like this to be more like a living document.
Was there a particular reason Tox (https://tox.im/) wasn't included?
I second that. First thing I looked for was how they rate tox. Hopefully it gets added :)
Tox doesn't even have an official release. It's still in alpha stage development.
TigerText also seems noticeably absent.
Some extra info for commenters in this thread:
This Scorecard is phase 1 of our multi-stage campaign for Secure and Usable Crypto. We believe these criteria are necessary for any strong security tool, though meeting them doesn't guarantee that a system is perfect.
Subsequent phases of the project will focus on usability (which is a huge problem for activists and journalists who try to use encryption), metadata protection, openness and federatability of protocols, and much deeper audits of the design and implementation security of the software that is scoring highest.
This Scorecard is phase 1 of our multi-stage campaign for Secure and Usable Crypto. We believe these criteria are necessary for any strong security tool, though meeting them doesn't guarantee that a system is perfect.
Subsequent phases of the project will focus on usability (which is a huge problem for activists and journalists who try to use encryption), metadata protection, openness and federatability of protocols, and much deeper audits of the design and implementation security of the software that is scoring highest.
Who reviewed this scorecard for technical accuracy, and who reviewed the methodology?
Wow, I didn't think the EFF were this… misguided.
> 2014-11-04 : Snapchat app has audits from an internal security team.
That being enough for the "yes the app is audited" tick, plus CryptoChat looking to any passer by of this site as being A-OK is really concerning.
> 2014-11-04 : Snapchat app has audits from an internal security team.
That being enough for the "yes the app is audited" tick, plus CryptoChat looking to any passer by of this site as being A-OK is really concerning.
Thanks, I want to know about wechat. http://www.wechat.com/en/. Can any one provide the secure messaging score ?
Skype is definitely not safe since Microsoft purchased them and replaced the super-nodes with MS owned servers
Is Skype still encrypted end-to-end? I thought that a few months ago, Microsoft started running all Skype calls through Microsoft servers, decrypted.
Something that worries me is that while all the attributes the scorecard uses are good, there's at least one other I care about (in addition to those listed).
Uploading my contacts.
Even TextSecure does it. I'm still looking for something that gets a deserved perfect score and doesn't go near my contacts.
Uploading my contacts.
Even TextSecure does it. I'm still looking for something that gets a deserved perfect score and doesn't go near my contacts.
TextSecure does it in a nice way: https://whispersystems.org/blog/contact-discovery/
I also find it interesting that mumble wasn't mentioned in this secure messaging scorecard, but since this is the first step of a multi-phase project, I imagine the EFF will be updating it.
[1]. https://openintegrity.org