I'm Open Sourcing the Have I Been Pwned Code Base(troyhunt.com)
troyhunt.com
I'm Open Sourcing the Have I Been Pwned Code Base
https://www.troyhunt.com/im-open-sourcing-the-have-i-been-pwned-code-base/
20 comments
> They almost never tell you which site was actually breached
That's probably true for most people. But that's what email aliases are for. If I notice (through 1Password alerts) that my [email protected] email got leaked, I pretty much get an idea which site was breached. At least GMail and Fastmail support email aliasing.
That's probably true for most people. But that's what email aliases are for. If I notice (through 1Password alerts) that my [email protected] email got leaked, I pretty much get an idea which site was breached. At least GMail and Fastmail support email aliasing.
I keep using that scheme but it’s been clear for a while that most systems (and hence leakers) clean up those addresses. Which it has to be expected for a feature that has been around for 15 years.
I use `[email protected]` instead, which is easy enough to do with catch-all aliases.
Or me+Facebook and block email to me.
I have HIBP alerts setup for client domains and it quickly notifies me, shows me what user emails were in the breach, and a general idea of what data was in the breach (usernames, passwords, financial info). Maybe if you're reviewing old breaches it's pointless, and if you're using a pw manager and taking security seriously you don't need it. But most users will use the same password for work as they do with some stupid app so at least with these alerts I can go and remind them not to use work email to signup for a free hamburger or whatever
A lot of the "breaches" added to the site are from unknown origins.
Like, a large mixed collection sold online that somehow found it's way to Troy.
Last time I used HIBP, it told me what site the leak was from.
If your password shows up at all, you should change it wherever you use it.
Last time I used HIBP, it told me what site the leak was from.
If your password shows up at all, you should change it wherever you use it.
> FWIW, I once worked somewhere that somehow had access to a far better version of this data than they'll ever let the public get access to
If you really want to hunt it down, a lot of this data is available via torrent, although probably not too useful for generating alerts.
If you really want to hunt it down, a lot of this data is available via torrent, although probably not too useful for generating alerts.
1Password and other services plug into HIBP to alert you when an account has been pwned and the password should be changed.
If you use KeepassXC or another password manager of some kind, they are able to tell you which username/password combinations have been compromised.
Can you expand on that for me, please?
If your password is unique for each website because you use a randomly generated one, you can immediately [1] tell by looking at the password which website got compromised.
[1] keepassx does not give a way of searching passwords. You can manually look through the list of passwords though.
[1] keepassx does not give a way of searching passwords. You can manually look through the list of passwords though.
This is the same problem we've had with Certificate Transparency.
In grad school we looked at integrating CT monitors into web servers that manage certificates for your sites (including monitoring lookalike or spoofy names), but then weren't sure what to do when a suspicious certificate appeared in the logs. Do you email the site owner? Then what? Sure, you can report to CAs and web hosts and all that, but who will actually go to that trouble? By the time you do that, the site will probably already be blocked by SafeBrowsing and whatever other blocklists.
In grad school we looked at integrating CT monitors into web servers that manage certificates for your sites (including monitoring lookalike or spoofy names), but then weren't sure what to do when a suspicious certificate appeared in the logs. Do you email the site owner? Then what? Sure, you can report to CAs and web hosts and all that, but who will actually go to that trouble? By the time you do that, the site will probably already be blocked by SafeBrowsing and whatever other blocklists.
Wow this is awesome news. I always wondered how the internals could work. Can’t wait to take a peak...
One of the tweets in the article says Github is open-source: it's not.
True but FYI if you want to, you can reverse engineer the code of the GitHub enterprise version: https://blog.orange.tw/2017/01/bug-bounty-github-enterprise-...
The fact that you can (probably under break of license terms) get to source, doesn'tale it open source, as understood by majority of the community (which is somewhere close to OSI's definition)
Older versions of GH Enterprise even don't have ruby_concealer
You just have an open VM
You just have an open VM
dustinmoris(6)
What is this M&A process he keeps referring to?
He tried to sell it at one point.
https://threatpost.com/have-i-been-pwned-no-longer-for-sale/...
https://threatpost.com/have-i-been-pwned-no-longer-for-sale/...
Thanks!
For the lazy ones searching for a github link: It is not open source yet:
> HIBP isn't in a state to simply flick the visibility of it in GitHub, but it needs to get to that point. Instead, I need to choose the right parts of the project to open up in the right way at the right time.
and then further down:
> I don't have a timeline for each step along the way yet as HIBP remains something I do in my spare time and I've always got a bunch of other stuff on my plate, but the process has already begun and I'll be sharing more on that as soon as I can.
> HIBP isn't in a state to simply flick the visibility of it in GitHub, but it needs to get to that point. Instead, I need to choose the right parts of the project to open up in the right way at the right time.
and then further down:
> I don't have a timeline for each step along the way yet as HIBP remains something I do in my spare time and I've always got a bunch of other stuff on my plate, but the process has already begun and I'll be sharing more on that as soon as I can.
It's an interesting question: Could some open group etc replicate it entirely, e.g. something like Let's Encrypt (which is also a free service funded by various companies)? The data sources and import are the key bit where trust is an issue, and weirdly enough a single individual might have an easier time than a well-funded foundation etc.
Im not sure if those corporate institutions will allow it to notify breaches quicker, if HIBP is backed by likes of Facebook, google etc. Its better for HIBP to stay away from them.
I asked in comments on post which open source license he picked. Will update if he replies.
Hopefully it'll be a proper free one.
Hopefully it’ll actually be open source, as in free as in money AND free as in beer. That is, I hope the license Troy Hunt picks for Have I Been Pwned is OSI-approved:
https://opensource.org/licenses
https://opensource.org/licenses
Needless to say, I hope it’s also free as in speech. Saved the best for last.
Is there a reason (Security, Competitive, etc.) that the code wasn't open source from the start?
Very likely just boring reasons like not having thought about it, not having made the code in a way that's easy to publish.
Think something like he started the project at some point, never planning for it to become this big. It was probably some code written in a random directory, with no regard for whether test fragments are around, whether it contains test data that maybe shouldn't be public, different pieces that are closely tied to a server config that itself is very specific to what he already had running and would need some documentation to be useful etc. pp.
Think something like he started the project at some point, never planning for it to become this big. It was probably some code written in a random directory, with no regard for whether test fragments are around, whether it contains test data that maybe shouldn't be public, different pieces that are closely tied to a server config that itself is very specific to what he already had running and would need some documentation to be useful etc. pp.
Troy points out in the article that it started as and has mostly entirely been a series of hacks done in free time, including a lot of done with not enough sleep on long distance plane flights. It sounds reasonable to assume that a lot of it is duct tape and straw wire and that the imposter syndrome "it's not good enough to open source" played some presumably large factor.
Not wanting to have toxic user requests and issues open for something you do in your spare time could be one good reason.
I'm not sure where you get the impression those somehow don't exist if you don't have a bug tracker. They just go to email, or Twitter.
For something like this, keeping things private has allowed Troy to work out the best way to do things with complete autonomy. That's really useful if you have the time and resources to get to market on your own.
Now it's stable and best security practices are nailed down, he can open it up to a bit of scrutiny and feature bloom.
For something like this, keeping things private has allowed Troy to work out the best way to do things with complete autonomy. That's really useful if you have the time and resources to get to market on your own.
Now it's stable and best security practices are nailed down, he can open it up to a bit of scrutiny and feature bloom.
It's entirely possible to run a Git server which is read-only (for the rest of the world, anyway) and doesn't allow anyone to bother you.
What's the point of doing that though? It's just chumming the water for unsolicited feedback.
It allows others to easily mirror your code. The original code owner may disappear for any number of reasons (burnout, early retirement, being run over by a bus, etc.).
What would the reason for it being open-source anyways? Unless it's in a state for someone else to use it in their own space/context with separate data, etc, then making it open source is just a marketing/promotion tactic with the side-benefit of maybe having your code audited on some level. The rest is just negatives, tbh.
Respectfully, I disagree with your premise/question. Open Source is about way more than marketing/promotion. It's a means to make the world a better place. Unless something needs to be private because its a state secret (security through obscurity), all software should be open source by default. That way all of humanity can benefit from it and we can reduce duplication of effort.
What are the negatives? The value of HIBP is in its data, not in its algorithm. They lose nothing by open-sourcing.
[deleted]
I'm curious what the code would be/how far of an extent it goes. I mean from my experience using it it's a search box, type in email, get results... so is this going to show scrapers or something where the data comes from?
Maybe has efficient hash comparisons or something...
Maybe has efficient hash comparisons or something...
Troy does talk a little bit about the data here, but the codebase is not entirely useful without the data. I don't doubt it can be improved from Troy's one-man-efforts in that area, but the real value of haveibeenpwnd is in the data - and the code is useless without a huge trove of breach data - which I guess is never going to be open.
There’s no perfect solution here: publish the raw data, and people who haven’t changed their passwords get cracked, or publish hashed (in some way) passwords, and people will crack the passwords themselves. (That said, I guess most crackers will just get the data straight from the source, but still.)
He does publish the passwords hashed, with the number of times each password was leaked (without salt and the weak sha1).
Incorporate a non profit and have it own the data and run the service? No different than Let's Encrypt.
It is trivial to find (most) of the datasets HIBP uses
Breach data is, by definition, open. You just have to dig a little, and sometimes pay criminals.
Troy got it; so can you.
Troy got it; so can you.
That implies the same data still exists, which I don't think is true for many breaches.
I think it’s safe to assume that all breached info is archived and available forever.
Both as a curiosity, a research tool, and a crime tool.
As a hobbyist, I used to have a folder of breached data (ashley Madison was really fun). And if you add in groups like /r/datahoarders, the data will probably be kept someone.
The dataset could be reconstructed if needed.
Both as a curiosity, a research tool, and a crime tool.
As a hobbyist, I used to have a folder of breached data (ashley Madison was really fun). And if you add in groups like /r/datahoarders, the data will probably be kept someone.
The dataset could be reconstructed if needed.
Yeah make the data available so that we can have AWS Elastic Pwned, and HIBP to be eclipsed by it
It's good just for the sake of transparency. Maybe he will find someone who is interested in helping maintain and contribute.
[deleted]
I give it 24 hours after release before someone rewrites it in Rust.
48 hours before its on HN.
48 hours before its on HN.
This man knows
Very naive question. Suppose I offer a service (e.g. Have I Been Pwned) and I want my users to trust my service not to store/share their data. I understand that open sourcing the code base is one step, but how can I know that the server running the service actually does run this codebase and not something else?
On the side of your question, but HIBP's API is designed in such a way that you don't have to give them your password to get notified if it has been hacked. You instead provide HIBP with parts of the hash, and then it answers if you maybe have been compromised. Take a look at requests with devtools on the page https://haveibeenpwned.com/Passwords
So here it's taken one step further, you don't have to trust that it's the same code running on HIBP, because you don't have to trust them at all for the service to work.
So here it's taken one step further, you don't have to trust that it's the same code running on HIBP, because you don't have to trust them at all for the service to work.
This seems like it would still leave a vulnerability: There are many passwords that are commonly used, e.g. "password". If I submit "password" to haveibeenpwned, it hashes and sends the range "5BAA6". It seems to me that now anyone in control of haveibeenpwned could reasonably guess that my password has a decent chance of being "password". Using this methodology for the top 100 or 1000 passwords would presumably uncover a lot of passwords.
HIBP (or an eavesdropping attacker) can’t really assume that your password is in the list. There’s an essentially infinite number of passwords that would result in a hash ending with 5BAA6. “password” may be the most likely if it was in the list, but surely attackers would have been guessing commonly used passwords either way.
If your password is in the list, you should probably be changing. It being bad is how it ended up in the list.
If your password is in the list, you should probably be changing. It being bad is how it ended up in the list.
>>> HIBP (or an eavesdropping attacker) can’t really assume that your password is in the list.
According to [1] by Troy Hunt, around 86% of passwords from one dump were already in the HIBP database, so attackers could assume your password is in the list with 86% certainty, extrapolating from that.
If we look at the returned data for a sample hash prefix (that of "password", which is 5BAA6), and sort them by count of password use, we get these password hashes with the most usages:
Going with a more obscure password, like say "obscure", you get 1801/4319 = about 42% of passwords being "obscure". This means that if I search for the password "obscure" on HIBP, HIBP can be about 40% sure that my password is "obscure".
How is this not a huge security issue? I would agree that folks with unique passwords not appearing the database are safe, but anyone using a common password can be identified with pretty high probability, and even someone with a password only occurring once in the database is at risk because a typical query returns about 500 results: low enough that a human could brute force a web input that didn't have rate limiting in an hour or two.
[1]: https://www.troyhunt.com/86-of-passwords-are-terrible-and-ot...
According to [1] by Troy Hunt, around 86% of passwords from one dump were already in the HIBP database, so attackers could assume your password is in the list with 86% certainty, extrapolating from that.
If we look at the returned data for a sample hash prefix (that of "password", which is 5BAA6), and sort them by count of password use, we get these password hashes with the most usages:
~ curl -s 'https://api.pwnedpasswords.com/range/5BAA6' | sort -t: -k2 -n | tail
42BAADCD710F9EA7E62B60E01D05469AC64:14
EA2008F79BE2B0E0C02A1642725433BBB2F:15
3A8ADE4CF1DAD5342AF2F9FC9247EC21943:18
5E2BCB2FEF09257B0306B4744418999611B:18
A516C42C8CD4C7E7E328ABB90D002A9890E:29
CF2F87E596758D031C0006D1827C9908E5C:34
EF0E14CCB17E525D76050283148A57828F8:44
8E0D5C9D144BACC76E52C44F5B61E8DF629:213
2648FB0B2EDA4FDFF99BF51E912CD95C023:7201
1E4C9B93F3F0682250B6CF8331B7EE68FD8:3759315
Going with a more obscure password, like say "obscure", you get 1801/4319 = about 42% of passwords being "obscure". This means that if I search for the password "obscure" on HIBP, HIBP can be about 40% sure that my password is "obscure".
How is this not a huge security issue? I would agree that folks with unique passwords not appearing the database are safe, but anyone using a common password can be identified with pretty high probability, and even someone with a password only occurring once in the database is at risk because a typical query returns about 500 results: low enough that a human could brute force a web input that didn't have rate limiting in an hour or two.
[1]: https://www.troyhunt.com/86-of-passwords-are-terrible-and-ot...
> a typical query returns about 500 results
If you personally want to check your password wouldn't it help to curl more queries? Probably also taking care to make not too much per some unit of time?
If you personally want to check your password wouldn't it help to curl more queries? Probably also taking care to make not too much per some unit of time?
Even with 10 or 100 queries you still will get a probability distribution of (I'm guessing) 20-200 passwords that make up the most likely 50% of passwords, so I'm not sure how much good this would do if your password is relatively common. If it's very obscure this could help more, but you're still narrowing the password search space from 500M to 5k-50k.
Yes, if you suspect your password is common maybe you shouldn‘t use that service at all, but just download some smaller list of “common passwords.”
I also see problematic that, if I understood correctly, some browsers query automatically that service with your passwords which browser collected whenever you “saved” some login.
And they claim that is “for your security.”
I also see problematic that, if I understood correctly, some browsers query automatically that service with your passwords which browser collected whenever you “saved” some login.
And they claim that is “for your security.”
Wait, if you query HIBP, you are actually asking: is it safe to use thus password?
If you have a match, you will certanly use different password.
And, by the way, they have username/hashes before you even use their API.
If you have a match, you will certanly use different password.
And, by the way, they have username/hashes before you even use their API.
> Wait, if you query HIBP, you are actually asking: is it safe to use thus password?
I agree - this is the main use of HIBP.
> If you have a match, you will certanly use different password.
I doubt this is true in all cases. Sometimes people get lazy. But nonetheless, a malicious actor could still capitalize on the few minutes between a user checking a password on HIBP and that user changing their password. For example: for every password lookup, pick the most likely password with that hash, and if it has at least 90% share of the passwords with that hash, try to correlate an email address to their IP address (using leaked dumps), and then try to log into gmail with those credentials. Even if it works 1% of the time, a lot of harm could be done in a few minutes. (For example, US Social Security Numbers may appear in a years-old employment document in the email).
> And, by the way, they have username/hashes before you even use their API.
But they don't necessarily have yours. If I am somebody who uses a different password on every site, and I check a password for a site that has not had a leak, then they do not have an association between my email and my password before my query, and they do after my query.
Regardless of the particulars of possible exploits, I'm mainly claiming that the original comment above (by @matsemann) is not true, in that some level of trust in HIBP is necessary to input a sensitive password.
I agree - this is the main use of HIBP.
> If you have a match, you will certanly use different password.
I doubt this is true in all cases. Sometimes people get lazy. But nonetheless, a malicious actor could still capitalize on the few minutes between a user checking a password on HIBP and that user changing their password. For example: for every password lookup, pick the most likely password with that hash, and if it has at least 90% share of the passwords with that hash, try to correlate an email address to their IP address (using leaked dumps), and then try to log into gmail with those credentials. Even if it works 1% of the time, a lot of harm could be done in a few minutes. (For example, US Social Security Numbers may appear in a years-old employment document in the email).
> And, by the way, they have username/hashes before you even use their API.
But they don't necessarily have yours. If I am somebody who uses a different password on every site, and I check a password for a site that has not had a leak, then they do not have an association between my email and my password before my query, and they do after my query.
Regardless of the particulars of possible exploits, I'm mainly claiming that the original comment above (by @matsemann) is not true, in that some level of trust in HIBP is necessary to input a sensitive password.
This isn't really the way HIBP is used. While you can query the HIBP endpoint, the hashes are also available as (large) torrent downloads too. For many use-cases this means as the HIBP operator you won't see those requests since other service providers are likely using a local cache and not taxing Troy's infrastructure. Even when you are directly querying, the request is very likely proxied by the service operator so you won't be able to derive IPs specifically, just someone using Okta tried the password `obscure` (but not that they set their password to this - whomever they were, note I'm not saying Okta directly queries - just an example). Further, security recommendations (eg OWASP) require that companies test potential passwords against known leaks and discourage or prevent users from using frequently found values (typically 100+ uses or leaks.. note this either means one user has unluckily been leaked 100 times, or 100 people have used the same password) - this again encourages both proxied and possibly cached lookups (vs trusting client side code which could be disabled). Note that the expected use case is at password creation or password change, the new candidate is tested - not upon every login.
Ok so many use cases are removed - maybe your locally running password manager allows you to test a potential new password against leaks. Now we have a rare example of what you're hypothesizing, at this point you're trying to tie an IPv4 address to an identity. Many networks have either shared IPv4 addresses (NAT/GNAT in major metros, corporations, public Wifi), or dynamic IPv4 addresses (change daily/weekly/monthly). It's pretty hard (though not impossible) to link an IP to a person.. and vice versa, between 3g, 4g, 5g, wifi, work, home, complementary, wired and wireless connections a single person may appear across several addresses.
Finally the idea of accounts - knowing a specific IP just tested the password `password` and Aretha Franklin is most likely at that IP.. well, which of the hundreds of services might she be potentially considering the new password for? (again - testing, not setting the password to). If you could narrow it to one service, or brute force all of them, and assume the user ignores the dire warnings, you still need to know their user credential (be it email - of which they may have many, username - of which they may have many, or service-generated username which you'd need insider knowledge to obtain)
If you're worried about trust of your sensitive password, your password manager and the service you're using it on (if you reuse passwords) need far more.
Ok so many use cases are removed - maybe your locally running password manager allows you to test a potential new password against leaks. Now we have a rare example of what you're hypothesizing, at this point you're trying to tie an IPv4 address to an identity. Many networks have either shared IPv4 addresses (NAT/GNAT in major metros, corporations, public Wifi), or dynamic IPv4 addresses (change daily/weekly/monthly). It's pretty hard (though not impossible) to link an IP to a person.. and vice versa, between 3g, 4g, 5g, wifi, work, home, complementary, wired and wireless connections a single person may appear across several addresses.
Finally the idea of accounts - knowing a specific IP just tested the password `password` and Aretha Franklin is most likely at that IP.. well, which of the hundreds of services might she be potentially considering the new password for? (again - testing, not setting the password to). If you could narrow it to one service, or brute force all of them, and assume the user ignores the dire warnings, you still need to know their user credential (be it email - of which they may have many, username - of which they may have many, or service-generated username which you'd need insider knowledge to obtain)
If you're worried about trust of your sensitive password, your password manager and the service you're using it on (if you reuse passwords) need far more.
But how many people will check in detail what‘s submitted? And again, how do you prevent that different users get different functionality?
Only a few need to check to see what’s submitted and serve as a canary for others.
It’s possible some bad actor could do dumb stuff, but SSL helps in that at least the server would have a unique id.
Personally the first time I used HIBP I checked the javascript that was running and read about what happened.
It’s possible that someone could hack their server and replace the javascript with new bad versions. I don’t check anymore. But it’s unlikely, and a shared, common risk of using any site, I think.
But the important thing with HIBP (and other sites) is that I never send them any sensitive information.
It’s possible some bad actor could do dumb stuff, but SSL helps in that at least the server would have a unique id.
Personally the first time I used HIBP I checked the javascript that was running and read about what happened.
It’s possible that someone could hack their server and replace the javascript with new bad versions. I don’t check anymore. But it’s unlikely, and a shared, common risk of using any site, I think.
But the important thing with HIBP (and other sites) is that I never send them any sensitive information.
> Only a few need to check to see what’s submitted and serve as a canary for others.
I couldn't agree more. Even if you have root on their server and do a full audit, there's nothing to stop them from changing the code the moment you log out.
I couldn't agree more. Even if you have root on their server and do a full audit, there's nothing to stop them from changing the code the moment you log out.
> But how many people will check in detail what‘s submitted?
That's a valid criticism. It only takes one person finding clear evidence of problematic behavior to advertise that fact to the entire community. So long as a small fraction of people do actually check, the whole community will be fine. But if only a negligible number of people check it, then perhaps no one will be checking when it is abused.
> And again, how do you prevent that different users get different functionality?
Well, it depends on how the system is discriminating among users. To a great extent, that kind of abuse is prevented by anonymous users. You don't have to log into HIBP in order to use it. The system could still be discriminating by IP address, or even by various kinds of browser fingerprinting (including just collecting the advertising company cookies that so effectively eliminate anonymity for much of the web). Occasionally using TOR won't help against this -- a malicious operator could simply provide "clean" functionality to all TOR users.
- - - -
On the whole, things like, k-anonymous API design, making source code open, or having a small fraction of users checking for security issues all help a great deal and make it difficult for a malicious operator to abuse the system. But none of them are perfect. In the end it comes down to trust. Troy Hunt has EARNED that trust due to his openness, and in no small part by choosing to implement protections like these that he didn't need to implement, and I find that extremely persuasive. If you take steps to ensure that your CANNOT cheat your "customers", then that's pretty good evidence you aren't likely to be trying to cheat them.
That's a valid criticism. It only takes one person finding clear evidence of problematic behavior to advertise that fact to the entire community. So long as a small fraction of people do actually check, the whole community will be fine. But if only a negligible number of people check it, then perhaps no one will be checking when it is abused.
> And again, how do you prevent that different users get different functionality?
Well, it depends on how the system is discriminating among users. To a great extent, that kind of abuse is prevented by anonymous users. You don't have to log into HIBP in order to use it. The system could still be discriminating by IP address, or even by various kinds of browser fingerprinting (including just collecting the advertising company cookies that so effectively eliminate anonymity for much of the web). Occasionally using TOR won't help against this -- a malicious operator could simply provide "clean" functionality to all TOR users.
- - - -
On the whole, things like, k-anonymous API design, making source code open, or having a small fraction of users checking for security issues all help a great deal and make it difficult for a malicious operator to abuse the system. But none of them are perfect. In the end it comes down to trust. Troy Hunt has EARNED that trust due to his openness, and in no small part by choosing to implement protections like these that he didn't need to implement, and I find that extremely persuasive. If you take steps to ensure that your CANNOT cheat your "customers", then that's pretty good evidence you aren't likely to be trying to cheat them.
How do we know if anyone is checking?
Another good question but since it’s trivial (opening the browser dev tools during a submission), we can assume a significant number. Doubly so because any service that says it’s secure gets a lot more scrutiny.
You can't verify. Even if you could, you can't ensure that the code will be replaced the moment you turn around.
I investigated this thoroughly a few years ago, and the solution I came to is distributed computing. Join a peer-to-peer network an run the code directly on your own hardware or depend on someone that you actually trust. As far as I can tell there are no shortcuts that allow you to trust strangers with sensitive data.
I investigated this thoroughly a few years ago, and the solution I came to is distributed computing. Join a peer-to-peer network an run the code directly on your own hardware or depend on someone that you actually trust. As far as I can tell there are no shortcuts that allow you to trust strangers with sensitive data.
This is a very difficult problem to solve. As far as I know (and am welcome to be corrected!), the current state of the art is remote attestation - Signal wrote a post about how they're doing it for their contact discovery service[0]. It's a very limited solution though.
[0]: https://signal.org/blog/private-contact-discovery/
[0]: https://signal.org/blog/private-contact-discovery/
IDK, trusting a remote system just seems bad. Either give me the data to process locally, or I'll give you some data you couldn't possibly understand and you do some stuff to it and return the results, having never known the meaning of the data.
In Signal's case, they could just let you send messages that might never be received because the contact doesn't currently have the app. Also gives the receiver plausible deniability unless they respond. And could help get more people on Signal if they did the "Someone sent you a message, download the app now" trick.
In Signal's case, they could just let you send messages that might never be received because the contact doesn't currently have the app. Also gives the receiver plausible deniability unless they respond. And could help get more people on Signal if they did the "Someone sent you a message, download the app now" trick.
TL;DR they use Intel's SGX.
Intel's 20GB data leak was on HN yesterday, with a hint that more is coming. Will be interesting to watch ;-)
Intel's 20GB data leak was on HN yesterday, with a hint that more is coming. Will be interesting to watch ;-)
For my projects, I provide read-only SSH access when asked, but that probably does not work for everyone.
In theory you could maintain a separate system that the SSH accesses, changing the real system without revealing those changes.
True, and there is probably a way to prove that is not the case, but I have no idea what that way is right now.
On the other hand, if you are bothering to email me to ask for SSH access to ensure what I say is true, you probably have the knowledge to detect if I am lying from inside the server.
On the other hand, if you are bothering to email me to ask for SSH access to ensure what I say is true, you probably have the knowledge to detect if I am lying from inside the server.
Why bother? Change your password to a randomly generated string then send over the old password. You've cut off the threat vector.
Flipping the question around: How do YOU know that the server running the service is actually running the codebase and not something else? You might have been hacked. Someone at your hosting provider might be exfiltrating data. Intel might have a backdoor in their CPUs. It’s not practical for you to continuously verify all of these things, at the end of the day you have to trust that things do what they say they do until you have reason to believe otherwise.
Of course, if you can’t verify what’s running in your server then neither can your users. All you can do is be as transparent as possible about how you process their data and hope that’s enough to earn their trust.
As an aside, homomorphic encryption has been touted for a while now as a way for services to process encrypted user data without ever being able to decrypt it. Haven’t seen much use of it in the industry though, last I heard it was enormously impractical.
Of course, if you can’t verify what’s running in your server then neither can your users. All you can do is be as transparent as possible about how you process their data and hope that’s enough to earn their trust.
As an aside, homomorphic encryption has been touted for a while now as a way for services to process encrypted user data without ever being able to decrypt it. Haven’t seen much use of it in the industry though, last I heard it was enormously impractical.
You're actually implying that you can't trust any computer with the code you are telling it to execute. This is an exaggeration IMO. Sure you can have backdoors and hacks, but with what probability?
On the other hand, users need to put their trust in humans, not computers.
On the other hand, users need to put their trust in humans, not computers.
Ken Thompson wrote a paper about trust that was published in ACM in '84 entitled "Reflections on Trusting Trust". He makes the case for a trojan that could be built in such a way that it would infect a compiler, self-replicate into new versions of the compiler, and inject itself into specific binaries; all without appearing in the source code of the new versions of the compiler or the target binary.
Arguably, the probability of such an attack is extremely low--slim to none might be an overstatement--but it's an interesting thought exercise.
Arguably, the probability of such an attack is extremely low--slim to none might be an overstatement--but it's an interesting thought exercise.
I remember reading about this attack being implemented and executed in a university system decades ago. Perhaps apocryphal, but it didn’t seem that way.
You're possibly thinking of this: https://www.quora.com/What-is-a-coders-worst-nightmare/answe...
That's the one!
For what it’s worth you can detect a “trusting trust” attack using Diverse Double Compiling. David A. Wheeler implemented it as part of his dissertation back in 2009 and verified that gcc had not been infected.
https://dwheeler.com/trusting-trust/
https://dwheeler.com/trusting-trust/
I read the abstract and it seems kind of obvious: Yes if you have a trusted compiler you can bootstrap another.
But have can we know the "trusted compiler" is really to be trusted?
But have can we know the "trusted compiler" is really to be trusted?
All the godbolt users looking at their binaries from various compilers would quickly notice this, I think.
It doesn't need to go in every binary, the conventional attack only outputs the malware when it's compiling a compiler or sudo (or something).
This is the classic conundrum when thinking about supply chain attacks. It can happen, but at what probability threshold do you stop trying to take security measures?
On the other hand, users need to put their trust in humans, not computers.
It’s still a human trust problem all the way down because you had to buy your computer from someone, and they had to buy the parts from someone, and so on.
There are some places where it’s less of an issue due to cryptographic magic, but those are few and far between.
On the other hand, users need to put their trust in humans, not computers.
It’s still a human trust problem all the way down because you had to buy your computer from someone, and they had to buy the parts from someone, and so on.
There are some places where it’s less of an issue due to cryptographic magic, but those are few and far between.
Good question, built in content hash on those lambda function would be great to have.
You make your webpage have all logic client-side, and use standardized serverside API's (for example google datastore, or amazon s3).
Do not run any of your own application logic on the server - your javascript client should talk to the standardized datastore directly.
Then your users can see all the logic on their side, and inspect all the data, and if they want they can host the client side stuff themselves and run their own server.
Do not run any of your own application logic on the server - your javascript client should talk to the standardized datastore directly.
Then your users can see all the logic on their side, and inspect all the data, and if they want they can host the client side stuff themselves and run their own server.
This doesn't seem sustainable either though. Isn't server-side code required to ensure the data you're getting from the client is legit. You can't do everything client side especially because not everything can be done in javascript.
> Isn't server-side code required to ensure the data you're getting from the client is legit.
In this model, the client must validate the data before it puts it into the data store, and again when it gets it out of the data store (since the data could have been put there by a malicious/modified client)
In this model, the client must validate the data before it puts it into the data store, and again when it gets it out of the data store (since the data could have been put there by a malicious/modified client)
Open sourcing code is one thing. Open sourcing the email addresses of vulnerable victims is something else. It’s like publishing the largest vulnerability of all time before it can be patched
Am I missing something? I understand from the article that he is open sourcing the codebase, not the data...?
All of which is readily available on tor anyway. It's just now centralised.
All the email addresses are pulled from public leaks, often from years old leaks. Nothing like your analogy, if you're still vulnerable you should consider yourself lucky you haven't already been hacked.
Didn't Troy want to sell to someone just a few years ago?
Yes. Just last year. Sounds like this action is a result of that failure.
https://threatpost.com/troy-hunt-sell-have-i-been-pwnd/14556...
https://threatpost.com/troy-hunt-sell-have-i-been-pwnd/14556...
As mentioned in the article in the first paragraph:
"and it took a failed M&A process to get here"
Second paragraph:
"especially in the wake of the M&A process[0] that ended earlier this year right back where I'd started"
[0]: https://www.troyhunt.com/project-svalbard-have-i-been-pwned-...
"and it took a failed M&A process to get here"
Second paragraph:
"especially in the wake of the M&A process[0] that ended earlier this year right back where I'd started"
[0]: https://www.troyhunt.com/project-svalbard-have-i-been-pwned-...
> I'm sure I speak for [Junade] as well when I say we couldn't be happier that other companies have taken the model we pioneered and applied it to their own services too because at the end of the day, that's in everyone's best interests.
Socialism for "other companies," capitalism for us. It's details like this that prove to me how purely ideological it is to claim that we need capitalism to "produce value." _We_ produce value already. Capitalism uses that value for free or a ridiculously low rate, then turns around and charges us for it.
Socialism for "other companies," capitalism for us. It's details like this that prove to me how purely ideological it is to claim that we need capitalism to "produce value." _We_ produce value already. Capitalism uses that value for free or a ridiculously low rate, then turns around and charges us for it.
Isn't the data whats important?
Isn't the code just essentially a text input box that takes a string, hashes it, and runs it against hashed passwords in a database?
Isn't the code just essentially a text input box that takes a string, hashes it, and runs it against hashed passwords in a database?
Yup. Gathering the data is a large part of the value he provides. I wrote an article about how to implement this, and data gathering tasks are crucial: https://fusionauth.io/learn/expert-advice/security/breached-...
He touches on the non technical difficulties as well with his comment: "We invite parties to form their own views on the legality of the data." So the fact he's gathered it all lets HIBP be an service that other companies can use without worrying about the thorny legal question.
But there's also his reputation as a steward of the system, which is valuable beyond the data itself.
Anyway, while he didn't actually open source anything yet, I'm glad he's committing to it, as hopefully that will allow this internet resource to continue.
He touches on the non technical difficulties as well with his comment: "We invite parties to form their own views on the legality of the data." So the fact he's gathered it all lets HIBP be an service that other companies can use without worrying about the thorny legal question.
But there's also his reputation as a steward of the system, which is valuable beyond the data itself.
Anyway, while he didn't actually open source anything yet, I'm glad he's committing to it, as hopefully that will allow this internet resource to continue.
Yes, and Troy mentions this in the comments of the blogpost.
He's open-sourcing it so the community can help him manage the project, which I think is a fair request/hope.
He's open-sourcing it so the community can help him manage the project, which I think is a fair request/hope.
They almost never tell you which site was actually breached, nor do they ever give you any hints as to what password was actually compromised.
So really, when I show up in one of these alerts, I'm always asking myself:
- Was this a recent breach, or a redundant alert from something I dealt with months ago?
- What account do I actually need to update, if any, to be safe from this alert?
These questions almost never seem to be answered. As someone who uses a password manager and a different random password for every site, there's no way I'm going to proactively hunt down and change every single entry in its DB when I get the "alert of the week."
(FWIW, I once worked somewhere that somehow had access to a far better version of this data than they'll ever let the public get access to. That system actually did generate alerts that were actionable. I only wish I had a way to get useful alerts like that as a private individual.)