Auditor's response to "Our security auditor is an idiot" (Update 3)(serverfault.com)
serverfault.com
Auditor's response to "Our security auditor is an idiot" (Update 3)
http://serverfault.com/q/293217
21 comments
Customers of a particular gigantic consulting company have to be routinely dissuaded from "passwords similar to any old passwords should not be allowed", which is often in bid documents. To a man, when told the requirement is unsound, they suggest infinite retention of passwords in plaintext plus Levenstein distance and ask the consultancy why their engineers couldn't come up with that obvious solution themselves.
The reason for this requirement to repeatedly pop up in bid documents is misapplication of a poorly drafted standard by people who are theoretically experts at enforcing it's normative intent.
I would love to throw in an additional anecdote here, but it would violate a confidence. Pretend that $500k had been allocated to produce a list of plain text passwords and a combination of bureaucratic inertia, competence issues, and internal politics made that requirement an unstoppable freight train. That gives you the flavor of it.
The reason for this requirement to repeatedly pop up in bid documents is misapplication of a poorly drafted standard by people who are theoretically experts at enforcing it's normative intent.
I would love to throw in an additional anecdote here, but it would violate a confidence. Pretend that $500k had been allocated to produce a list of plain text passwords and a combination of bureaucratic inertia, competence issues, and internal politics made that requirement an unstoppable freight train. That gives you the flavor of it.
Lots of security standards require passwords to have at least one digit. Lots of security standards require passwords to be changed every so often.
The end result? A user starts with the password "Seekrit1" when they join the company, then the next month they change to "Seekrit2", "Seekrit3", and so on.
This is hardly what the security policy intended, so stopping that happening isn't actually a bad idea.
Obviously storing a complete password history in plaintext (which would even have to be online for the consultants' plan to work) is ridiculous, but pre-calculating a bunch of hashes of similar passwords every time a new password is set would certainly be feasible.
Changing passwords is usually a rare occurrence, but making this at least slightly computationally-expensive shouldn't be a problem. Of course, the more expensive the password-hashing algorithm, the fewer "similar" passwords you would reasonably be able to pre-calculate, which is an odd trade-off to have to make a call on.
One thing to be wary of, though: by pre-calculating and storing a bunch of hashes of similar strings, this opens you up to (what might be called) a related plaintext attack. I have no idea if existing password hashes are specifically designed to be resistant to this, but would guess not. (edit: I didn't think this sentence through correctly. Thanks, Joachim)
The end result? A user starts with the password "Seekrit1" when they join the company, then the next month they change to "Seekrit2", "Seekrit3", and so on.
This is hardly what the security policy intended, so stopping that happening isn't actually a bad idea.
Obviously storing a complete password history in plaintext (which would even have to be online for the consultants' plan to work) is ridiculous, but pre-calculating a bunch of hashes of similar passwords every time a new password is set would certainly be feasible.
Changing passwords is usually a rare occurrence, but making this at least slightly computationally-expensive shouldn't be a problem. Of course, the more expensive the password-hashing algorithm, the fewer "similar" passwords you would reasonably be able to pre-calculate, which is an odd trade-off to have to make a call on.
One thing to be wary of, though: by pre-calculating and storing a bunch of hashes of similar strings, this opens you up to (what might be called) a related plaintext attack. I have no idea if existing password hashes are specifically designed to be resistant to this, but would guess not. (edit: I didn't think this sentence through correctly. Thanks, Joachim)
> Lots of security standards require passwords to be changed every so often.
Which is almost as idiotic as storing a complete password history in plaintext, because it pretty much guarantees that passwords either (as you note) follow a simple pattern, or if that is made impossible, are written down in an easily accessible place.
Which is almost as idiotic as storing a complete password history in plaintext, because it pretty much guarantees that passwords either (as you note) follow a simple pattern, or if that is made impossible, are written down in an easily accessible place.
Draconian password schemes lead to really insecure user behavior. For example, an iTunes account requires upper-case, lower-case, and a number; but it doesn't tell you that until you try "insecure" passwords. So I bet a lot of users, feeling really frustrated, just use their name (capitalized), and birthday, rather than the semi-secure password the they wanted to use (or in my case, a fairly strong pass-phrase with no numbers).
A very convenient place to store and retrieve them is under a "passwords" folder in your private mail account or your Dropbox, both of which get synchronized with your unprotected smartphone...
One option might be to store a hash of the password with all digits removed alongside the password used. Then once someone uses Seekrit1 as their password, you have the hash of Seekrit too, so if they try to use Seekrit2 later you can tell them not to. Could apply other normalization rules - remove repeated letters; force all lowercase; remove punctuation and digits. That way once someone's used Seekrit1, because you've hashed 'sekrit', they can no longer use seEeEe_kriT23...
This is an interesting idea, but my very naive question would be:
Even if the datastore contained only password hashes (which is good), wouldn't it be (potentially multiple) orders of magnitude easier to brute-force password attempts in the event of a large number of potential permutations to be eliminated right off the bat?
Even though bruteforcing the stripped passwords wouldn't provide a full set of information, it might be a good starting point and quickly reduce the search or guessing space.
Disclaimer: I have no idea if this is a practical concern, just one that comes to mind when thinking about your suggestion.
Even if the datastore contained only password hashes (which is good), wouldn't it be (potentially multiple) orders of magnitude easier to brute-force password attempts in the event of a large number of potential permutations to be eliminated right off the bat?
Even though bruteforcing the stripped passwords wouldn't provide a full set of information, it might be a good starting point and quickly reduce the search or guessing space.
Disclaimer: I have no idea if this is a practical concern, just one that comes to mind when thinking about your suggestion.
To some extent, you're right. Bruteforcing the password becomes a case of first bruteforcing the kernel by searching the much smaller password space of lowercase letters, then once that's matched using that kernel to generate candidates by altering case, adding digits, etc. Obviously a smart search will immediately try uppercasing the first letter, and adding a 1 to the end, and in a lot of cases that's going to be right. So yes, it does reduce the strength of your encryption against bruteforce password search, by several orders of magnitude.
This is where advances in hashing approaches might negate that problem for you, though - and you'd need a proper cryptographer to figure this out. If it was a plain MD5 or SHA of the normalized password, you're absolutely vulnerable to an attacker using a rainbow table of lowercase alphabet words figuring out the 'root' of the actual password; then they could use that to construct password candidates to hash and compare to the full password. But just adding a big block of salt to the normalized password though would take it out of the pure-lowercase searchspace and force you back to bruteforce searching of the (admittedly small) password space. However, multiround salted hashing strategies like bcrypt rule out rainbow tables and allow you to tune how expensive bruteforcing is, and even though the password space of the password kernel is limited, you may be able to turn bcrypt up high enough to make it unfeasible to bruteforce.... maybe?
This is where advances in hashing approaches might negate that problem for you, though - and you'd need a proper cryptographer to figure this out. If it was a plain MD5 or SHA of the normalized password, you're absolutely vulnerable to an attacker using a rainbow table of lowercase alphabet words figuring out the 'root' of the actual password; then they could use that to construct password candidates to hash and compare to the full password. But just adding a big block of salt to the normalized password though would take it out of the pure-lowercase searchspace and force you back to bruteforce searching of the (admittedly small) password space. However, multiround salted hashing strategies like bcrypt rule out rainbow tables and allow you to tune how expensive bruteforcing is, and even though the password space of the password kernel is limited, you may be able to turn bcrypt up high enough to make it unfeasible to bruteforce.... maybe?
I might suggest that the password pattern strings are encrypted using a key based on the user's password plus salt. This way, the password pattern list is only recoverable AFTER recovering the users password (via brute force). I think this solves the reduced search space problem of your scheme.
$a = passwO_rd1; $b = ($a striped down to lower case); $c = ($b's metaphone); $d = md5(md5($a.'salt1'.b).$c.'salt2'); $e = md5(md5($c.'salt3').$c.'salt4'); Store $d as encrypted password then store $e as a string to check against. have $e saved in a separate communal database so to brute force $e and then guess the missing characters and the right user it would be. ([missing characters]^[character set])^([password lengh] ^ [users])^2 or something along those lines
I've seen a 30 day password requirement with numeric digits circumvented simply by month + year. jun2001, jul2001, etc.
Instead of storing those pre-calculated hashes of similar strings, generate hashes on many variations of the new password and see if any match the old hash.
"Related plaintext" should not be an issue for any competent password hash (classical crypt and bcrypt are based on a block cipher, which means this should work; PBKDF2 is based on a cryptographic hash, which means this should work; etc.)
I'm not exactly sure what you're saying, but it sounds like you're saying bcrypt is reversible? It's not... it's "based on" a block encryption algorithm in the sense that it uses the encryption algorithm's key scheduling functions. Irreversibility is the intent.
bcrypt isn't reversible (with respect to the password). The point is that Blowfish(key1, "message") is not at all similar to Blowfish(key2, "message"). This is called resistance to related-key attacks (look this up for a formal definition), and is an explicit design goal of every real-world block cipher.
bcrypt is just Blowfish(key-derived-from-salt-and-password, "OrpheanBeholderScryDoubt"), so it inherits this property.
bcrypt is just Blowfish(key-derived-from-salt-and-password, "OrpheanBeholderScryDoubt"), so it inherits this property.
It's not reversible, but block encryption should be easy to compare for similarity. If the old password is 5 blocks, and 4 of 5 blocks match the new password (in order)... I'm guessing this is what the gp was talking about. (of course block size matters, if all passwords are a single block that doesn't help)
That's why we use a fresh salt for each password. Bcrypt incorporates a fresh salt into each new hash. Your and the GP's concern in this area is not warranted.
My personal favorite is the "at least one capital". I've taken mnemonics with a large subset of the asciibet, but I can't memorize case well. So what happens? Every password begins with one capital. "Ehsiwvky" instead of "ehsiwvky", right?
I enjoy the juxtaposition of following the letter of the rule, violating the intent of the rule, but nonetheless honoring the spirit of the rule.
I enjoy the juxtaposition of following the letter of the rule, violating the intent of the rule, but nonetheless honoring the spirit of the rule.
While that is an absolutely ridiculous requirement, couldn't it be satisfied securely by creating on the order of 1-10k modifications of each password when created, and hashing those? That would be enough to add numbers to the end, switch case, reverse, randomly change letters etc.
It's a lot of trouble to go through just because of the ignorance of a client, but it would be enough to check that magic box on the requirements list and be done with it, rather than having to go to all of the trouble of trying to educate the client and risk them going elsewhere.
It's a lot of trouble to go through just because of the ignorance of a client, but it would be enough to check that magic box on the requirements list and be done with it, rather than having to go to all of the trouble of trying to educate the client and risk them going elsewhere.
It's 3am and I've been drinking, but this seems feasible: When the user changes their password, encrypt the old password with a key derived from the new one (you have both in plaintext at this point, when entering old/new passwords). Then hash the new one as usual.
Then when the user goes to change their password again, you have the plaintext of the current one, which unlocks the previous password, and so on back in the chain. You can then compare the new password with previous passwords for similarity.
Then when the user goes to change their password again, you have the plaintext of the current one, which unlocks the previous password, and so on back in the chain. You can then compare the new password with previous passwords for similarity.
I like this idea, but I don't think it can be enforced. If I "forget" my password, then all the history is lost with it.
If the old password is "forgotten" and can't be retrieved by the user who made it up, then it doesn't seem like that big a risk.
The issue is that once your users figure out the system, they'll always "forget" what their password was once it's getting around to expiring.
The method I am about to mention here is based on that which was outlined earlier by somebody else [1]. As I will mention further down, the method I suggest here will give the administrator and the CEO the opportunity to conspire and retrieve users passwords without the affected users knowledge, but I will here assume that such could not happend and that if they really wanted to do so, they could find other means anyway (keylogging, etc.)
First time: On his client, the user is asked to enter a password which is checked to comply with password rules. This password is salted and hashed before it is stored on a central machine.
n-th time: On his client, the user first enters password "n - 1" which is then salted and hashed and sent to the central machine for comparison. If hashes match, you may enter the new password "n" which is checked to comply with password rules.
Password "n - 1" is then encryted with password "n" and stored on the central machine. Password "n" is stored on the central machine in two forms:
1) Salted and hashed.
2) Encrypted with a public key.
Password "n" is encrypted with a public key and stored on a he central machine. Also, when this machine stores the new public key encrypted password, it deletes its own records of any previous public key encrypted password for the user in question.
Through recursion, all old passwords ("n - 2" ... "1") are decrypted and the new password is checked so that it does not match any previous password and that it is not a simple permutation of any of those.
If the user forgets his password, the administrator physically logs into a machine which is very tightly secured. This machine does not allow inbound connections of any kind over the network and can only make outbound connections to the central storage machine. Furthermore, the machine does not allow the usage of any removable media (CD, DVD, USB stick, etc.). The drives of the machine in strongly encrypted, as is the private key which is stored on this machine. In order to decrypt the disc of the machine, two passwords are needed. One of the passwords is held by the administrator, the other by the CEO of the company. The same goes for the private key which also has two passwords, both of which are needed to decrypt it. Thus, two people must be involved in order to decrypt the password for a user. Once the administrator and the CEO has logged into the machine, they connect to the central storage and retrieve the encrypted key for the user and then they open their shared private key and decrypt the password. After the user has been given his old password, he must create a new password following the procedure described above. That way, the administator and the CEO does not know the users new password (unless they conspire to do so, but that would be breach of trust).
http://news.ycombinator.com/item?id=2820976
First time: On his client, the user is asked to enter a password which is checked to comply with password rules. This password is salted and hashed before it is stored on a central machine.
n-th time: On his client, the user first enters password "n - 1" which is then salted and hashed and sent to the central machine for comparison. If hashes match, you may enter the new password "n" which is checked to comply with password rules.
Password "n - 1" is then encryted with password "n" and stored on the central machine. Password "n" is stored on the central machine in two forms:
1) Salted and hashed.
2) Encrypted with a public key.
Password "n" is encrypted with a public key and stored on a he central machine. Also, when this machine stores the new public key encrypted password, it deletes its own records of any previous public key encrypted password for the user in question.
Through recursion, all old passwords ("n - 2" ... "1") are decrypted and the new password is checked so that it does not match any previous password and that it is not a simple permutation of any of those.
If the user forgets his password, the administrator physically logs into a machine which is very tightly secured. This machine does not allow inbound connections of any kind over the network and can only make outbound connections to the central storage machine. Furthermore, the machine does not allow the usage of any removable media (CD, DVD, USB stick, etc.). The drives of the machine in strongly encrypted, as is the private key which is stored on this machine. In order to decrypt the disc of the machine, two passwords are needed. One of the passwords is held by the administrator, the other by the CEO of the company. The same goes for the private key which also has two passwords, both of which are needed to decrypt it. Thus, two people must be involved in order to decrypt the password for a user. Once the administrator and the CEO has logged into the machine, they connect to the central storage and retrieve the encrypted key for the user and then they open their shared private key and decrypt the password. After the user has been given his old password, he must create a new password following the procedure described above. That way, the administator and the CEO does not know the users new password (unless they conspire to do so, but that would be breach of trust).
http://news.ycombinator.com/item?id=2820976
Proposing that any 'forgotten password' reset requires the physical presence of the CEO and a sysadmin at a dedicated machine is a really good way to ensure that within six months all passwords are either stored in plain text or written down on a post-it note.
"It's 3am and I've been drinking, but this seems feasible"
I have come up with some astoundingly dumb ideas that were preceded by exactly the above thought.
However, there was some sheer brilliance, as well.
I have come up with some astoundingly dumb ideas that were preceded by exactly the above thought.
However, there was some sheer brilliance, as well.
If you are storing passwords properly with bcrypt, then the 1-10k permutations will take far too long to calculate, especially considering that you need to give the user feedback on their new password choice being ok or not within ~100ms
(Edit: so the match might be quick, but pre-comp of the hashes and storing them is not. how the hell would you store 1-10k hashes associated with a user anyway? :) )
(Edit: so the match might be quick, but pre-comp of the hashes and storing them is not. how the hell would you store 1-10k hashes associated with a user anyway? :) )
What kind of bcrypt run times should be targeted? In my back of the envelope calculations I used 100ms per core, which would make 80/s on an 8 core machine, or 12.5s for 1k hashes and 2 minutes for 10k. Which doesn't seem unreasonable.
You can give the user feedback on their new password choice after just one hash. The permutations would be stored from their last password set, so it would just be a simple lookup to determine whether a given new password is similar to an old one.
I'm in no way recommending this - it was just an amateur guess at a method to fulfill the requirements without exposing a massive security vulnerability.
You can give the user feedback on their new password choice after just one hash. The permutations would be stored from their last password set, so it would just be a simple lookup to determine whether a given new password is similar to an old one.
I'm in no way recommending this - it was just an amateur guess at a method to fulfill the requirements without exposing a massive security vulnerability.
I aimed for 10-20/sec in dev on my macbook, but I have been doing everything on appengine recently, and haven't been getting consistant results on the live environment. It varies a lot in profiling - and I am thinking of slowing it down further since I have long-lived sessions on by default (set to a year) so password checks are rare
I made the session tokens crazy long though, and have one for the user session and another for auth, and a sanity check within the auth token that will add any IP that attempts to forge the session token added to the ddos blacklist after 3 attempts. The auth token generation is slow as well - which is something that is often overlooked in webapps. I got a bit carried away in implementing all of it - but it was fun
I made the session tokens crazy long though, and have one for the user session and another for auth, and a sanity check within the auth token that will add any IP that attempts to forge the session token added to the ddos blacklist after 3 attempts. The auth token generation is slow as well - which is something that is often overlooked in webapps. I got a bit carried away in implementing all of it - but it was fun
When setting a password, you compare the new password to the old hashes, which were calculated the last time the password was changed.
So user feedback should be speedy enough.
Only once the password is deemed acceptable does the system need to pre-calculate the hashes for the next time the password is changed.
Of course, pre-calculating 10,000 bcrypt hashes might be too computationally expensive anyway, but the user wouldn't see a delay.
So user feedback should be speedy enough.
Only once the password is deemed acceptable does the system need to pre-calculate the hashes for the next time the password is changed.
Of course, pre-calculating 10,000 bcrypt hashes might be too computationally expensive anyway, but the user wouldn't see a delay.
If you use a different salt for each modification of the old password, you must compute the all the hashes of the new password.
Another problem is that to prevent the use of the very old passwords, you should keep the hashes of the modifications. To make the comparations fast, you have to use the same hash.
Now you have a list of #password-changes * #passwords-modifications, and if an intruder get them, it is possible to make a mini-rainbow attack.
Another problem is that to prevent the use of the very old passwords, you should keep the hashes of the modifications. To make the comparations fast, you have to use the same hash.
Now you have a list of #password-changes * #passwords-modifications, and if an intruder get them, it is possible to make a mini-rainbow attack.
you are right - I have it the wrong way around. still you need to do the calculations and find somewhere to store all of those hashes
but I can't think of any other way of making 'similar to' work
but I can't think of any other way of making 'similar to' work
A better method would be to save previous password hashes when a password is changed, then check hashes of permutations of the new password against the previous ones.
It's a storage versus computation time trade off, and with something like bcrypt doing the hashing, hashing 1-10k passwords will be too slow to provide real time feedback as a user is changing their password. On the other hand, storing even 10k hashes per employee per month of employment just isn't that much storage.
That would verify they aren't bouncing from password A to password B and back to A again... but that's all.
That would only be true if it didn't keep history. Let's assume my previous two passwords are "love123" and "hate098". If I try to use a new password "love124", then the permutation checker (which, for the moment, knows the new password in plaintext) should check the hash of "love123" and reject it.
Oh. If you write a permutation checker that happens to output one of your old passwords, then yeah, that'd work. I wonder if that's feasible. It's an interesting idea. Computing hashes is relatively cheap, so I wonder if, say, 1,000 "permutation checks" would be enough.
(Although if you're using bcrypt to store passwords, which you should be, then apparently it's not so cheap.)
But.. 26 letters in the alphabet + ten digits = 36... and 36^n grows really, really fast. So maybe you'd need way more than 1k permutation checks.
(Although if you're using bcrypt to store passwords, which you should be, then apparently it's not so cheap.)
But.. 26 letters in the alphabet + ten digits = 36... and 36^n grows really, really fast. So maybe you'd need way more than 1k permutation checks.
That was my first thought too, but it does not satisfy the "similar" part of the requirement.
It does, because if the new password is similar to the old one, you can either modify the old one to get the new one or modify the new one to get the old one.
But if you have hashed the password correctly (i.e. a one way hash) you shouldn't know what the previous passwords are. This means that you would only be able to test if the password was the same, not similar.
You could require users to enter their old password when changing it. That way, you can compare two plain-text passwords without the need for plain-text storage (or pre-computing, like other commenters suggested).
This works well to make sure that your new password isn't similar to your old password.
Unfortunately, it can't be extended to make sure that the new password isn't similar to the password from two generations back.
Unfortunately, it can't be extended to make sure that the new password isn't similar to the password from two generations back.
You are correct, of course. Eventually some user would figure out they can rotate "aaaaa" and "bbbbb" every other week to avoid having to remember a better password. That being said, they could just follow up with "ccccc" instead of revert to "aaaaa" (or go through the words of some pop song, which would arguably be a somewhat safer pattern to follow). In other words: creative users will find ways to cheat whatever measure you come up with. The only solution is giving them no choice in the matter and to randomly generate passwords for them, but that opens up another can of worms as they will start writing them down on post-it notes attached to their monitors.
The only real solution is educating users. Comparing to the previous password (provided by the user when changing it) is close enough to the spec to be able to get away with it in most circumstances (possibly combined with saving a history of password hashes, so you can see if people don't repeatedly use the same password).
The only real solution is educating users. Comparing to the previous password (provided by the user when changing it) is close enough to the spec to be able to get away with it in most circumstances (possibly combined with saving a history of password hashes, so you can see if people don't repeatedly use the same password).
Actually, I think there is a technical solution. Say that the password system keeps a complete history of a user's plaintext passwords, stored encrypted using the current password as a symmetric key. Administrators cannot access this password list without the current password.
When a user changes his password, he enters the current password and a proposed new password. The system uses the current password to decrypt the list of previous passwords. It checks the proposed password against the list. If accepted, the old password is added to the list, and it's encrypted using the new password and stored.
Are there any threats against this scheme, or will it work?
When a user changes his password, he enters the current password and a proposed new password. The system uses the current password to decrypt the list of previous passwords. It checks the proposed password against the list. If accepted, the old password is added to the list, and it's encrypted using the new password and stored.
Are there any threats against this scheme, or will it work?
Has any work been done to solve reusing passwords with homomophic encryption? (this is a scheme that allows you to perform operations on the encrypted text without decrypting it).
[deleted]
You do not have to retain plaintext passwords to implement that. Just remember the past X hashes, generate all passwords within edit distance D of the new one, and compare the two lists.
Edit distance 3, even with "letter stays letter, digit stays digit, random additions and removals" will already give millions of variants, so this may be impractical, but impossible? No.
Edit distance 3, even with "letter stays letter, digit stays digit, random additions and removals" will already give millions of variants, so this may be impractical, but impossible? No.
Oops, I should remember to check whether my replies are still useful after taking a break for a couple of hours. Finishing a once-original reply and hitting submit not always is the best idea. Apologies.
I'm afraid you are naive.
This is not a lone idiot in an otherwise sane industry. People like this can thrive and remain blissfully ignorant because the entire ecosystem around them is incompetent, from senior executive to junior trainee.
In our industry, entire companies operate in complete isolation, basing their practices on the knowledge of their most "senior" engineer who was there at the beginning, however clueless that engineer might be. Companies like this can continue to operate successfully for many years, moving from one client to the other, and you will find them in any niche of the software industry, including startups.
This is not a lone idiot in an otherwise sane industry. People like this can thrive and remain blissfully ignorant because the entire ecosystem around them is incompetent, from senior executive to junior trainee.
In our industry, entire companies operate in complete isolation, basing their practices on the knowledge of their most "senior" engineer who was there at the beginning, however clueless that engineer might be. Companies like this can continue to operate successfully for many years, moving from one client to the other, and you will find them in any niche of the software industry, including startups.
Why won't someone expose them?
Expose them publically? Because that would you get you in trouble with your job at nearly every company.
Expose them within the company? Only works if your management supports you, and they are probably only doing a security audit so they can get a piece of paper with some initials on it. They just want it to go away, not to hire another auditor to start from scratch.
Expose them within the company? Only works if your management supports you, and they are probably only doing a security audit so they can get a piece of paper with some initials on it. They just want it to go away, not to hire another auditor to start from scratch.
Because as with this instance they threaten to sue.
And being right in a court of law just means you win, it doesn't mean that you don't get your time, energy and money sapped so really, where's the benefit for the person exposing them?
And being right in a court of law just means you win, it doesn't mean that you don't get your time, energy and money sapped so really, where's the benefit for the person exposing them?
"maybe I'm just naive about how messed up the real world is"
I'm afraid you might be. Having read the auditor's responses I recognized the tone instantly. Idiots like this do exist - they advance by throwing around lots of buzzwords and touting their "years of experience" which is often enough to bamboozle their equally incompetent superiors.
I'm afraid you might be. Having read the auditor's responses I recognized the tone instantly. Idiots like this do exist - they advance by throwing around lots of buzzwords and touting their "years of experience" which is often enough to bamboozle their equally incompetent superiors.
To be fair, startups aren't any different. They do things they shouldn't do just as often. I honestly wish I could share some stories, but I can't. Suffice it to say, don't assume a startup is better just because it's backed by YC or some other fund.
All startups do things they shouldn't do; they are in the business of experimentation, iterations and improvements. However the important thing is that most of the startups realize their mistakes, acknowledge them, learn from them and move on. In this case, the guy is insistent upon being a nutcase.
Sorry, I'm not referring to mistakes. Instead, I'm referring to start-ups doing things they know they shouldn't be doing in the first place. Taking the short cuts that can/will cause real harm. I'm not talking about ignorance here. I'm talking about Russian Roulette.
Then again, after working at startups my whole career, maybe I'm just naive about how messed up the real world is. I second this... Working in a startup every time I see someone having to work around corporate BS to get something simple done, this comes to mind.
> maybe I'm just naive about how messed up the real world is.
Try coming into Enterprise software :)
Try coming into Enterprise software :)
When "I have been in this industry longer than anyone on that site" is a cornerstone of your defence, you've already lost. Also, I know the UK has really strict laws, but I doubt he'd be able to sue for "liable".
Actually we do have ridicules libel laws here in the UK. If someone accuses you of libel the onus is on you to prove that you didn't.
I think the point is that someone who can't spell libel correctly is not very scary.
Stupid people can still cause huge amounts of trouble. With legal things like that maybe even more so than non-stupid people.
Also, apparently I can't spell "ridiculous" :p
Also, apparently I can't spell "ridiculous" :p
Inconceivable!
Very good point on idiots causing huge amounts of trouble.
Ouch, thanks for reminding me.
Very good point on idiots causing huge amounts of trouble.
Ouch, thanks for reminding me.
being that ignorant and having important hi-tech industry job - he's already won.
Nitpick: there are no such things as "UK libel laws" - England and Wales have one system, Scotland has a completely different set of laws.
This is true but the distinction is pretty minimal. Scots judges have traditionally followed the English lead seeing no sense in having different interpretations North and South of the border for the same story in potentially the same publication.
About the only real difference when it comes to defence is that up here you can defend something based on it being made in the heat of an argument, and that the person claiming defamation has to be able to show harm (though not necessarily financial).
About the only real difference when it comes to defence is that up here you can defend something based on it being made in the heat of an argument, and that the person claiming defamation has to be able to show harm (though not necessarily financial).
Having been involved on the periphery of a major criminal case which started off as a defamation case (there being no libel in Scotland) I can assure you that that is not the case. Scottish Defamation and English Libel are quite different laws. A number of people have made good money from arbitraging the difference.
I understand that it's defamation (which combines both libel and slander) but what other differences are there?
According to this BBC article about the Tommy Sheridan case
:
"In Scotland, only compensatory damages are allowed. Punitive damages are not competent."
http://news.bbc.co.uk/1/hi/scotland/5246716.stm
There are a lot of differences between English and Scots law - the most obvious being the extra verdict of "not proven" in criminal cases.
"In Scotland, only compensatory damages are allowed. Punitive damages are not competent."
http://news.bbc.co.uk/1/hi/scotland/5246716.stm
There are a lot of differences between English and Scots law - the most obvious being the extra verdict of "not proven" in criminal cases.
I know there are a lot of differences (I'm English and live in Scotland), just my understanding of libel / defamation is that broadly the nature of rulings was similar - essentially that Scotland's system while slightly different wasn't that much better than the English system.
As far as I understand it, the "no punitive damages" is a huge difference.
I think grandparent was chuckling at suing for "liable" not "libel".
It's a flat lie as well.
He doesn't know, and can't know, everyone commenting to HN. At best hes fabricating. I was going to top post that but checked first to see if anyone had surfaced that line.
The auditor is an idiot and feels cornered.
Also: "I'm going to assume you do not have PCI installed on your servers".
PCI is an auditing standard. Not something "you have installed on your servers".
And I guess I'm getting too far ahead of myself, because the OP comments on this in his serverfault posting.
Classic! "PCI SSC have responded and are investigating him and the company. "
I'd say samarudge did a bang-on job here.
He doesn't know, and can't know, everyone commenting to HN. At best hes fabricating. I was going to top post that but checked first to see if anyone had surfaced that line.
The auditor is an idiot and feels cornered.
Also: "I'm going to assume you do not have PCI installed on your servers".
PCI is an auditing standard. Not something "you have installed on your servers".
And I guess I'm getting too far ahead of myself, because the OP comments on this in his serverfault posting.
Classic! "PCI SSC have responded and are investigating him and the company. "
I'd say samarudge did a bang-on job here.
He should have explained that the servers don't have any more PCI slots available.
[deleted]
I can't help but feel there is some kind of social experiment going on here.
I mean I know there are a lot of incompetent people out there, but a security auditor asking for a list of plaintext passwords is not something that should take more than an email or two to resolve even in Bizarro world. "Techies" exhibiting this kind of willful ignorance are usually a bit better and hiding under their rock.
I mean I know there are a lot of incompetent people out there, but a security auditor asking for a list of plaintext passwords is not something that should take more than an email or two to resolve even in Bizarro world. "Techies" exhibiting this kind of willful ignorance are usually a bit better and hiding under their rock.
Some programmer in AZ told me a couple days ago to in order to search their user profile database(location, department etc), instead of changing his web service to accept some criteria, I should just use the GetAll() method and do it locally (a few thousand profiles), when presented with the logical use cases and the little "performance issue" he quit his job.
He'd be a good actor, because he clearly doesn't know what PCI compliance is. He mentions PCI should be installed.
I also really feel this guy is trying to get the password by social engineering means or something it just seems completely ridiculous to ask for this stuff. Why would he need the info in the first place? I wished the author of the post had asked him that. I'd like to know his answer to that.
I can tell you what it is, he's got some requirement he needs to fulfill (like, say, ensuring password strength) and he's figured out some way to satisfy it by analyzing all of the plain-text passwords. He refuses to give his reasons and refuses to back down because those are parts of the personality flaws which have led him down this road to start with. People who never accept they are wrong, never accept criticism, and never give up on bad ideas: those are the people who hold on to bad ideas forever. How do you think incompetence is maintained in the face of a world filled with people hostile to it? It takes a strong defense mechanism.
People are naturally unresponsive to criticism. If you can get someone to admit they're wrong without shame or guilt, they will happily concede.
Social engineering was my initial suspicion as well. However, that really doesn't fit with the auditor's obstinate response. At this point an intelligent auditor should be coming clean so that this doesn't go public (and so that his company doesn't lose the contract and get further heat).
I have the same concern you do that he shouldn't need this information. Reading some of the other comments here, though, leads me to believe that there really are people who feel like this is "security". I've definitely encountered sites and password systems that require you choose drastically different passwords from any you've chosen in the past. This may be what the auditor was getting at (though in my opinion it's a weak, even counter-productive security measure)
I have the same concern you do that he shouldn't need this information. Reading some of the other comments here, though, leads me to believe that there really are people who feel like this is "security". I've definitely encountered sites and password systems that require you choose drastically different passwords from any you've chosen in the past. This may be what the auditor was getting at (though in my opinion it's a weak, even counter-productive security measure)
I take it you've never worked with someone who was truly incompetent. Often such people have some sort of defense mechanism that allows them to cover up their incompetence with some degree of effectiveness. But sometimes it shines right through, and then you end up gobsmacked that it's even possible for someone to be that ignorant/illogical.
But it does happen, I've seen it too many times.
But it does happen, I've seen it too many times.
Sure, but such people can not remain the lynchpin of a PCI auditing operation. I mean how many WTFs from anyone who has half a clue have to be sent to the brass at this company and the ones they are auditing on behalf of before someone thinks "holy shit our entire reputation and business is being flushed down the toiler on a daily basis?"
It just doesn't pass the smell test for me. I don't know what it is, but something is missing from this story.
It just doesn't pass the smell test for me. I don't know what it is, but something is missing from this story.
Maybe not forever, but long enough to cause pain in your life? Sure. Perhaps the company was bought or created by someone with too much money and not much domain expertise and then they gave their incompetent relative a position without much oversight. Wouldn't be the first or last time that happened.
Yes, but if your payment gateway is going to shut you off based on this nonsense you call them up and explain the situation to them and then they investigate. There are too many parties involved for this kind of willful ignorance to be the last word.
I dunno, maybe it's true, but for me the story still isn't adding up.
I dunno, maybe it's true, but for me the story still isn't adding up.
The auditor was actually serious. Wow. I really thought he just wanted to check the reaction of the admin, to see if he'd actually hand over sensitive stuff.
The auditor has a hugely inflated ego, something that might be a sign you're dealing with a narcissist. It's a pretty huge claim to make that you've "been in the industry longer" than anyone else, but that's the sort of thing you'd say when you're that kind of person.
Everything they said is surely made up, and if you challenge them on their facts, they'll attack you in return.
Everything they said is surely made up, and if you challenge them on their facts, they'll attack you in return.
Yup. Sounds like a fairly classic case of workplace sociopathy -- the sociopath who gets promoted because they tell lies and bully the folks below them into obedience. (Not the kind of -- much rarer -- sociopath who is violent or robs banks, but still a royal pain in the ass to run across in a business context because any attempt you make to deal with them technically will result in manipulative social retaliation rather than "oh, my bad".)
Please tell me this is a troll. I mean, I've seen some pretty incompetent people doing jobs they're not cut out for before, but I really want to believe this level of incompetence can't actually find jobs.
I wish I could be saying I believe he was being tested by the auditor, but it's just gross incompetence...
Some people excel.... on well... giving good interviews.
Back then, I used to believe that people know what they are doing, but since I've lived long enough know, maybe not.
You gotta be careful, people like that will come to personality attack when push come to shove. When facts are against them, it is now time to yell.
Back then, I used to believe that people know what they are doing, but since I've lived long enough know, maybe not.
You gotta be careful, people like that will come to personality attack when push come to shove. When facts are against them, it is now time to yell.
"Some people excel.... on well... giving good interviews."
The sort of interview that this guy would do well in is a poorly-conducted one. A simple question like "what does it mean to hash a password and why would one do that?" or "what is PCI?" would expose his incompetence.
The sort of interview that this guy would do well in is a poorly-conducted one. A simple question like "what does it mean to hash a password and why would one do that?" or "what is PCI?" would expose his incompetence.
Your assuming that the person interviewing him knew what the answer was supposed to be. It sounds like the incompetence goes all the way up and he was probably hired by a businessman who had no idea what PCI even is, much less the details of proper security.
Right. But we're both making the point that "gives a good interview" is contingent on "isn't given a good interview."
What's interesting is that the PCI standard seems to be unclear in this respect. He quotes from the standard:
At the very least, quoting this isn't going to convince someone that passwords should not be decryptable.
8.4 Render all passwords unreadable during transmission
and storage on all system components using strong
cryptography.
At the very least, quoting this isn't going to convince someone that passwords should not be decryptable.
The ambiguity lies in the word 'unreadable'. The key phrase is 'unreadable... using strong cryptography'. Can strong cryptography be reversed by decryption, or is strong cryptography a non-reversible hash?
It's not unclear, and you're right. It has to be encrypted on the wire and in storage. This paragraph does not say you must hash, or must not be able to get the plaintext passwords back.
That may be, but even if PCI doesn't require you to have the passwords unrecoverable, I'm pretty sure that transferring all of them in bulk to someone else via email should violate some part of PCI. Even if that person is an auditor. The potential for abuse is too high.
Transferring them in bulk wouldn't technically be against any of the rules. There are provisions that access to bulk data be limited to those that 'need' it. This is obviously ambiguous and up to interpretation of your auditor. In this case, since the auditor is asking for the data, one would assume they see no issue with it.
Typically communication of secure information to the auditor would need to be encrypted so there is little to worry in terms of a mail administrator having access to the list.
Remember, PCI very frequently ignores practical reality and common sense.
Typically communication of secure information to the auditor would need to be encrypted so there is little to worry in terms of a mail administrator having access to the list.
Remember, PCI very frequently ignores practical reality and common sense.
Actually, hashed password ARE recoverable, technically. Though it would take a VERY long time to brute force the original passwords.
In theory, there are infinitely many passwords that map to the same hash, so not all passwords are recoverable. In practice most passwords will be recoverable (given enough computing power), because there will only be a single 'reasonable' match (you will probably find only one for passwords of reasonable length).
But would you consider that 'recovery' or 'discovery'. To me, recovering passwords means applying an operation to reverse encryption. Brute-forcing passwords is simply guessing, which makes it a discovery.
From the perspective portrayed in this article, the auditor seems misinformed about fundamentals in his industry and his response to being called on this seemed superficial and borderline childish.
It must be difficult to be ridiculed in a public form of your own profession. Not to mention being called stupid and ignorant for misunderstanding something. I hope I always have the humility to admit when I'm wrong, but also have the patience and understanding when other people don't.
It must be difficult to be ridiculed in a public form of your own profession. Not to mention being called stupid and ignorant for misunderstanding something. I hope I always have the humility to admit when I'm wrong, but also have the patience and understanding when other people don't.
A colleague of mine once had to stand one foot on top of the other and chew his own lip when an SI's "security expert" introduced himself as an 'old-school CISSP' just to stop laughing out loud at him.
There are many idiots in the information security industry (I should know, I are one) - we're doing our best to get rid, but more keep showing up.
There are many idiots in the information security industry (I should know, I are one) - we're doing our best to get rid, but more keep showing up.
Can you explain? I have no idea what a CISSP is.
It's full name is the Certified Information Systems Security Professional. There's a good writeup of the problems with CISSP here:
https://infosecisland.com/blogview/15450-My-Canons-on-ISC-Et...
and here:
http://erratasec.blogspot.com/2011/07/ethical-problems-of-ci...
It's often described as 'a mile wide and an inch deep'. However, it's not really the example of broad security knowledge it sets out to be and is considered a joke certification in my niche (technical security).
https://infosecisland.com/blogview/15450-My-Canons-on-ISC-Et...
and here:
http://erratasec.blogspot.com/2011/07/ethical-problems-of-ci...
It's often described as 'a mile wide and an inch deep'. However, it's not really the example of broad security knowledge it sets out to be and is considered a joke certification in my niche (technical security).
Interesting. I've been considering working towards it, but if competent security professionals think it's a joke.... :-)
Quite a few managers and auditors really don't understand the difference between a password hash and a password.
I expect the auditor was asking for password hashes, although he was using the phrase "plaintext passwords". Who knows, but that part of the story may just be a wording misunderstanding.
Not everyone speaks geek, and it's important to know when you're talking to someone who does not ;)
Edit: Not sure why this is being down voted as it is a true statement. I've had to explain what a password hash is on several occasions. And even after that, there was still some misunderstanding/confusion.
I expect the auditor was asking for password hashes, although he was using the phrase "plaintext passwords". Who knows, but that part of the story may just be a wording misunderstanding.
Not everyone speaks geek, and it's important to know when you're talking to someone who does not ;)
Edit: Not sure why this is being down voted as it is a true statement. I've had to explain what a password hash is on several occasions. And even after that, there was still some misunderstanding/confusion.
You're being voted down (not by me, mind you) as it can be interpreted as a defense of the indefensible.
Even with the charitable interpretation of "this guy doesn't know what plaintext means", the overall conclusion "this guy doesn't know what he's doing and is downright dangerous" still stands.
Even with the charitable interpretation of "this guy doesn't know what plaintext means", the overall conclusion "this guy doesn't know what he's doing and is downright dangerous" still stands.
The auditor specifically said "plaintext". He's clearly a moron, but there's no evidence that he somehow thinks plaintext and hashed are the same thing. He specifically said that the passwords should be stored in a "recoverable format for later use". He wants plaintext.
There's still the question of why he would need the password hashes. Assuming he wanted the plaintext passwords to see if they are 'complex' and 'strong,' he would have a hard time telling that from the hashes.
5f4dcc3b5aa765d61d8327deb882cf99 and 05b28d17a7b6e7024b6e5d8cc43a8bf7: Which is a dictionary word and which is a string of punctuation? (I didn't salt :))
5f4dcc3b5aa765d61d8327deb882cf99 and 05b28d17a7b6e7024b6e5d8cc43a8bf7: Which is a dictionary word and which is a string of punctuation? (I didn't salt :))
Please. 16Crack broke these in less than 5 seconds. Plain md5.
05b28d17a7b6e7024b6e5d8cc43a8bf7 = !@#$%^&*()
5f4dcc3b5aa765d61d8327deb882cf99 = password
05b28d17a7b6e7024b6e5d8cc43a8bf7 = !@#$%^&*()
5f4dcc3b5aa765d61d8327deb882cf99 = password
Yep. But they are hex strings that look like (are) hashes, and that's all I need to make the point. Next time I'll use /dev/urandom for that :)
In a real scenarios, use PBKDF2 or bcrypt!
In a real scenarios, use PBKDF2 or bcrypt!
John says the first one is "password", in under a second. I'll wait for a few hours to see if I can get the second one speedily. (By the way, this is why you shouldn't use MD5!)
You're right. I would never advocate using MD5. Use PBKDF2, bcrypt, etc! I just needed some hex strings to make my point about the hashes (hopefully) not being reversible to the actual password, and thus useless for the purposes of seeing if the passwords are 'strong.'
md5 doesn't have anything to with it. sha256 maybe takes 10% longer to compute the hash.
grand-parent's point is presumably that bcrypt (or similar) is a better choice than md5.
Which is useful if you're brute forcing, md5 is also algorithmatically broken AFAIK.
Define broken. It is possible to generate (within reasonable time) two files which have the same md5 sum, which means you shouldn't use it to sign anything somebody else have given you.
On the other hand collisions between two different files are still not something you would ever expect to see in the wild so if you are trying to find duplicated files, then you don't have to worry.
On the other hand collisions between two different files are still not something you would ever expect to see in the wild so if you are trying to find duplicated files, then you don't have to worry.
No, that extra 10% doesn't mean shit against a brute force attack. As others have stated, use PBKDF2 or bcrypt.
> I've had to explain what a password hash is on several occasions.
To security auditors? Who then told you you were wrong and that you should install PCI?
To security auditors? Who then told you you were wrong and that you should install PCI?
"any inventive suggestions for how to troll him [the security auditor] a bit?"
Nothing productive could come of that. This situation is not for your entertainment. Just move along...
Nothing productive could come of that. This situation is not for your entertainment. Just move along...
the Daily WTF - live!
As I live and work in Birmingham, UK, I hope this is all an elaborate hoax.
I haven't been in the industry that long, but I've encountered a few "security professionals"--auditors, penetration testers, etc. All of them have been totally incompetent; they could tell you the definition of, say, a SQL injection attack but had no idea how to really analyze a system. On the other hand, all of the great programmers I've met have had a really good grasp of security. I'm starting to think that if you don't write code, you're not qualified to audit it.
At first, I thought that this was a bogus post...but it seems to be real - or this guy is keeping up with the story.
That security auditor could give a lot of the guys featured on The Daily WTF a good run for their money.
Argument from authority. Nice try.
This sounds more like social engineering to me. My first thought was this was a hacker who took control of an email address from the security firm, and just tried to exploit the weakest link in order to get plaintext passwords.
Apparently, neither of these guys paid attention in high school civics class. Slander involves oral, not written, communication. Libel (not "liable") is the term for a tort involving false and damaging written communication.
Incompetent people in the security field really need to be called out like this.
Incompetent developers generally only hurt the company; incompetent security professionals hurt every single customer as well.
Incompetent developers generally only hurt the company; incompetent security professionals hurt every single customer as well.
If it's in the UK they may be liable under the Data Protection Act, too.
I like the auditor's take on this:
"I see no data protection issues for these requests, data protection only applies to consumers not businesses so there should be no issues with this information."
The Data Protection Act applies to individuals within the EEA trusted circle nations, full stop. Whether or not the information being requested requires individual consent under the DPA is a different point altogether.
"I see no data protection issues for these requests, data protection only applies to consumers not businesses so there should be no issues with this information."
The Data Protection Act applies to individuals within the EEA trusted circle nations, full stop. Whether or not the information being requested requires individual consent under the DPA is a different point altogether.
Dreaming of a world without passwords... Any ideas?
The problem with not having passwords is that it usually requires an authority to distribute keys (and not retain the initialization parameters for that key, cough RSA cough). And that could potentially require a third party having access to information a lot of people don't want. Or trusting the government to generate them -- which opens up another can of worms.
Things like OAuth and/or federated login still rely on a password at some level.
Pick your poison (personally, I wouldn't mind using an RSA Soft Token type technology with federated access requiring token + pin, but that's just me).
Things like OAuth and/or federated login still rely on a password at some level.
Pick your poison (personally, I wouldn't mind using an RSA Soft Token type technology with federated access requiring token + pin, but that's just me).
Then again, after working at startups my whole career, maybe I'm just naive about how messed up the real world is.