Likely the most painful password hashing discussion you'll ever read(scott.arciszewski.me)
scott.arciszewski.me
Likely the most painful password hashing discussion you'll ever read
https://scott.arciszewski.me/misc/Linkedin/Fail.html
15 comments
This summary created more questions than it eliminated.
Nobody has explained this better than Coda Hale:
http://codahale.com/how-to-safely-store-a-password/
(in place of "bcrypt", you can substitute "PBKDF2" [weaker] or "scrypt" [stronger], but all those options are just fine).
Nobody has explained this better than Coda Hale:
http://codahale.com/how-to-safely-store-a-password/
(in place of "bcrypt", you can substitute "PBKDF2" [weaker] or "scrypt" [stronger], but all those options are just fine).
I'm a bit confused. This article says SHA-256 is a general purpose hash algorithm, but wikipedia says it's a cryptographic hash function. Is the wiki article wrong?
Both are correct. A cryptographic hash function is a hash function that is not invertible, resists tampering and collisions, and has other properties. Using cryptographic hash functions in certain cryptographic settings is appropriate. SHA256("My Super Secret Password") is not one of those settings.
SHA256 can be the basis of a scheme to securely store passwords. There are many different names for that:
* key derivation function
* secure password storage scheme
* password scrambler (PHK's term)
* (...)
A secure password storage scheme uses a cryptographic primitive like a cryptographic hash or a cipher to increase the time it takes to decrypt passwords. That's a good thing, because when it takes your user 0.5 seconds to go from "right password -> on disk hash", it'll take a prohibitively long time for an attacker to guess through passwords.
SHA256 can be the basis of a scheme to securely store passwords. There are many different names for that:
* key derivation function
* secure password storage scheme
* password scrambler (PHK's term)
* (...)
A secure password storage scheme uses a cryptographic primitive like a cryptographic hash or a cipher to increase the time it takes to decrypt passwords. That's a good thing, because when it takes your user 0.5 seconds to go from "right password -> on disk hash", it'll take a prohibitively long time for an attacker to guess through passwords.
Okay, that makes sense. SHA-256 for 50 iterations, with salting: is that equivalent to bcrypt in the matter of slowing things down....is it good enough?
> SHA-256 for 50 iterations (...) is it good enough?
If you mean they call SHA256(SHA256(...48 more calls("password" + salt))?
That's not remotely good enough.
http://hashcat.net/oclhashcat/ -- look at the hardware and see for yourself; those values for SHA256 are million hashes per second.
So 50 rounds of SHA256 can be computed in less than a microsecond.
https://twitter.com/hashcat/status/349192539443699713 -- now, Bcrypt.
I'm not sure what their bcrypt difficulty is, but whatever it is: bcrypt makes it five or six orders of magnitude harder. And remember, bcrypt (like good KDFs) is tunable, so you could make that as hard as you want.
EDIT: Some further consideration: http://arstechnica.com/security/2012/12/25-gpu-cluster-crack...
> As a result, the new cluster, even with its four-fold increase in speed, can make only 71,000 guesses against Bcrypt and 364,000 guesses against SHA512crypt.
https://github.com/freedomofpress/securedrop/issues/180 has some more numbers as well.
If you mean they call SHA256(SHA256(...48 more calls("password" + salt))?
That's not remotely good enough.
http://hashcat.net/oclhashcat/ -- look at the hardware and see for yourself; those values for SHA256 are million hashes per second.
So 50 rounds of SHA256 can be computed in less than a microsecond.
https://twitter.com/hashcat/status/349192539443699713 -- now, Bcrypt.
I'm not sure what their bcrypt difficulty is, but whatever it is: bcrypt makes it five or six orders of magnitude harder. And remember, bcrypt (like good KDFs) is tunable, so you could make that as hard as you want.
EDIT: Some further consideration: http://arstechnica.com/security/2012/12/25-gpu-cluster-crack...
> As a result, the new cluster, even with its four-fold increase in speed, can make only 71,000 guesses against Bcrypt and 364,000 guesses against SHA512crypt.
https://github.com/freedomofpress/securedrop/issues/180 has some more numbers as well.
bcrypt makes it five or six orders of magnitude harder.
And its work factor is adjustable at run-time for the time that hardware speed increases.
And its work factor is adjustable at run-time for the time that hardware speed increases.
Not quite-- 50 iterations of `hash = SHA256(hash + salt)`
Any better? (EDIT: Still in process of reading links.)
Any better? (EDIT: Still in process of reading links.)
>(EDIT: Still in process of reading links.)
Sorry, I'll try to stop incessantly editing ;-)
> Any better?
No.
General purpose cryptographic hash functions are designed to be incredibly fast.
http://www.intel.com/content/dam/www/public/us/en/documents/...
But convince yourself. Do you have Ruby? http://pastebin.com/zjLiYt2j
If I bump up your password scheme by 4 orders of magnitude it's still under <.06 seconds on my machine. And that's with a ridiculously inefficient password generator I banged up in ten seconds.
Use bcrypt. Use scrypt. Use PBKDF2. Don't roll your own password stretching function, it's a recipe for disaster.
Sorry, I'll try to stop incessantly editing ;-)
> Any better?
No.
General purpose cryptographic hash functions are designed to be incredibly fast.
http://www.intel.com/content/dam/www/public/us/en/documents/...
But convince yourself. Do you have Ruby? http://pastebin.com/zjLiYt2j
If I bump up your password scheme by 4 orders of magnitude it's still under <.06 seconds on my machine. And that's with a ridiculously inefficient password generator I banged up in ten seconds.
Use bcrypt. Use scrypt. Use PBKDF2. Don't roll your own password stretching function, it's a recipe for disaster.
Thanks very much, all very clear now. I'll try to persuade the people who have a say in the matter...it's tricky when you're not really an expert yourself so you can't fully explain the matter to a perfect extent, and you're stuck saying "but people on the internet said..."
Half the problem is that quite a few standards/compliance groups don't impose the right requirements. PCI audits used to just check for "encrypted passwords."
OWASP does make a decent recommendation, maybe that'll help give you a leg to stand on: https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet...
OWASP does make a decent recommendation, maybe that'll help give you a leg to stand on: https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet...
> PCI audits used to just check for "encrypted passwords.
PCI is a joke. The self-fill audit: "Are you secure?" -- Uh... sure?
The various scanning companies are of differing quality too, adding to the difficulty in getting any real security improvements from the standard across all ecommerce shops.
PCI is a joke. The self-fill audit: "Are you secure?" -- Uh... sure?
The various scanning companies are of differing quality too, adding to the difficulty in getting any real security improvements from the standard across all ecommerce shops.
AFAIK, PCI is an unfunny joke. This is why my pentester friends have told me about it:
> Release v1.0 of super cool app.
> v1.0 gets PCI certified
> Someone discloses a vuln in v1.0
> Release v1.1
> Companies are stuck with v1.0 for at least 90 days until v1.1 can be certified
> Release v1.0 of super cool app.
> v1.0 gets PCI certified
> Someone discloses a vuln in v1.0
> Release v1.1
> Companies are stuck with v1.0 for at least 90 days until v1.1 can be certified
If you mention Thomas Ptacek of Matasano Security (that's who tptacek is), they might be more inclined to listen. Sadly, my name is of little value and if they had heard of it, they'd probably assume malicious intent.
It's a cryptographic hash function for general cryptographic usage (e.g. HMAC), but it's not a password hash function.
To expand upon that...
The problem with md5 and sha and such is that they are fast. bcrypt is useful because it takes a fixed amount of complexity which can be specified by the programmer (and later increased). It also uses salts to thwart rainbow tables. The complexity bit is important, though, because as computers get better, you can make it harder to reverse your passwords.
Also, the biggest reason any of this is a problem is because password reuse is rampant. You are being entrusted with the password to your user's life - their email, their bank account, their facebook account... treat it with respect.
The problem with md5 and sha and such is that they are fast. bcrypt is useful because it takes a fixed amount of complexity which can be specified by the programmer (and later increased). It also uses salts to thwart rainbow tables. The complexity bit is important, though, because as computers get better, you can make it harder to reverse your passwords.
Also, the biggest reason any of this is a problem is because password reuse is rampant. You are being entrusted with the password to your user's life - their email, their bank account, their facebook account... treat it with respect.
True, that's the tech behind it.
Is there any reason most programmers should know any of this ?
The code should look more like this(and should use a standard library):
import Password_protector
P = Password_protector(date_of_first_use) #library upgrades algorithms every once in a while, only a single algorithm available per date. If there's a need for automatically upgraded hashing library should handle covertly.
protected_password = P.protect_password(password)
if P.compare_password(password, protected_password) == True ...
Or something similar. That at least should cover the basics - safely.
Is there any reason most programmers should know any of this ?
The code should look more like this(and should use a standard library):
import Password_protector
P = Password_protector(date_of_first_use) #library upgrades algorithms every once in a while, only a single algorithm available per date. If there's a need for automatically upgraded hashing library should handle covertly.
protected_password = P.protect_password(password)
if P.compare_password(password, protected_password) == True ...
Or something similar. That at least should cover the basics - safely.
I suppose that's Python? The whitespace got chewed up by the comment system; please pastebin/gist it instead.
Quite simply, the reason programmers should know about this (if they're implementing a password system) is because the library you propose often doesn't exist, and when it does exist it's often implemented incorrectly.
In an ideal world, programmers shouldn't have to worry about this stuff. But this is not an ideal world.
In an ideal world, programmers shouldn't have to worry about this stuff. But this is not an ideal world.
I was under the impression SHA256 is valid...it's a cryptographic hash function that hasn't had weaknesses identified yet. 40 iterations of that + salting would suffice, no?
If you're going to use SHA256 for hashing, why not use it in PBKDF2-SHA256?
https://defuse.ca/php-pbkdf2.htm
https://defuse.ca/php-pbkdf2.htm
I do realize bcrypt is arguably the best option, but why is SHA not valid also?
Because it has no work factor. It's uniformly fast, which makes it very easy to brute force. The general-purpose strong cryptographic hashes --- your SHA2s, SHA3s, and BLAKE2s --- are all amenable to fast implementation on GPU.
What you need is a password hash, not merely a cryptographic hash.
What you need is a password hash, not merely a cryptographic hash.
As noted above, this discussion prompted me to investigate my company's methods, a fairly major US website, and I found they're using SHA-256 for 50 iterations with salting. Does that slow things down sufficiently, or is that not comparable to the internal mechanism of bcrypt?
No. Currently, PBKDF2 (which iterates HMAC-SHA1, HMAC-SHA256, etc.) typically is used with a count in the thousands.
I'd recommend upgrading to follow best standards. If you use PHP, scrypt isn't that hard to set up.
https://scott.arciszewski.me/blog/2013/10/php-scrypt-setup
I'd recommend upgrading to follow best standards. If you use PHP, scrypt isn't that hard to set up.
https://scott.arciszewski.me/blog/2013/10/php-scrypt-setup
Company is stuck with ASP.
If you change 50 to (say) 1000 you are getting most of the benefits.
The big problem you are trying to solve is "our password database just leaked; how fast can attackers figure out the passwords?" Say you can run SHA in 1 microsecond, and 1000 iterations in 1 millisecond. Even though it's 1000x slower, that's probably an unnoticeable difference as far as your performance is concerned. (YMMV)
But, anyone trying to brute-force the passwords out of your database will now have to do 1000x the work.
BTW, none of this will help you much if your users choose passwords like "abc" or "password".
(IIRC scrypt lets you blow up in size as well as CPU, which makes GPUs impractical for attacking.)
The big problem you are trying to solve is "our password database just leaked; how fast can attackers figure out the passwords?" Say you can run SHA in 1 microsecond, and 1000 iterations in 1 millisecond. Even though it's 1000x slower, that's probably an unnoticeable difference as far as your performance is concerned. (YMMV)
But, anyone trying to brute-force the passwords out of your database will now have to do 1000x the work.
BTW, none of this will help you much if your users choose passwords like "abc" or "password".
(IIRC scrypt lets you blow up in size as well as CPU, which makes GPUs impractical for attacking.)
Thank you!
Because a lot of passwords aren't particularly strong, and thus are feasibly crackable when you use something like SHA-256. You can test millions or billions of candidates per second on commodity hardware, and you'll be able to recover a lot of passwords.
The purpose of systems like PBKDF2, bcrypt, scrypt, etc. is to intentionally slow the whole thing down so that it takes on the order of one second to test a candidate. This is a minor burden for normal use, since you don't check passwords very often, but it makes it vastly harder for an attacker to crack a password.
Edit: just to put some numbers on this, it looks like high-end GPUs can brute-force about 2 billion SHA-256 hashes per second. That means that an eight-character alphanumeric password could be recovered from a SHA-256 hash in less than a day, on average. Add 20 non-alphanumeric symbols to your character set and you're still only looking at about six days. Eight alpha+num+symbol characters isn't a terribly great password anymore but it still shouldn't be recoverable if your password database leaks.
The purpose of systems like PBKDF2, bcrypt, scrypt, etc. is to intentionally slow the whole thing down so that it takes on the order of one second to test a candidate. This is a minor burden for normal use, since you don't check passwords very often, but it makes it vastly harder for an attacker to crack a password.
Edit: just to put some numbers on this, it looks like high-end GPUs can brute-force about 2 billion SHA-256 hashes per second. That means that an eight-character alphanumeric password could be recovered from a SHA-256 hash in less than a day, on average. Add 20 non-alphanumeric symbols to your character set and you're still only looking at about six days. Eight alpha+num+symbol characters isn't a terribly great password anymore but it still shouldn't be recoverable if your password database leaks.
the SHA algorithms were designed to digest data as fast as possible -- which things like bcrypt purposefully slow it down plus can perform multiple iterations with the output of one iteration feeding as the input to another.
Here is some more info: http://forums.udacity.com/questions/6016855/hashing-password...
Here is some more info: http://forums.udacity.com/questions/6016855/hashing-password...
Thank you for posting this. +1
[deleted]
"Do not use MD5" is repeated often in the discussion.
But are the weaknesses in MD5 relevant to hashing someone's 17 character password?
The problem with MD5 is that it's possible to alter a document such that the forged document still has the same digest. This is a problem for digital signing.
That doesn't seem directly relevant to password hashing. Nobody wants to find some 596 character string which has the same MD5 hash as your 17 character password, even if it does work in its place.
Your password isn't a document that is being digitally signed by its hash for the sake of certifying its authenticity. Nobody is even supposed to store that document!
(Someone tell me where I went wrong, if so.)
But are the weaknesses in MD5 relevant to hashing someone's 17 character password?
The problem with MD5 is that it's possible to alter a document such that the forged document still has the same digest. This is a problem for digital signing.
That doesn't seem directly relevant to password hashing. Nobody wants to find some 596 character string which has the same MD5 hash as your 17 character password, even if it does work in its place.
Your password isn't a document that is being digitally signed by its hash for the sake of certifying its authenticity. Nobody is even supposed to store that document!
(Someone tell me where I went wrong, if so.)
Let's make the following assumption: That no matter what algorithm we pick we're utilising per-user salts. So you'll always need to search the full keyspace (e.g. 6-9 character length passwords).
The reason why MD5, 3DES, and similar algorithms aren't really recommended in 2014 is that they're very fast/efficient on modern hardware. This is due to utilising GPU cores to do the hashing (which wasn't done back when MD5 was popular). This means the "time to crack" is low.
Here is benchmarks of different hashing algorithms (faster is WORSE): http://thepasswordproject.com/oclhashcat_benchmarking
But the really nice part about things like BCrypt and PBKDF2(Rfc2898) is that they accept "rounds." Which is number of repeated iterations over the password+salt. So every year you can respond to faster hardware by adding more rounds (e.g. 2014, have 10K rounds, in 2020 have 15K rounds, etc). If MD5 had supported rounds, it might still be safe to use today with enough iterations.
Even things like BCrypt and PBKDF2 will eventually fall by the wayside (MD5 used to be a more secure alternative to the much faster DES algorithm). But at least rounds (additional iterations to respond to changes in hardware) will keep them relevant for longer.
The reason why MD5, 3DES, and similar algorithms aren't really recommended in 2014 is that they're very fast/efficient on modern hardware. This is due to utilising GPU cores to do the hashing (which wasn't done back when MD5 was popular). This means the "time to crack" is low.
Here is benchmarks of different hashing algorithms (faster is WORSE): http://thepasswordproject.com/oclhashcat_benchmarking
But the really nice part about things like BCrypt and PBKDF2(Rfc2898) is that they accept "rounds." Which is number of repeated iterations over the password+salt. So every year you can respond to faster hardware by adding more rounds (e.g. 2014, have 10K rounds, in 2020 have 15K rounds, etc). If MD5 had supported rounds, it might still be safe to use today with enough iterations.
Even things like BCrypt and PBKDF2 will eventually fall by the wayside (MD5 used to be a more secure alternative to the much faster DES algorithm). But at least rounds (additional iterations to respond to changes in hardware) will keep them relevant for longer.
Just suggesting you hash a password is pretty poor form as it'll be easy generate a rainbow table for, and likely many already exist.
You use a salt to mitigate this but even if you use a salt, it _has_ to be stored somewhere. Once an attacker has it, they'll be able to create rainbow tables with this in mind.
You then move to using a different salt for every password so any rainbow table only works for one result but if you use a fast hashing algorithm, attackers will still be-able generate rainbow tables based on those salts.
So you pick a slower hashing method but in the future, as computer hardware advances, you'll be in the same situation where it's trivial to brute force some rainbow tables.
So you need to keep on top of it and keep updating the stored hashes with something that is non-trivial to brute force, in PHP this gives you these functions: http://php.net/manual/en/ref.password.php.
In reality though, in the event of a breach, all you're doing is buying time for your users to change their passwords before the relevant rainbow tables are generated, so the next important step is to let your users know their hashed passwords were leaked, you followed best practices, but they should update the passwords wherever they used it.
You use a salt to mitigate this but even if you use a salt, it _has_ to be stored somewhere. Once an attacker has it, they'll be able to create rainbow tables with this in mind.
You then move to using a different salt for every password so any rainbow table only works for one result but if you use a fast hashing algorithm, attackers will still be-able generate rainbow tables based on those salts.
So you pick a slower hashing method but in the future, as computer hardware advances, you'll be in the same situation where it's trivial to brute force some rainbow tables.
So you need to keep on top of it and keep updating the stored hashes with something that is non-trivial to brute force, in PHP this gives you these functions: http://php.net/manual/en/ref.password.php.
In reality though, in the event of a breach, all you're doing is buying time for your users to change their passwords before the relevant rainbow tables are generated, so the next important step is to let your users know their hashed passwords were leaked, you followed best practices, but they should update the passwords wherever they used it.
> That doesn't seem directly relevant to password hashing. Nobody wants to find some 596 character string which has the same MD5 hash as your 17 character password, even if it does work in its place.
A SQL injection vulnerability lets you get the MD5 hashed passwords out of a database. As most people don't use 17 character passwords, you can check the MD5 hashes against a rainbow table (you can even do it online) and probably find a privileged user account pretty quick.
I actually broke into a legacy system my predecessor programmed in this manner when I couldn't find documented admin credentials.
A SQL injection vulnerability lets you get the MD5 hashed passwords out of a database. As most people don't use 17 character passwords, you can check the MD5 hashes against a rainbow table (you can even do it online) and probably find a privileged user account pretty quick.
I actually broke into a legacy system my predecessor programmed in this manner when I couldn't find documented admin credentials.
You raise some good points, but the issue with MD5 are largely down to speed.
MD5 is fast - and while computational speed is normally your friend, with cryptography it's your worst enemy. Simply put, the faster the cipher, the more break in attempts a cracker can have per second.
This is why password strengths are often represented in duration (ie how many years would it take to crack). Though those figures are hugely estimated - to the point of being practically worthless.
MD5 is fast - and while computational speed is normally your friend, with cryptography it's your worst enemy. Simply put, the faster the cipher, the more break in attempts a cracker can have per second.
This is why password strengths are often represented in duration (ie how many years would it take to crack). Though those figures are hugely estimated - to the point of being practically worthless.
But other digest algorithms are also fast, so the argument is stupidly strong; it should be "do not use any fast message digest algorithm in its naked form for password hashing".
They might appear fast on a personal web server, but the point behind it is the accumulative effect. When you have millions of possible password permutations to attempt against thousands of passwords hashes (as crackers would when they have a stolen password database) the additional overhead that more computationally expensive ciphers have makes a huge difference.
That's essentially what is meant when people say "use a 'password hashing' function, not a mere cryptographic hash"
Thanks folks; I brushed myself up on this thanks to this discussion.
An interesting feature of some dedicated password hashes is that the salt input can contain encoded parameters, such as the number of rounds or maybe even choice of algorithm. This means that we can "future proof" a hash to some extent, while allowing it to support existing old passwords, while increasing the number of rounds for new passwords. A plain message digest function doesn't have these features, needless to say.
An interesting feature of some dedicated password hashes is that the salt input can contain encoded parameters, such as the number of rounds or maybe even choice of algorithm. This means that we can "future proof" a hash to some extent, while allowing it to support existing old passwords, while increasing the number of rounds for new passwords. A plain message digest function doesn't have these features, needless to say.
You can also encode "version" information (e.g. $2a$ vs $2y$ for bcrypt) so that, if a vulnerability is discovered in a prior implementation, you can re-hash with a safe algorithm transparently without having to go "everyone, reset your passwords!"
MD5 is vulnerable to collision and preimage attacks, which is bad for your password hashes, period. Other functions provide additional computational difficulty in doing exhaustive search of the keyspace. Even more functions prevent hardware implementations from speeding up the process by requiring excessive amounts of memory. This is why everyone suggests using bcrypt or scrypt, or if they're not available, PBKDF2.
How are collisions and preimage attacks relevant for password hashes?
In really simple terms, if either attack is possible, it potentially becomes easier to find the password. If either attack's implementation becomes slightly faster at finding the password than a full search of the key space, this makes cracking easier.
Collisions are the most practical method of attack on password hashes. If you develop a way to find collisions faster than an exhaustive search of the key space, you shorten the time it takes to find a password that matches the hash. Collisions have now been found on MD5 very quickly (on the order of hours).
Preimage attacks are not all that useful when password cracking because you don't need to generate a new message that hashes to the same hash you have now, you're looking for any message that hashes to the hash you have now.
Second preimage attacks aren't relevant because they assume you know the password already and are looking for a new message/hash that match.
Collisions are the most practical method of attack on password hashes. If you develop a way to find collisions faster than an exhaustive search of the key space, you shorten the time it takes to find a password that matches the hash. Collisions have now been found on MD5 very quickly (on the order of hours).
Preimage attacks are not all that useful when password cracking because you don't need to generate a new message that hashes to the same hash you have now, you're looking for any message that hashes to the hash you have now.
Second preimage attacks aren't relevant because they assume you know the password already and are looking for a new message/hash that match.
Collisions for arbitrary hashes? If I give you an MD5 hash, you can find a collision within hours?
What's the smallest message anyone has gotten MD5 collisions on?
The problem is, simply, that cryptographic hashes are very fast and not meant for user-generated passwords. For that, a special sub-class of cryptographically hashes (called Key Derivation Functions) are recommended to make brute-force computation much more difficult.
KDFs are meant to require a great deal more computation (e.g. an example of PBKDF2 might iterate HMAC-SHA256 8,000 times).
KDFs are meant to require a great deal more computation (e.g. an example of PBKDF2 might iterate HMAC-SHA256 8,000 times).
But then "do not use MD5" is still a cluelessly strong statement.
It should be: "do not use any un-iterated message digest function for passwords: password hashing functions iterate a basic hash through numerous rounds, perhaps thousands; and some even encode the number of rounds as part of the salt, so the rounds can be increased in the future without changing the function, so it still recognizes old passwords."
It should be: "do not use any un-iterated message digest function for passwords: password hashing functions iterate a basic hash through numerous rounds, perhaps thousands; and some even encode the number of rounds as part of the salt, so the rounds can be increased in the future without changing the function, so it still recognizes old passwords."
If your 17 characters are distributed equally along just the letters a-z, no one can build a useful codebook during their lifetime.
If your 17 characters are 3 lowercase non-archaic English words, maybe with two digits at the end, then a codebook for them already exists in MD5.
If your 17 characters are 3 lowercase non-archaic English words, maybe with two digits at the end, then a codebook for them already exists in MD5.
Technically, no, it's not relevant. (Things change if you use "only" 13 characters.)
In practice, we don't know how crypto analisys will evolve, and non-broken algorithms are free for use, and just as easy as MD5. So, why do you insist on using a broken one?
In practice, we don't know how crypto analisys will evolve, and non-broken algorithms are free for use, and just as easy as MD5. So, why do you insist on using a broken one?
[deleted]
"either you can use md5 or base64, there are also many other encryption are there.. md5 is very secure because it cannot be decrypted."
I'd say this has to be a joke, but Poe's Law is a tricky thing.
I'd say this has to be a joke, but Poe's Law is a tricky thing.
I'm pretty sure a couple of the posts here are tongue-in-cheek, but most of them reek of cluelessness and tl;dr
>> "MD5 is fine as long as your database doesn't get leaked and is the subject of offline cracking (like adobe)"
Wow. So MD5 is fine as long as it isn't needed to protect the data. Looks like a typical PHP discussion thread: plenty of misconceptions, a mix of of bad and plain stupid advice with a pinch of SQL injectable code. Silly PHP developers.
Wow. So MD5 is fine as long as it isn't needed to protect the data. Looks like a typical PHP discussion thread: plenty of misconceptions, a mix of of bad and plain stupid advice with a pinch of SQL injectable code. Silly PHP developers.
To be fair, there are a class of problems you solve by storing your passwords behind a simple one-way function instead of as plaintext. I was asked earlier this week to audit a system that made the old passwords available to the user on their profile page. Using md5(), for all its shortcomings everyone has mentioned here, would at least make this architecture mistake impossible.
[deleted]
The title of this post is lost on me. What's so painful about this discussion? It's not that I'm stupid, it's just that I'm ignorant and don't know.
It's painful because:
- Bad advice is posted everywhere.
- Good advice is disregarded because nobody reads the preceding posts in the discussion.
- Bad advice is posted everywhere.
- Good advice is disregarded because nobody reads the preceding posts in the discussion.
Warning: This might ruin your day. Also, don't let tptacek see this or he might get very [angry/existential] (select appropriate).
I'm floored by the amount of people with "fancy" titles from supposed "consulting" companies in the link discussion.
The question was "I want to store password[s] in encrypted form." The answer should have been immediately -- "you don't encrypt passwords, you hash passwords using a well known, established, and tested cryptographic hashing algorithm + a salt, damnit!"
The question was "I want to store password[s] in encrypted form." The answer should have been immediately -- "you don't encrypt passwords, you hash passwords using a well known, established, and tested cryptographic hashing algorithm + a salt, damnit!"
> The answer should have been immediately -- "you don't encrypt passwords, you hash passwords using a well known, established, and tested cryptographic hashing algorithm + a salt, damnit!"
Your answer is as wrong as anything posted on that discussion.
The correct answer is that you don't encrypt passwords. You also don't hash passwords with a salt.
Instead, you use a key derivation function such as PBKDF2, bcrypt, or scrypt.
Your answer is as wrong as anything posted on that discussion.
The correct answer is that you don't encrypt passwords. You also don't hash passwords with a salt.
Instead, you use a key derivation function such as PBKDF2, bcrypt, or scrypt.
You should hash passwords with a long random salt. You advocate using bcrypt, and bcrypt requires a salt.
If you are not using a salt, then you are vulnerable to rainbow tables which contain a few hundred thousand of the most common passwords. Even if your hashing algorithm is slow and complex, there are still rudimentary rainbow tables for them. If you haven't used a salt then with one quick scan at least of few of your passwords will be revealed.
The effect of a long random salt is to make every password appear to longer to the hashing algorithm. What's being hashed is the users password plus 20 random characters, and nobody has a rainbow table that contain those 20 random characters. If your hashing algorithm, like bcrypt, accepts the salt as an argument, then that makes things all the easier.
If you are not using a salt, then you are vulnerable to rainbow tables which contain a few hundred thousand of the most common passwords. Even if your hashing algorithm is slow and complex, there are still rudimentary rainbow tables for them. If you haven't used a salt then with one quick scan at least of few of your passwords will be revealed.
The effect of a long random salt is to make every password appear to longer to the hashing algorithm. What's being hashed is the users password plus 20 random characters, and nobody has a rainbow table that contain those 20 random characters. If your hashing algorithm, like bcrypt, accepts the salt as an argument, then that makes things all the easier.
> You should hash passwords (...)
You should not hash passwords. You should never hash a password.
You should use an appropriate key derivation function (bcrypt, scrypt, PBKDF2). Key derivation functions are designed to address the issues you note and many more.
Now, let me give a few choice tptacek quotes on this:
>The socialbookmarkosphere is abuzz with talk of “rainbow tables”(...) This really freaks me out. If the “advanced” pole of your threat model is “rainbow tables”, stop working on your social shopping cart calendar application right now: I can’t trust you with my Reddit karma score, let alone my credit card number.
> Rainbow tables are easy to beat. For each password, generate a random number [... and do what you said to do ...] Cool, huh? Yeah, and Unix crypt —- almost the lowest common denominator in security systems —- has had this feature since 1976. If this is news to you, you shouldn’t be designing password systems. Use someone else’s good one.
---
I'm banging on this drum for a very simple reason: even if you are smart enough to know that salt+general purpose cryptographic hash function isn't good enough, when you say "hash" programmers that aren't smart enough are getting the wrong message.
And if you do believe a "hashing algorithm with a long random salt" (which I translate to "general-purpose cryptographic hash function, such as SHA/RIPEMD + a nonce") is sufficient to store a password... well, let's hope you don't.
You should not hash passwords. You should never hash a password.
You should use an appropriate key derivation function (bcrypt, scrypt, PBKDF2). Key derivation functions are designed to address the issues you note and many more.
Now, let me give a few choice tptacek quotes on this:
>The socialbookmarkosphere is abuzz with talk of “rainbow tables”(...) This really freaks me out. If the “advanced” pole of your threat model is “rainbow tables”, stop working on your social shopping cart calendar application right now: I can’t trust you with my Reddit karma score, let alone my credit card number.
> Rainbow tables are easy to beat. For each password, generate a random number [... and do what you said to do ...] Cool, huh? Yeah, and Unix crypt —- almost the lowest common denominator in security systems —- has had this feature since 1976. If this is news to you, you shouldn’t be designing password systems. Use someone else’s good one.
---
I'm banging on this drum for a very simple reason: even if you are smart enough to know that salt+general purpose cryptographic hash function isn't good enough, when you say "hash" programmers that aren't smart enough are getting the wrong message.
And if you do believe a "hashing algorithm with a long random salt" (which I translate to "general-purpose cryptographic hash function, such as SHA/RIPEMD + a nonce") is sufficient to store a password... well, let's hope you don't.
https://github.com/defuse/password-hashing/issues/39#issueco...
"Therefore all KDF functions are cryptographic hash functions. All hash functions are not KDFs, due to speed."
"Therefore all KDF functions are cryptographic hash functions. All hash functions are not KDFs, due to speed."
"Cryptographic hash" has been used to refer to non-invertable hash functions since the late 70s.
Using that term to refer to a KDF is exceptionally misleading and foolish.
If you gave that answer to those PHP developers, they'd come out thinking MD5 and a salt would Work Just Fine.
Using that term to refer to a KDF is exceptionally misleading and foolish.
If you gave that answer to those PHP developers, they'd come out thinking MD5 and a salt would Work Just Fine.
Point of order: The post you replied to was not "as wrong" as the participants in that forum ;)
A KDF, as your link notes, meets all the properties of a cryptographic hash.
The important property it has that other non-cryptographic hashes do not have is speed (actually, a lack thereof) and (typically) the ability to tune that speed.
If we want "technically correct" points I won't grant them (;-)) because KDFs have a property (slowness) cryptographic hashes do not. It would be technically correct to state that KDFs are strict supersets of cryptographic hashes.
Practically speaking, calling a KDF a cryptographic hash is very harmful because the average programmer doesn't have the understanding to make the distinction. It's important to give them two very clearly separated categories - cryptographic hashes are {SHA1, RIPEMD160, ...} and are FAST, KDFs are {scrypt, bcrypt, ...} and are SLOW - and a clear rule that "you MUST always use a KDF to store passwords".
If you blur the lines between the two, the average programmer will misunderstand and use the wrong tool for the job.
The important property it has that other non-cryptographic hashes do not have is speed (actually, a lack thereof) and (typically) the ability to tune that speed.
If we want "technically correct" points I won't grant them (;-)) because KDFs have a property (slowness) cryptographic hashes do not. It would be technically correct to state that KDFs are strict supersets of cryptographic hashes.
Practically speaking, calling a KDF a cryptographic hash is very harmful because the average programmer doesn't have the understanding to make the distinction. It's important to give them two very clearly separated categories - cryptographic hashes are {SHA1, RIPEMD160, ...} and are FAST, KDFs are {scrypt, bcrypt, ...} and are SLOW - and a clear rule that "you MUST always use a KDF to store passwords".
If you blur the lines between the two, the average programmer will misunderstand and use the wrong tool for the job.
Alright, so, question.
Why would the speed of the hash function make any difference, assuming it's a cryptographic hash? If it's faster, run it for more iterations - if it's slower, run it for less. You pick a maximum time, figure out how many iterations can be run in that time, and go from there. I would have thought that a fast hash function would be better, if anything - as it's potentially already more optimized for current hardware, which means that it's less likely that it will get faster on better hardware.
I always thought it was the memory complexity of the hash function that was the difference, not speed. Makes it harder to run on GPUs / ASICs.
Why would the speed of the hash function make any difference, assuming it's a cryptographic hash? If it's faster, run it for more iterations - if it's slower, run it for less. You pick a maximum time, figure out how many iterations can be run in that time, and go from there. I would have thought that a fast hash function would be better, if anything - as it's potentially already more optimized for current hardware, which means that it's less likely that it will get faster on better hardware.
I always thought it was the memory complexity of the hash function that was the difference, not speed. Makes it harder to run on GPUs / ASICs.
If you’re iterating the hash function until it’s “sufficiently slow", the result is a (one-off, badly-designed) key-derivation function. That’s how they work.
> I'm floored by the amount of people with "fancy" titles from supposed "consulting" companies in the link discussion.
I'm gently curious if black hats will now target some of those companies?
Or are there enough low-hanging fruit available from www searches for them to not worry?
I'm gently curious if black hats will now target some of those companies?
Or are there enough low-hanging fruit available from www searches for them to not worry?
In many lines of work, it's more important to be visible than to be correct.
Yeah, I attempted to do that (politely) with my first post. Though I made a few mistakes (it's password_hash() not hash_password()). As the discussion grew on, I became less nice.
Not to mention that encryption means being able to take some input and generate what appears to be a random sequence of characters,numbers etc, but being able to transform it back to plaintext/original format by using the corresponding decryption key.
Indeed too many people there failed.
Indeed too many people there failed.
I saw that -- but then a bunch of "titles" came in and flooded the discussion with the worst possible advice. I mean, seriously, have these guys been living under a rock? This is (well, should be) common sense by now.
Heh. And they all probably get paid a heck of a lot more than I do too. Oi. :]
wow. just wow.
Then I noticed "PHP" and then I was enlightened.
Then I noticed "PHP" and then I was enlightened.
I'm of the opinion that PHP's reputation for being a "bad language" is due to the sheer volume of idiots who can write a script that doesn't parse error.
I got out of web development in general (I still do it now and then) because there's too many idiots writing bad code, and then clients asking me to fix up bad code ... and then expecting to pay the same rates as the cowboy who wrote the original code.
It's a very popular language. It's got a lot of developers at all levels.
Nope. The reputation is due to the number of such kind of developers who post at the offical bug-tracking software as official developers, and with sufficient capability to direct the design of the standard library API.
I'm sorry, maybe I need more coffee, but I'm having trouble grokking your intended meaning.
It has a low barrier to entry. This tends to welcome really bad developers.
I'm disappointed nobody suggested XOR
/s
/s
I actually saved this before the grand-daddy of bad advice was posted. http://www.cryptofails.com/post/87697461507/46esab-high-qual...
I really want to suggest XORing the password with itself, as a goof, but it seems like the type of thing that will bite me in the ass later in life.
But someone did suggest Base64!
Base64 does have one use: assume you need to store a plaintext password because you need your service to login to another service. (Use oauth, yada yada. Not everybody has it.) base64 coding passwords prevents support staff from trivially seeing and learning passwords when poking about the database. It's not to protect the password from bad people, it's to keep the good people from accidentally learning something they don't want to know.
Multiple someones, including someones with fancy titles.
Why mention md5 at this point, or even SHA1, when BLAKE2 exists? (assuming the people there don't want SHA2 because of slower performance): https://blake2.net/
Nothing in that thread annoyed me as much as this comment does. Yes, BLAKE2 is trendy, and name-dropping is tempting. But BLAKE2 is also not a password hash, and can't safely be used to store authenticators for passwords.
Right; it's a cryptographic hash. Technically there's no such thing as a 'password hash', only password KDFs and their results, which are composed primarily by cryptographic hashes. I think the guy's point was that if you're going to build a password KDF, you might as well use a hash function that's less shitty than MD5 and SHA1.
These people would disagree with you:
https://password-hashing.net/
I was on your side for awhile, advocating that we just teach people about what a KDF is, but my side lost.
https://password-hashing.net/
I was on your side for awhile, advocating that we just teach people about what a KDF is, but my side lost.
The only winning move is not to play.
Well, that's kind of the point. You don't want performance here.
[deleted]
Could a hacker learn the hash key and salting by opening a bunch of accounts on the site before stealing the database, then comparing his known passwords with the hashed values?
Salts are usually dependent on user details (like the username), so knowing your salt doesn't help you with other passwords.
The discussion is great - it shows all of the companies you should absolutely avoid doing business with, as they have no sense of security.
What about this is so bad to begin with? It looks like this is standard industry practice now?
It's linkedin. As expected.
in general anything coming from a php developer should be ignored. even reading official php documentation when it comes to anything concerning security should scare the shit out of grandmothers everywhere.
The documentation has actually been going through a clean-up lately. For example, http://php.net/manual/en/function.crypt.php#refsect1-functio...
Do you have any specific examples of bad security advice in the documentation? I can try to raise hell with the doc maintainers and maybe even suggest an amendment.
Do you have any specific examples of bad security advice in the documentation? I can try to raise hell with the doc maintainers and maybe even suggest an amendment.
I see that MySQL's documentation [1] recommends SHA2:
> Exploits for the MD5 and SHA-1 algorithms have become known. You may wish to consider using one of the other encryption functions described in this section instead, such as SHA2().
Further down:
> The PASSWORD() function is used by the authentication system in MySQL Server; you should not use it in your own applications. For that purpose, consider MD5() or SHA2() instead. Also see RFC 2195, section 2 (Challenge-Response Authentication Mechanism (CRAM)), for more information about handling passwords and authentication securely in your applications.
1: http://dev.mysql.com/doc/refman/5.5/en/encryption-functions....
> Exploits for the MD5 and SHA-1 algorithms have become known. You may wish to consider using one of the other encryption functions described in this section instead, such as SHA2().
Further down:
> The PASSWORD() function is used by the authentication system in MySQL Server; you should not use it in your own applications. For that purpose, consider MD5() or SHA2() instead. Also see RFC 2195, section 2 (Challenge-Response Authentication Mechanism (CRAM)), for more information about handling passwords and authentication securely in your applications.
1: http://dev.mysql.com/doc/refman/5.5/en/encryption-functions....
I may have voted this comment up, but it makes me feel very :(
Related, the only hashing algorithm supported by postgresql for authenticating to the pgsql server is md5.
Related, the only hashing algorithm supported by postgresql for authenticating to the pgsql server is md5.
Let's see:
> If you are at all concerned about password "sniffing" attacks then md5 is preferred. Plain password should always be avoided if possible. However, md5 cannot be used with the db_user_namespace feature. If the connection is protected by SSL encryption then password can be used safely (though SSL certificate authentication might be a better choice if one is depending on using SSL).
Strongly implies that md5sum offers protection from an attacker sophisticated enough to sniff passwords of the network. Snif.
> If you are at all concerned about password "sniffing" attacks then md5 is preferred. Plain password should always be avoided if possible. However, md5 cannot be used with the db_user_namespace feature. If the connection is protected by SSL encryption then password can be used safely (though SSL certificate authentication might be a better choice if one is depending on using SSL).
Strongly implies that md5sum offers protection from an attacker sophisticated enough to sniff passwords of the network. Snif.
Take a look at the docs. MD5 is an option for network transportation of the password - almost the entire world transports it in plain text. It's a "you are doing it wrong if you need this, but it won't hurt if you don't need - so we'll keep it" feature, with the correct recomendation (using SSL) made all around.
Anyway, the password will be stored in plain-text just a few servers away. It has to be. Whatever hash they use, hashing it will be as useless.
Anyway, the password will be stored in plain-text just a few servers away. It has to be. Whatever hash they use, hashing it will be as useless.
The first thing i did after reading the post was to check the official documentation. The docs are explicit about not using MD5 for hashing password. Good on PHP doc maintainers.
In order for a server to use encrypted passwords for authentication, the server must have access to the decryption key -- which means an attacker has access to your decryption key if they can compromise your box. Furthermore, if they don't get your decryption key, but they get your database dump -- they can then just brute your db and decrypt the passwords. They can do this because encryption is, by design, reversible. Encrypting passwords is almost as bad as storing in plain text.
Cryptographically hashing a password is the only way to go. When you cryptographically hash a password, using a known library that has been tested and written by actual cryptographers with passwords/authentication in mind (like bcrypt, etc -- no, sha's and md5's are not safe to use) - you get a one-direction algorithm in that a password goes in, is digested, and a fixed-length hash output comes out. There is no way to reverse this hash back into the original input. Adding salt(s) combats people making rainbow tables against your implementation, as they then would need to know not only which algorithm you are using, how many iterations it does, but also what your salt(s) are (sometimes even varied every so many iterations). This makes it almost impossible for someone to brute your leaked password database.