OpenBSD forks, prunes, fixes OpenSSL(zdnet.com)
zdnet.com
OpenBSD forks, prunes, fixes OpenSSL
http://www.zdnet.com/openbsd-forks-prunes-fixes-openssl-7000028613/
11 comments
It's cool they're doing this cleanup, but overly self-righteous commit messages (http://opensslrampage.org/) on patches like http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/cry... rub me the wrong way (I'm sure comparing a string pointer to '5' / decimal 53 is _exactly_ what the old, ridiculed code meant to do…)
Well, fixing legacy code is generally regarded as crap work and developers need to vent. Still, they're lucky that they're not implementing the actual crypto code, otherwise they'll face much worse public beatdowns.
(To their credit, they did fix this bug 2 hours later: http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/cry... http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/cry... )
That is pretty bad. I can't see how that wouldn't give a new compiler warning that would tell the dev that they probably shouldn't commit that patch.
You may need a few compiler flags turned off to compile OpenSSL ;)
That said, many opensource projects ignore the vast majority of compilation warnings. Many projects do the same.
That said, many opensource projects ignore the vast majority of compilation warnings. Many projects do the same.
This is why I would never use OpenBSD (but yeah, I use OpenSSH). Can't use an OS for people with this type of attitude.
> Some of that is indentation, because we are trying to make the code more comprehensible. 99.99% of the community does not care for VMS support, and 98% do not care for Windows support. They care for POSIX support, so that the Unix and Unix derivatives can run.
Do they not realize that clients also need SSL support? Servers that don't have anyone to talk to them are kind of useless, so at best this is half of solution (unless it really is the year of the linux desktop, of course). There are a ton of applications out there that run on Windows and use OpenSSL for handling SSL connections.
That being said this may be a great replacement for servers, and there really isn't a requirement for there to be "one library to rule them all". This may be a great step towards diversifying the encryption-library ecosystem a bit.
Do they not realize that clients also need SSL support? Servers that don't have anyone to talk to them are kind of useless, so at best this is half of solution (unless it really is the year of the linux desktop, of course). There are a ton of applications out there that run on Windows and use OpenSSL for handling SSL connections.
That being said this may be a great replacement for servers, and there really isn't a requirement for there to be "one library to rule them all". This may be a great step towards diversifying the encryption-library ecosystem a bit.
They're building it first and foremost for OpenBSD. If anybody's interested in using their fork, they'll create a portable version, as they do for OpenSSH, OpenSMTPD, &c.
The act of stripping the library down is worthwhile though: OpenSSL has accumulated a lot of cruft over the years.
The act of stripping the library down is worthwhile though: OpenSSL has accumulated a lot of cruft over the years.
I think restructuring OpenSSL with better modularization could achieve / maintain more compatibility than just ripping things out.
The OpenSSL codebase is already pretty modular, but it's not very clean, and that's its major issue. Needless to say, it could be made even more modular, with the crypo code, X.509 code, the SSL/TLS protocol implementation, &c., entirely separated from one another, but doing that will still benefit from pulling out as much cruft as possible. They're ripping out all that stuff because it's more hard to test stuff that can just go wrong, and when it comes to critical security infrastructure, that's a bad thing to have lying around.
All they really need to do to maintain compatibility is maintain the same API. And they're doing just that. In fact, it's in their best interests to maintain the same API as deviating in any significant way is just going to complicate porting software that uses OpenSSL over to use their implementation. So long as they do that, there's little to worry about.
And maybe some day their fork will end up becoming the dominant SSL/TLS implementation, much as OpenSSH became the dominant SSH implementation.
All they really need to do to maintain compatibility is maintain the same API. And they're doing just that. In fact, it's in their best interests to maintain the same API as deviating in any significant way is just going to complicate porting software that uses OpenSSL over to use their implementation. So long as they do that, there's little to worry about.
And maybe some day their fork will end up becoming the dominant SSL/TLS implementation, much as OpenSSH became the dominant SSH implementation.
Ripping out FIPS and Windows Support based on their use case isn't ripping out cruft. I think the codebase needs attention (I am personally way to comfortable with it), but a dramatic fork instead of collaboration isn't necessarily best for the open source community as a whole...
If they can't maintain it, then it certainly is cruft from their point of view.
Keep in mind that first and foremost they're doing this for OpenBSD: all other platforms are secondary.
Keep in mind that first and foremost they're doing this for OpenBSD: all other platforms are secondary.
Ack - looks like I posted something quite similar.
Maybe Theo is thinking only about the "OpenBSD community".
Maybe Theo is thinking only about the "OpenBSD community".
Yes. This is an OpenBSD fork. It will run on OpenBSD.
That said, later on folks can create compatibility layers (like OpenSSH) to make it work elsewhere. It's better to keep that code outside the core anyway.
That said, later on folks can create compatibility layers (like OpenSSH) to make it work elsewhere. It's better to keep that code outside the core anyway.
I don't necessarily disagree with the changes, overall. I think there would be a benefit from both companies and projects that leverage OpenSSL to have contributed back to it both in terms of $ as well as change sets to make it better instead of another fork of a library.
(I realize the issues previously expressed about the state of change within the OpenSSL project, as well as the lack of general give back from the many people that use it).
(I realize the issues previously expressed about the state of change within the OpenSSL project, as well as the lack of general give back from the many people that use it).
I'm finding the unjustified down voting of the benefits of a FIPS compliant mode interesting. I, personally, don't really care. So open question -
Those down voting, have you actually dealt with getting a product through government certification? Specifically in security would be more useful.
Those down voting, have you actually dealt with getting a product through government certification? Specifically in security would be more useful.
Don't worry HN turned into this down voting community where if your opinion does not meet with what people want to see you get down voted. It is the best way to get rid of different views so after a while just those guys hang around who's view is the same. RIP HN
It may still run on Windows, just on top of a POSIX compatibility layer, no?
I have to say -- and I am not a system programmer -- this shit is pretty hilarious. E.g. Multiple unnecessary and broken implementations of common library functions (printf!).
A quote in there says that customers don't care for FIPS support. I hope OpenBSD is not planning on dropping FIPS support from LibreSSL -- it seems like that would be a huge mistake. Government agencies are among the many that are open to SSL vulnerabilities. Our tax, financial, SSN, and healthcare information are all open to vulnerabilities in buggy code. Dropping FIPS support means that government agencies whose security we (as consumers) care a whole lot about will not be benefiting from the massive improvements that the OpenBSD community hopes to bring about.
Since when does OpenBSD (a Canadian project with a developer group all over the planet) need to care about some certification and standard bullshit by the government of the United States of America?
What have the USA done for us lately? Oh yes, fought a long battle against cryptography in the hands of world citizens, spied on everyone and their dog, canceled OpenBSD funding, started multiple illegal wars and threatened every single country into submission (except for the one with nukes pointed at the White House).
Does it get any more selfish? Bah.
What have the USA done for us lately? Oh yes, fought a long battle against cryptography in the hands of world citizens, spied on everyone and their dog, canceled OpenBSD funding, started multiple illegal wars and threatened every single country into submission (except for the one with nukes pointed at the White House).
Does it get any more selfish? Bah.
Even if OpenBSD did not drop the FIPS code, LibreSSL would not be FIPS compliant because that involves paying large amounts of money for certification, which with their limited resources, and the way FIPS nessisarily constrains development, they would be unwilling to do.
It appears doubtful that FIPS will be restored. See the commit: http://marc.info/?l=openbsd-cvs&m=139775148919571&w=2
"Forcible certification conflicts with the goals of a free software project."
"Forcible certification conflicts with the goals of a free software project."
I wonder about if the NSA, who was supposedly aware of the bugs (and likely others), had to leave other US gov agencies open to the exploits because they would otherwise blow their cover? Or might they have had internal custom OpenSSL (or just didn't use OpenSSL)?
Somehow, I don't see removing FIPS or Windows support being in the best interest of the community as a whole. For startups wanting to have a product that requires some level of FIPS support, starting with a decent base makes things easier.
Windows support? There are more than a couple of applications out there running on Windows that make use of OpenSSL.
Theo's quote -- "99.99% of the community does not care for VMS support, and 98% do not care for Windows support." -- is, at least to me, narrow minded in the scope of what he defines as "community" -- unless he specifically means the OpenBSD community.
Windows support? There are more than a couple of applications out there running on Windows that make use of OpenSSL.
Theo's quote -- "99.99% of the community does not care for VMS support, and 98% do not care for Windows support." -- is, at least to me, narrow minded in the scope of what he defines as "community" -- unless he specifically means the OpenBSD community.
I'm sure they have all the best intentions for sharing the wisdom they learn here.
But right now OpenSSL is a VERY VERY messy and dangerous library.
The OpenBSD devs need to get a working implementation, that catches up on 15 years of refactoring and clarity in (optimistically) 2 months -- or risk the bulk of the world's sensitive information leaking yet again (perhaps even sooner, depending on what is discovered!)
It seems that with the kind of things they're finding, there are a lot lesser understood vulnerabilities in the code.
With the kind of standardization they're working on, a Windows port will be a LOT easier and a LOT simpler after they're finished. Best let them make haste, tear out everything that's leaky, unclear, or dangerous -- and with that will come a lot of simplification and fixes for the entire implementation.
Once it's clean, focus on a Ports. The code is too bloated to even consider ports right now. Quite frankly you might even be safer using Microsoft's SSL/TLS and other crypto implementation for the time being... (Only time will tell!)
But right now OpenSSL is a VERY VERY messy and dangerous library.
The OpenBSD devs need to get a working implementation, that catches up on 15 years of refactoring and clarity in (optimistically) 2 months -- or risk the bulk of the world's sensitive information leaking yet again (perhaps even sooner, depending on what is discovered!)
It seems that with the kind of things they're finding, there are a lot lesser understood vulnerabilities in the code.
With the kind of standardization they're working on, a Windows port will be a LOT easier and a LOT simpler after they're finished. Best let them make haste, tear out everything that's leaky, unclear, or dangerous -- and with that will come a lot of simplification and fixes for the entire implementation.
Once it's clean, focus on a Ports. The code is too bloated to even consider ports right now. Quite frankly you might even be safer using Microsoft's SSL/TLS and other crypto implementation for the time being... (Only time will tell!)
Except, for example, FIPS support mandates availability of operating modes which are broken by design.
There are ways of crafting the software that could make sure the broken operating modes aren't necessarily part of the FIPS build profile.
More than one company I have worked at built on top of OpenSSL and went for FIPS compliance. Sometimes, if you want to be able to qualify for contracts that require things like FIPS, building off a base is better than starting from complete scratch.
More than one company I have worked at built on top of OpenSSL and went for FIPS compliance. Sometimes, if you want to be able to qualify for contracts that require things like FIPS, building off a base is better than starting from complete scratch.
I can't help feeling a bit bad for the OpenSSL team in all of this. For years they did a thankless yeoman's work with insufficient resources and now that people are recognizing the work they did it's almost uniformly negative.
Of course you can argue that they brought it upon themselves, but man would I love to watch a 60-minute documentary on how OpenSSL came to be, the people behind it, and how it went wrong.
Of course you can argue that they brought it upon themselves, but man would I love to watch a 60-minute documentary on how OpenSSL came to be, the people behind it, and how it went wrong.
This seems like a good time to mention that 50% of the problem is the overcomplexity of TLS itself not necessairly OpenSSL...
Cryptographic algorithm usually are pretty straight forward when you look at them.
Beside good verifiable coding practice to have a quality implementation, it seems that the issue is with the many side channels attack that needs to be obfuscated and make the whole thing a lot harder to maintain and verify.
So I'm guessing the issue is probably more about the very badly engineered protocols with blurry designs and too many bells and whistles rather than with the basic cryptographic building blocks?
Beside good verifiable coding practice to have a quality implementation, it seems that the issue is with the many side channels attack that needs to be obfuscated and make the whole thing a lot harder to maintain and verify.
So I'm guessing the issue is probably more about the very badly engineered protocols with blurry designs and too many bells and whistles rather than with the basic cryptographic building blocks?
This is a quote from a OpenBSD developer that sums up the work they are doing nicely:
"Of necessity, some crypto code will be a bit scary to look at. anything that has to do regular jobs, like building a pathname, in a security library, should be a thing of beauty, not something that makes you want to throw up in your mouth."
"Of necessity, some crypto code will be a bit scary to look at. anything that has to do regular jobs, like building a pathname, in a security library, should be a thing of beauty, not something that makes you want to throw up in your mouth."
Cryptographic algorithm usually are pretty straight forward when you look at them.
Take a look at the hand-optimized for speed immplementations in assembly sometime.
Take a look at the hand-optimized for speed immplementations in assembly sometime.
They're still just doing a bunch of math; the complexity is in I/O and protocol interactions, not the algorithms themselves.
The AES is a good example of a cryptographic algorithm that is rather not straightforward to implement in an efficient and safe manner.
AES is defined on a 4x4 byte state, and every operation (even the sbox) is defined over GF(2^8). However, fast implementations resemble very little of that definition, and end up being a large number of xor and (large) table lookups. This is a much faster way to do it, on general-purpose machines, but it often leaves you open to cache-timing attacks.
So to come up with a fast and secure AES implementation you need to transpose the state in a way that resembles processing 8 blocks in parallel, and emulate hardware-level bitwise arithmetic on software. This is a so-called bitsliced implementation. By this point the code is much more complex and tricky than the original specification, and it will only actually be fast for parallel modes of operation such as CTR, leaving CBC hanging. Thankfully processors now implement AES round instructions, which simplifies secure implementations, but makes portable implementations difficult.
Cryptographers have learned from this state of affairs, though. New algorithms being created (like Keccak, aka SHA-3) now try harder to make the straightforward implementation secure and reasonably fast by default.
Public-key algorithms, on the other hand, are always tricky to implement, even if the arithmetic itself is not very complicated. One of the appeals of curve25519 (and other so-called "safe curves") is precisely that it is rather easy (but still not trivial) to implement them in a safe manner.
AES is defined on a 4x4 byte state, and every operation (even the sbox) is defined over GF(2^8). However, fast implementations resemble very little of that definition, and end up being a large number of xor and (large) table lookups. This is a much faster way to do it, on general-purpose machines, but it often leaves you open to cache-timing attacks.
So to come up with a fast and secure AES implementation you need to transpose the state in a way that resembles processing 8 blocks in parallel, and emulate hardware-level bitwise arithmetic on software. This is a so-called bitsliced implementation. By this point the code is much more complex and tricky than the original specification, and it will only actually be fast for parallel modes of operation such as CTR, leaving CBC hanging. Thankfully processors now implement AES round instructions, which simplifies secure implementations, but makes portable implementations difficult.
Cryptographers have learned from this state of affairs, though. New algorithms being created (like Keccak, aka SHA-3) now try harder to make the straightforward implementation secure and reasonably fast by default.
Public-key algorithms, on the other hand, are always tricky to implement, even if the arithmetic itself is not very complicated. One of the appeals of curve25519 (and other so-called "safe curves") is precisely that it is rather easy (but still not trivial) to implement them in a safe manner.
In my experience, people tend to wildly over- or under- estimate the complexity of anything they don't understand.
Here you go.
http://www.opensource.apple.com/source/OpenSSL098/OpenSSL098...
Note: it's Perl.
http://www.opensource.apple.com/source/OpenSSL098/OpenSSL098...
Note: it's Perl.
Protecting against sidechannels doesn't mean the code has to be obfuscated. You do need to reimplement some things that are usually pretty simple (like string comparison) but it does not mean you can't abstract that away into a library like anything else.
OpenSSL is beyond repair. We need a new clean start, sane design and rock solid implementation with principles and following best practices.
"We need a new clean start, sane design and rock solid implementation with principles and following best practices."
We need everything after the comma, sure, but there's no particular logical connection with the bit before your comma. A drop-in, API-compatible replacement (or at least one where the bits of the API thrown out are carefully chosen and generally unused) with OpenSSL that isn't scary and has been vetted is by far the fastest way to get to the second bit. This is probably the best thing that could have happened to OpenSSL, and I predict a decent chance this will be the dominant branch in under a year.
We need everything after the comma, sure, but there's no particular logical connection with the bit before your comma. A drop-in, API-compatible replacement (or at least one where the bits of the API thrown out are carefully chosen and generally unused) with OpenSSL that isn't scary and has been vetted is by far the fastest way to get to the second bit. This is probably the best thing that could have happened to OpenSSL, and I predict a decent chance this will be the dominant branch in under a year.
While I can agree with that, how do you get people to use your new awesome crypto library? LibreSSL (the name of the OpenBSD fork, it seems) API should stay OpenSSL-compatible, making it mostly a drop-in replacement.
Also, I think that an OpenSSL cleanup is needed even if we get a new awesome library too. We can't drop OpenSSL overnight, and as you said it's bad. So this at least make it less bad, until there's something we can switch to.
Also, I think that an OpenSSL cleanup is needed even if we get a new awesome library too. We can't drop OpenSSL overnight, and as you said it's bad. So this at least make it less bad, until there's something we can switch to.
Nobody said we can drop OpenSSL over night. If you start now in 2 years we might see the results.
A clean start should include a better license. Either accept that SSL libraries _absolutely_must_ be update-able and auditable to be sure, which for myself implies nothing less than LGPLv3; or go for second-best and have just a plain old no-advert BSD license that is suitable to the GPL-has-cooties privatized crowd.
Like NaCl? http://nacl.cr.yp.to/
NaCl addresses a different problem than OpenSSL. NaCl is great if you have no legacy or compatibility requirements, and it should be the only crypto library most web and mobile developers ever come within a mile of. But it's not a workable replacement for OpenSSL.
What about this from the libsodium web site?
"NaCl ships with constructions that were just prototypes, and that shouldn’t be used any more."
What does "shouldn't be used any more" mean? Certainly not security, or djb would have released an update. Compatibility with other products? Something else?
"NaCl ships with constructions that were just prototypes, and that shouldn’t be used any more."
What does "shouldn't be used any more" mean? Certainly not security, or djb would have released an update. Compatibility with other products? Something else?
No, IIRC Bernstein improved his public key signing system, and libsodium uses the new version. Neither construction is useful for standard TLS.
For per packet encryption.
For authentication, there is CurveCP.
People seem confident with OpenSSH's authentication mechanism. Why not use that?
At some point one has to trust that the IP address one is sending/retrieving data to/from is the correct one. That's easier said than done if some host wants to keep changing its IP address every few days.
The SSL PKI scheme (the SSL approach to authentication), as implemented for public websites, is not much of a confidence-builder, IMO. Opinions may differ.
If websites maintained consistent IP addresses and we could authenticate these machines using OpenSSH keys, I would be more willing to believe we could verify their "authenticity".
For authentication, there is CurveCP.
People seem confident with OpenSSH's authentication mechanism. Why not use that?
At some point one has to trust that the IP address one is sending/retrieving data to/from is the correct one. That's easier said than done if some host wants to keep changing its IP address every few days.
The SSL PKI scheme (the SSL approach to authentication), as implemented for public websites, is not much of a confidence-builder, IMO. Opinions may differ.
If websites maintained consistent IP addresses and we could authenticate these machines using OpenSSH keys, I would be more willing to believe we could verify their "authenticity".
Exactly. The problem with SSL/TLS that it relies on a broken promise with the CAs, and as we have seen several times it is extremely easy to exploit. Hello rogue CAs.
IMO, Putting libre in front of any project name is a surefire way to loose enterprise any and all enterprise / corporate acceptance.
QAWSEDRTYGFRDESWAQSWEDSWAQ