Libsodium: A modern, portable, easy to use crypto library(github.com)
github.com
Libsodium: A modern, portable, easy to use crypto library
https://github.com/jedisct1/libsodium
44 comments
To be honest maintaining bindings for a whole library is kind of a bear, so many people who do this tend to stick to the most used high-level interfaces. Even for my library, which has fewer than 50 functions.
If you use a sufficiently narrow subset, my advice would be to do your bindings yourself. It’s thankless work, but it’s work you control, and you’ll be able to tailor your side to your liking.
If you use a sufficiently narrow subset, my advice would be to do your bindings yourself. It’s thankless work, but it’s work you control, and you’ll be able to tailor your side to your liking.
What command line tool do you recommend for file encryption and signatures by the end user?
Right now the most established are Age and Signify, respectively. I have no experience with them, but they’re reputable, and what little I saw was good.
The `dryoc' crate gives tit for tat compatibility with libsodium using native rust primitives.
Libsodium has been around for a while, so probably the reason it was posted is that version 1.0.19 was just released: https://github.com/jedisct1/libsodium/releases/tag/1.0.19-RE...
Updated NuGet and Swift packages are going to be uploaded soon.
AEGIS-128X and 256X are not there yet, but if you need them, they are available in libaegis: https://github.com/jedisct1/libaegis
All the code from libaegis will eventually be merged into libsodium, including the incremental update API which is especially useful for TLS.
Updated NuGet and Swift packages are going to be uploaded soon.
AEGIS-128X and 256X are not there yet, but if you need them, they are available in libaegis: https://github.com/jedisct1/libaegis
All the code from libaegis will eventually be merged into libsodium, including the incremental update API which is especially useful for TLS.
If I recall correctly Libsodium has an official web site: https://doc.libsodium.org/
No need to link to the repo aggregator.
No need to link to the repo aggregator.
> repo aggregator
Love it. Has a bit of passive aggressive pizzazz.
Love it. Has a bit of passive aggressive pizzazz.
This is arguably the safest library you can use if you are adopting crypto. The documentation is a delight to read and the abstractions make it difficult to make mistakes.
Fun fact, the author (Frank Denis) is an incredibly talented photographer[1].
[1]: https://prettysimpleimages.format.com/
Fun fact, the author (Frank Denis) is an incredibly talented photographer[1].
[1]: https://prettysimpleimages.format.com/
> This is arguably the safest library you can use if you are adopting crypto. The documentation is a delight to read and the abstractions make it difficult to make mistakes.
Its Daniel J Bernstein's world, we merely live in it:
- http://www.metzdowd.com/pipermail/cryptography/2016-March/02...
- https://cr.yp.to/highspeed/naclcrypto-20090310.pdf
Its Daniel J Bernstein's world, we merely live in it:
- http://www.metzdowd.com/pipermail/cryptography/2016-March/02...
- https://cr.yp.to/highspeed/naclcrypto-20090310.pdf
What a fantastic essay. I’ve stolen it and submitted it as a new story.
Poor Colin, always having to live in the shadow of the one critical Tarsnap failure due to a missing ++.
By the way, what does “oth” stand for in your profile? You’ve made the most extensive tagged list or HN users I’ve ever seen. Thanks for that.
Poor Colin, always having to live in the shadow of the one critical Tarsnap failure due to a missing ++.
By the way, what does “oth” stand for in your profile? You’ve made the most extensive tagged list or HN users I’ve ever seen. Thanks for that.
"other"?
Oh it's Peter Gutmann of the Gutmann method!
https://en.wikipedia.org/wiki/Gutmann_method
https://en.wikipedia.org/wiki/Gutmann_method
Very interesting, thanks for the links.
Fantastic documentation, very easy to use and it has bindings for pretty much every language you can imagine.
Yep, I used a js wrapper around it one time in a pet project involving e2ee. The creator(s) of libsodium seem to understand the importance of security features being easy to use and hard to misuse, which sounds obvious but is often not prioritized enough.
This guy is amazing. He might be the user jedisct1 above.
He has implemented a WIDE range of crypto! At the same time, he a professional photographer!
He has implemented a WIDE range of crypto! At the same time, he a professional photographer!
Not really. Its memsetter is unsafe still
Libsodium is an amazing crypto library.
At my previous job, we used libsodium as the basis for all our crypto on web, iOS and android. This after a myriad of subtle problems and inconsistencies using multiple libraries for RSA/AES.
Libsodium is pretty opinionated, which is good. The only thing that actually came close in dev experience was google’s ~~Think~~ Tink library, but that isn’t mature on web and probably never will be.
It’s also hella fast.
Edit: corrected library name
At my previous job, we used libsodium as the basis for all our crypto on web, iOS and android. This after a myriad of subtle problems and inconsistencies using multiple libraries for RSA/AES.
Libsodium is pretty opinionated, which is good. The only thing that actually came close in dev experience was google’s ~~Think~~ Tink library, but that isn’t mature on web and probably never will be.
It’s also hella fast.
Edit: corrected library name
The library is Google Tink (just in case someone wants to look it up)
I can fully recommend Tink, especially when libsodium isn’t available. Although its documentation isn’t very good, it‘s build to be hard to misuse. You still need some cryptographic knowledge though, as it isn’t that opinionated as libsodium.
Tink is pretty yuck, needs a JSON parser just to load keys.
List of authors is crazy.
https://raw.githubusercontent.com/jedisct1/libsodium/master/...
https://raw.githubusercontent.com/jedisct1/libsodium/master/...
When doing symmetric encryption you usually need a nonce or IV, which is also sent to the other party along with the ciphertext and authentication tag. Why does the API for libsodium allow you to specify your own nonce and keeps it separate from the ciphertext? The function crypto_secretbox_easy includes the authentication tag in the ciphertext, but you still have to provide the nonce yourself and it is not included in the ciphertext. Wouldn't it be easier still if the nonce was generated within this function and also added to the ciphertext?
Many protocols define how the nonce must be computed.
And the nonce is not sent explicitly; the different parties compute it themselves the same way they agree on a shared secret.
When a ciphertext is received, the recipient knows what the nonce is expected to be, instead of having to trust an arbitrary one.
In the context of a stream of messages, it binds the nonce to a position in the stream, allowing detection of frames that have been replayed, reordered or lost.
But when the intent is to send independent messages, the nonce should be included with the ciphertext.
For constructions with a short nonce size, the ability to choose the nonce is also very useful to improve the security bounds, by using a key derivation function to derive a new key and nonce from a larger nonce.
By the way, for streams, libsodium has the `crypto_stream_*()` functions that greatly simplifies the implementation of protocols and file encryption.
And the nonce is not sent explicitly; the different parties compute it themselves the same way they agree on a shared secret.
When a ciphertext is received, the recipient knows what the nonce is expected to be, instead of having to trust an arbitrary one.
In the context of a stream of messages, it binds the nonce to a position in the stream, allowing detection of frames that have been replayed, reordered or lost.
But when the intent is to send independent messages, the nonce should be included with the ciphertext.
For constructions with a short nonce size, the ability to choose the nonce is also very useful to improve the security bounds, by using a key derivation function to derive a new key and nonce from a larger nonce.
By the way, for streams, libsodium has the `crypto_stream_*()` functions that greatly simplifies the implementation of protocols and file encryption.
In addition to what sibling comments said, keeping stuff separate helps making the C API a bit more explicit and easier to intuit out of the box. For instance: https://monocypher.org/manual/aead
Yet another reason to let users specify the nonce is that portability stops me from accessing an RNG. Can’t produce random nonces without it, so I have to pass the burden to the user. (On the other hand, I get to run on tiny microcontrollers that don’t even have `malloc()`).
---
Speaking of streaming encryption, I have that too now, and unlike libsodium it’s compatible with the one-shot interface. Like libsodium, you only need to provide your nonce once.
void
crypto_aead_lock(
uint8_t *cipher_text,
uint8_t mac [16],
const uint8_t key [32],
const uint8_t nonce[24],
const uint8_t *ad, size_t ad_size,
const uint8_t *plain_text, size_t text_size);
Yet another reason to let users specify the nonce is that portability stops me from accessing an RNG. Can’t produce random nonces without it, so I have to pass the burden to the user. (On the other hand, I get to run on tiny microcontrollers that don’t even have `malloc()`).
---
Speaking of streaming encryption, I have that too now, and unlike libsodium it’s compatible with the one-shot interface. Like libsodium, you only need to provide your nonce once.
The docs have a page on encrypting multiple messages[0]. In it they use a single random IV and then increment it for each message. Under this scheme you only need to transmit or negotiate the IV once.
If you use something like TLS, The key exchange is expanded to produce an IV. Then each record has an incremented number[1] and this number is mixed with the IV to form a unique IV[2] for the message.
[0]: https://doc.libsodium.org/secret-key_cryptography/encrypted-... [1]: https://www.rfc-editor.org/rfc/rfc8446#section-5.3 [2]: https://www.rfc-editor.org/rfc/rfc8446#appendix-E.2
If you use something like TLS, The key exchange is expanded to produce an IV. Then each record has an incremented number[1] and this number is mixed with the IV to form a unique IV[2] for the message.
[0]: https://doc.libsodium.org/secret-key_cryptography/encrypted-... [1]: https://www.rfc-editor.org/rfc/rfc8446#section-5.3 [2]: https://www.rfc-editor.org/rfc/rfc8446#appendix-E.2
I agree. If you look at the Swift version of the library, it will generate and prepend the nonce for you automatically (if you use the right overloaded method). It will also strip it and use it during decryption.
My guess is they wanted to maintain compatibility with NaCl.
My guess is they wanted to maintain compatibility with NaCl.
That would probably be good for the overwhelming majority of users.
More rarely, you want to save space/reduce overhead by using a deterministic IV scheme. Either all 0 because you picked a random key and you _really_ care about space saving, or you're very very sure you can safely count IVs (0, 1, 2, ..) without ever reusing a single one.
In those rare cases you don't want to transmit the IV, you just know what the value is supposed to be on both sides. More complexity and danger, but a small percentage of users get to save a few bytes.
More rarely, you want to save space/reduce overhead by using a deterministic IV scheme. Either all 0 because you picked a random key and you _really_ care about space saving, or you're very very sure you can safely count IVs (0, 1, 2, ..) without ever reusing a single one.
In those rare cases you don't want to transmit the IV, you just know what the value is supposed to be on both sides. More complexity and danger, but a small percentage of users get to save a few bytes.
A important benefit is that it keeps the function hermetic. Reproducibility is very important for fuzzing and debugging purposes. If the function lets me provide the nonce, I can use my own CSPRNG to generate it, and ensure that my program executes deterministically if fed the same random seed.
Nonce could be an implicit counter in which case it's not transmitted.
Or perhaps use a crypto library that is actually formally proven correct:
https://project-everest.github.io/
https://project-everest.github.io/
Not to be confused with Sodium, the Minecraft rendering engine rewrite: https://modrinth.com/mod/sodium
How do C (C99) manage and use libraries from the web?
Do I just download the header files and use it that way, or is there a package manager for this?
In that case do I manage updates by removing the old header file and download the new version?
Do I just download the header files and use it that way, or is there a package manager for this?
In that case do I manage updates by removing the old header file and download the new version?
Generally you would use the operating system's package manager to install both the headers and the library. On distributions like Debian the library would be in a package called `libsodium` and the headers in `libsodium-dev`.
Once the files are on the system, you use command-line switches to tell the compiler where to look for headers and which libraries to link to. Libraries are often loaded dynamically when the binary is started, so they are not actually included in the file.
Because maintaining the correct switches is a bit of a pain, there are tools like `pkg-config` to generate them for you. And there are tools like `autoconf` and `automake` to generate build scripts to orchestrate those tools.
In libraries it's pretty common for APIs to a library to be fairly stable, so the function signatures generally don't change very much in the headers. This is also why your application can be dynamically linked to different versions of a library.
There are obviously limitations and that's when people come into the equation. For example, OS distribution maintainers juggle these dependencies for you.
Once the files are on the system, you use command-line switches to tell the compiler where to look for headers and which libraries to link to. Libraries are often loaded dynamically when the binary is started, so they are not actually included in the file.
Because maintaining the correct switches is a bit of a pain, there are tools like `pkg-config` to generate them for you. And there are tools like `autoconf` and `automake` to generate build scripts to orchestrate those tools.
In libraries it's pretty common for APIs to a library to be fairly stable, so the function signatures generally don't change very much in the headers. This is also why your application can be dynamically linked to different versions of a library.
There are obviously limitations and that's when people come into the equation. For example, OS distribution maintainers juggle these dependencies for you.
For package-manager-installed libraries in mainstream distros, you usually don't have to worry about the -I or -L flags, or deal with pkg-config, since the libraries are installed in places that are already under the compiler's default search paths. So the steps can be reduced to:
1. apt-get install libsodium libsodium-dev
2. #include "sodium.h" where needed
3. Add -lsodium flag to gcc/clang linking command
1. apt-get install libsodium libsodium-dev
2. #include "sodium.h" where needed
3. Add -lsodium flag to gcc/clang linking command
Someone more experienced at C can jump in and correct me but from my understanding you're correct, you download the files. You could use git to create subrepos for dependencies in a directory in your project, too. When I've asked this question before those were the answers I got.
I saw a project called conan.io, looked interesting.
I know Microsoft created vcpkg.io aswell.
I’m too used to Ruby and how gems are installed, I think I’m looking for a similar experience in C where you find the lib you want, install it and include it in your code.
I’m too used to Ruby and how gems are installed, I think I’m looking for a similar experience in C where you find the lib you want, install it and include it in your code.
Yeah I looked at conan and you can specify git projects to install which is nice. I expect vcpkg to be great for a Windows / Visual Studio setup but I'm on Mac/Linux so haven't really looked. Even with Conan I've not found something as nice as gems/pip/npm for ease of use and it's one of the reasons I end up looking at Rust - because cargo is so familiar to me from a Python background.
Edit: I saw a project recently written in C++ and they had subrepos in a `deps/` directory and then linked to those, so it was all manually managed. If I were to make a new project in C++ I think that's how I'd go too.
Edit: I saw a project recently written in C++ and they had subrepos in a `deps/` directory and then linked to those, so it was all manually managed. If I were to make a new project in C++ I think that's how I'd go too.
For C/C++ projects that use meson as the build system, there is an excellent way to manage (some, supported) dependencies:
https://mesonbuild.com/Wrapdb-projects.html
https://mesonbuild.com/Wrap-dependency-system-manual.html
meson will download and build the libraries automatically and give you a variable which you pass as a regular dependency into the built target:
https://github.com/qemu/qemu/tree/005ad32358f12fe9313a4a0191...
https://github.com/harfbuzz/harfbuzz/tree/main/subprojects
https://github.com/harfbuzz/harfbuzz/blob/37457412b3212463c5...
Or, if you're using proper operating systems, they're managed by the usual package manager, just like everything else.
https://mesonbuild.com/Wrapdb-projects.html
https://mesonbuild.com/Wrap-dependency-system-manual.html
meson will download and build the libraries automatically and give you a variable which you pass as a regular dependency into the built target:
https://github.com/qemu/qemu/tree/005ad32358f12fe9313a4a0191...
https://github.com/harfbuzz/harfbuzz/tree/main/subprojects
https://github.com/harfbuzz/harfbuzz/blob/37457412b3212463c5...
Or, if you're using proper operating systems, they're managed by the usual package manager, just like everything else.
It depends.
Some "single-file" libraries only have the one header, so you can use that one file. Though they often require a bit of trickery to be compiled correctly, so I prefer those with two file, one .h header and one .c source.
Some libraries are popular enough that they’re distributed in your package manager of choice: whatever your Linux/BSD distribution uses, Nix, a Cargo-like repository in some cases… In this case you can download that package and depend on it.
All working libraries can be built from source, one way or another. They each have their own preferred build system though. Most can be installed from source, so you don’t need to be aware of their build system to use it. But if you take on the dependency like that now your users will depend on the library so it’s not exactly free.
Some libraries offer several or all of the above.
Managing updates depend on how you integrate the library. If you use it as a system dependency, your users need to download and update the correct version themselves. Package managers can help there (use Ubuntu/Nix/Ports, get security updates for free… if you trust them to be quick enough, which they will be if the library is popular). The alternative is to ship your own copy of the library yourself, either by building it at home with their build system and manually importing headers & .so/.dll, or absorb their source code into your own build system (you’re on your own but you have full control).
Which is best is… up to who you’re speaking to. To oversimplify the issue, I’d say distro maintainers don’t trust developers to deliver security updates in time, so they want everyone to use .so and dynamic linking everywhere so security updates can happen in a timely manner. On the other hand developers don’t trust distro maintainers to do regular updates in a timely manner, so they’re stuck to old versions of their favourite libraries. Those who do both… maybe have gone insane from the cognitive dissonance?
Some "single-file" libraries only have the one header, so you can use that one file. Though they often require a bit of trickery to be compiled correctly, so I prefer those with two file, one .h header and one .c source.
Some libraries are popular enough that they’re distributed in your package manager of choice: whatever your Linux/BSD distribution uses, Nix, a Cargo-like repository in some cases… In this case you can download that package and depend on it.
All working libraries can be built from source, one way or another. They each have their own preferred build system though. Most can be installed from source, so you don’t need to be aware of their build system to use it. But if you take on the dependency like that now your users will depend on the library so it’s not exactly free.
Some libraries offer several or all of the above.
Managing updates depend on how you integrate the library. If you use it as a system dependency, your users need to download and update the correct version themselves. Package managers can help there (use Ubuntu/Nix/Ports, get security updates for free… if you trust them to be quick enough, which they will be if the library is popular). The alternative is to ship your own copy of the library yourself, either by building it at home with their build system and manually importing headers & .so/.dll, or absorb their source code into your own build system (you’re on your own but you have full control).
Which is best is… up to who you’re speaking to. To oversimplify the issue, I’d say distro maintainers don’t trust developers to deliver security updates in time, so they want everyone to use .so and dynamic linking everywhere so security updates can happen in a timely manner. On the other hand developers don’t trust distro maintainers to do regular updates in a timely manner, so they’re stuck to old versions of their favourite libraries. Those who do both… maybe have gone insane from the cognitive dissonance?
I use Nix as a package manager for my C projects. It works very well and has out of the box support for most C build systems (make, autotools, cmake, meson)
Really? You manage the libraries through nix?
Yes!
just pop in a:
just pop in a:
stdenv.mkDerivation {
pname = "myproject";
version = "0.0.0";
nativeBuildInputs = [ pkg-config ninja meson ];
buildInputs = [ openssl zlib libfido2 ];
}
This reminds me of the time a younger coworker asked me if there's a way to run a diff without putting the files in a git repo first. Some questions never even occur to us geezers to ask...
(Not razzing you or anything; I just genuinely found that question startling, having started programming in the 1990s.)
(Not razzing you or anything; I just genuinely found that question startling, having started programming in the 1990s.)
That’s interesting
[deleted]
When I first learned of NaCL/libsodium I was very interested in a new API but.. It seems to me like if you go with anything along these lines you have very little chance of ever supporting hardware protected keys (smartcards, hsms, etc) and, quite frankly, I'm not sure any data should be stored on a normal computer or mobile phone that you wouldn't publish in a blog, so having a software key is madness, IMO.
There's also https://monocypher.org
don't forget https://github.com/jedisct1/libhydrogen for microcontrollers
Then don’t forget https://monocypher.org as well. Bigger than libhydrogen but still small enough for many targets, faster across the board, and compatible with libsodium. If you can spare a couple more KB of flash in your microcontroller, you can get very good performance on the device and use libsodium on the server to scale like crazy.
It also doesn’t hurt that there is basically 1 header and 1 source file (2 of each with the options), which you can trivially copy & paste into your project.
It also doesn’t hurt that there is basically 1 header and 1 source file (2 of each with the options), which you can trivially copy & paste into your project.
Not just for microcontrollers.
If performance is not a priority, but you need something small, simple, and easy to embed, libhydrogen can come in handy.
I actually use it more than libsodium.
If performance is not a priority, but you need something small, simple, and easy to embed, libhydrogen can come in handy.
I actually use it more than libsodium.
Google search suggests there are already implemations of QUIC, TLS and SSH which use libsodium.
Given the dependency of Web and login on both, I like this.
Given the dependency of Web and login on both, I like this.
[deleted]
_ocro(1)
I previously used sodiumoxide[1] libsodium bindings in my projects but they are in maintenance only mode and wouldn't accept patches to build on other platforms (we had raspberry pi users complaining they couldn't build our software because of sodiumoxide). Maintaining cross platform C libraries which use assembly primitives is pretty tricky. The ring[2] library - which is used by rustls (a better tls library for rust) - is notoriously annoying to cross compile because of its use of C and assembly.
[0]: https://github.com/rustcrypto [1]: https://github.com/sodiumoxide/sodiumoxide [2]: https://github.com/briansmith/ring