BLAKE2, an improved version of the SHA-3 finalist BLAKE optimized for speed(blake2.net)
blake2.net
BLAKE2, an improved version of the SHA-3 finalist BLAKE optimized for speed
https://blake2.net/
4 comments
Over the past year and a half I've had it drummed into me that fast hashing is bad and slow hashing is good. If this is so, why have they optimised this new hashing algorithm to be as fast as possible?
To expand on what has already been noted: a good place for fast hashing is when you're using hashing millions of objects. For example, if I wanted to parse thousands of feeds and store items using a unique hash as their key, I'd want a fast hash that's collision resistant (for the uniqueness part) and one that's fast (for the millions of objects). With password hashing (KDFs, as tptacek noted), the collision resistance is useful, but we want to keep it fairly slow to avoid being able to brute force attacks.
Slow hashing is usually bad, except for password-derivation functions.
fast hashing is bad for storing one way functions of passwords because it makes them easier to attack and make many guesses about the original value from the transformed value, hence all the noise about using slow "key derivation functions" instead. for all other common use cases, faster is better, all else equal.
I'm not sure how reduced memory requirements are a benefit to encryption. Are there really any low end systems still in use today that actually have issues with memory usage? Even $150 notebooks come with 1 gig of ram. This seems more like it would help save ram on interception devices like the Narus Device or the huge datacenters owned by the NSA, which have a huge issue with storing all the data required to intercept and decrypt eavesdropped communications reliably.
The last system where I used SHA-256 had 16KiB of RAM (and 128KiB of direct-mapped flash). Yes, memory use for security algorithms is important.
I think of reduced memory requirements as a benefit to every application, including encryption. That's because nearly every application I use competes for memory; I also use memory as general storage media for stuff for which I want fast I/O and/or don't need to save permanently (e.g. mfs or tmpfs mounts). I think of memory as a precious resource.
That doesn't really apply to memory use on the order of a single kilobyte. You're not going to be hashing many different streams at once.
And why not? Ever considered a busy SSL offload device or VPN appliance?
A kilobyte here, a kilobyte there, and pretty soon you're talking real memory!
A kilobyte here, a kilobyte there, and pretty soon you're talking real memory!
I think something handling a TCP stream has better things to worry about than an extra buffer smaller than a single packet.
Well, you're wrong, at least for a specific segment of hardware and software on the market. The size of a packet is not related to the acceptable per-connection overhead. Some network devices like ASIC-accelerated routers can't handle any per-connection state.
If you don't believe me, hop on any kernel development mailing list and try to convince them they could solve world hunger by only adding 1 KB overhead to the TCP socket structure that tracks actve connection states.
If you don't believe me, hop on any kernel development mailing list and try to convince them they could solve world hunger by only adding 1 KB overhead to the TCP socket structure that tracks actve connection states.
Can't upvote this enough. When you live at the bottom of the stack, you can't be cavalier about your constant overheads. Practically any overhead you can think of is going to be multiplied by "n" in someone's O(n).
I'm sure someone will multiply anything by n but that doesn't mean their code is right. I can't think of any legitimate use for thousands of incomplete hashes without so much other memory the hash is dwarfed. Can you?
The argument about streamlined vpn or ssl devices is not relevant here: such a device has to calculate hashes, but it doesn't have to keep hash states open. Hash a message, make/check the signature, forward or discard, O(k) memory use where k is the number of cores.
I agree that a per connection overhead of this size for every TCP connection is a terrible thing, but I think marshray forgot we were talking about places where you use hashes, and keep them open.
The argument about streamlined vpn or ssl devices is not relevant here: such a device has to calculate hashes, but it doesn't have to keep hash states open. Hash a message, make/check the signature, forward or discard, O(k) memory use where k is the number of cores.
I agree that a per connection overhead of this size for every TCP connection is a terrible thing, but I think marshray forgot we were talking about places where you use hashes, and keep them open.
During the TLS handshake process, there are multiple running hashes kept of the handshake data.
For TLS records, there's a MAC at the end of each record. The MAC is based on HMAC, the most efficient implemenation involves priming two hash states with the MAC secret in advance. So with send and receive, every TLS connection already has four open hash states.
You're right though that most implementations seem to buffer a full record's data too. One could probably avoid this overhead when sending TLS (if you knew a minimum length in advance).
For TLS records, there's a MAC at the end of each record. The MAC is based on HMAC, the most efficient implemenation involves priming two hash states with the MAC secret in advance. So with send and receive, every TLS connection already has four open hash states.
You're right though that most implementations seem to buffer a full record's data too. One could probably avoid this overhead when sending TLS (if you knew a minimum length in advance).
Think of encryption as more secure the harder it is to do. Optimizing encryption for the benefit of other applications is backwards.
Think about RSA. Harder Encryption (computationally) is achieved with a high e. Too bad that Wiener in 1990 proved that if d (private key) is small enough we can easily crack it using continued fractions.
[deleted]
[deleted]
Think smart cards.
Or RFID. If you actually want to build secure RFID devices, you need really low-power crypto.
Some kind of next generation smart card, no doubt. Like a universal ID, replacement for mobile payment methods and Apple Passbook, since people don't trust their carriers to be reliable, unbiased communication networks or platforms... ah yes, I see.
[deleted]
> Even $150 notebooks come with 1 gig of ram.
The issue is with the processor's cache memory, which is small, typically 64 kiB or so. (The TLB cache is also important.) Even a few hundred bytes savings can give an important improvement in speed amd power consumption.
The issue is with the processor's cache memory, which is small, typically 64 kiB or so. (The TLB cache is also important.) Even a few hundred bytes savings can give an important improvement in speed amd power consumption.
A little off topic, but since it's on the page: Is it only me or does "mebibyte" simply look wrong?
You can't just decide that people aren't supposed to say "megabyte" anymore
You can't just decide that people aren't supposed to say "megabyte" anymore
http://en.wikipedia.org/wiki/Mebibyte
It's becoming more common. By the way, if you use OS X, your utilities output megabytes (1000 bytes), not mebibytes. GNU utilities switched to KiB, MiB, etc. (1024 bytes).
It's becoming more common. By the way, if you use OS X, your utilities output megabytes (1000 bytes), not mebibytes. GNU utilities switched to KiB, MiB, etc. (1024 bytes).
I agree that it looks odd. However, especially in applications like cryptography, I think that removing the ambiguity of "megabyte" (i.e. do we mean 10^6 or 2^20 bytes?) is worth the introduction of a new term.
Do you mean an additional new term to mean millionbyte? Because the existence of mebibyte only makes 'mega' even more ambiguous. You used to be able to know from context.
I was referring to the "mebi" prefix. I agree that, currently, the old SI prefixes have perhaps been made slightly more ambiguous due to the introduction of the new prefixes. It is my hope that the computing community will eventually reach the consensus that the SI prefixes refer only to powers of 10.
as other people pointed out, Mebibyte is the correct form.
You can't just make the wrong version right by means of usage.