Keccak's (pronounced: ketchup) team includes Joan Daemen, who was also on the Rijndael team, giving him a hand in both AES and SHA-3.
Keccak's structure is simple and markedly different from that of MD4, MD5, SHA1, and the SHA-2 family, all of which share a common design pattern called Merkle-Damgard (MD). MD-style hashes chunk their inputs into blocks and tag the last block with a length; they then run in a manner similar to a block cipher, where each block is combined with the previous block after the hash function core is applied. The resulting output of these MD-style hashes is literally the internal state of the hash at the last block; you can take most SHA-2 hashes, for instance, and "feed them back" to the SHA function with more data to generate a continued hash. This gives rise to a common crypto protocol flaw called a "length extension attack".
Unlike MD, Keccak uses a design called a "sponge function". It's called a "Sponge" because it splits hash into two distinct stages: one in which the hash function "absorbs" data, applying the hash core function to permute an internal state, and another in which the hash is "squeezed" to produce output (which further permutes the state). Somewhat like a cryptographic PRNG, the security of the hash function is delegated to the confidentiality of the internal state, which isn't disclosed by producing the hash.
Furthermore, Keccak's Sponge design derives security by only allowing inputs to directly influence a subset of the internal state bits. Like MD hashes, the inputs are chunked into blocks and fed to the hash algorithm. But each block fed to the hash affects only that block's worth of internal state bits; the remainder of the hash's state (called the "capacity bits") are mixed in with the "outer" bits during the application of the hash core. Here's a picture that may tell the story better:
Notice how the "M" bits of the input hit only the "r" bits of "outer" state in the hash, while the "c" bits of "capacity" are used only during the "f" function.
If all it does is prevent length extension attacks, then there are much simpler and less risky ways to do that (i.e., MD variants).
Also, your explanation of the sponge structure omits the real difference between it and MD: It is a transform that turns a non-compressing function (f in that diagram) into a compressing function. MD, on the other hand, starts with a fixed-input-size compressing function and extends its domain.
By the way, what do you mean by "Furthermore, Keccak's Sponge design derives security by only allowing inputs to directly influence a subset of the internal state bits."? That's as true for an MD-type construction as it is for a sponge construction. In fact, it's a crucial fact that allows us to build a reduction from, say, the collision-resistance of MD[f] to the collision-resistance of f.
If Keccak isn't vulnerable to a length extension attack, is using it to hash a secret key concatenated with a message a good MAC algorithm? If not, why not?
At some points throughout the competition, the determination criteria used by NIST were somewhat unusual, with the decision committee stating that
"We preferred to be conservative about security, and
in some cases did not select algorithms with exceptional performance, largely because something about them made us
“nervous,” even though we knew of no clear attack against the full algorithm."
My impression is that people, in the absence of obvious (to them) differences in security, will choose a cryptographic function based on speed, which is simple to understand and quantify.
Going by the eBASH benchmarks, software implementations of Keccak are mostly the same speed as SHA-2, and for some CPUs it's significantly slower.
It does have better throughput/area ratio in hardware, but chip manufacturers are not early adapters.
With that in mind I can't see any major push for general SHA-3 adoption unless the SHA-2 family is broken.
Still, I'm looking forward to the reading the report.
"Keccak, created by Guido Bertoni, Joan Daemen and Gilles Van Assche of STMicroelectronics and Michaël Peeters of NXP Semiconductors"
I don't think there will be any issue with regards to some early adopters. Maybe even a concideration with regards to common hardware usage like smartcards at some level.
But whilst your right about SHA-2 not being broken and making the rush to SHA-3 moot, it does not stop marketing types getting sales over SHA-2 only complient. In that it will be marketing and drive for sales and standing out that will drive it into market with all things being as they stand. Besides, everybody likes a plan B to fall back upon.
Surely, as tptacek's professional life (you know the one here on HN) is dedicated to proving that the security of the encryption algorithm is mostly incidental to the security of the system as a whole, then I would like the next competition to be about producing reference implementations or similar. Sort of like keyczar but down to the point where all browsers simply incorporate the same approved crypto libraries and Apis, securely use areas of memory without file handles anywhere in the background and all the other good stuff that takes us into a 80% done by default world.
I am not saying this isn't good stuff, but as someone struggling to implement a user login properly it would be much nicer to have a template to follow. (and he waits for someone to point to the template ...)
A user login, you say? First, find bcrypt or scrypt bindings for your programming language of choice. Here's one for Python, with some very concise example code:
Store the hashed password that bcrypt gives you in a database somewhere. Make sure your login goes over HTTPS. I think that should cover the basics; do you have any specific questions?
I don't think I understand Schneier's point here. This is only the second time (AFAIK) that NIST has held a contest like this to standardize a crypto primitive. In the first contest, there was a clear and urgent need to retire DES. In the case of SHA-2, it was clear going in that there wasn't a "speeds and feeds" problem with SHA so much as there was a gathering unease about the basic design of the family of hashes from which SHA-2 is derived: MD4 and MD5 are broken, and SHA-1 isn't looking long for this world. Meanwhile, hash functions are not as well studied as block ciphers and number theoretic crypto, and the pace of development in hash research is increasing.
There is value simply in diversifying the gene pool of hash functions.
[+] [-] tptacek|13 years ago|reply
Keccak's structure is simple and markedly different from that of MD4, MD5, SHA1, and the SHA-2 family, all of which share a common design pattern called Merkle-Damgard (MD). MD-style hashes chunk their inputs into blocks and tag the last block with a length; they then run in a manner similar to a block cipher, where each block is combined with the previous block after the hash function core is applied. The resulting output of these MD-style hashes is literally the internal state of the hash at the last block; you can take most SHA-2 hashes, for instance, and "feed them back" to the SHA function with more data to generate a continued hash. This gives rise to a common crypto protocol flaw called a "length extension attack".
Unlike MD, Keccak uses a design called a "sponge function". It's called a "Sponge" because it splits hash into two distinct stages: one in which the hash function "absorbs" data, applying the hash core function to permute an internal state, and another in which the hash is "squeezed" to produce output (which further permutes the state). Somewhat like a cryptographic PRNG, the security of the hash function is delegated to the confidentiality of the internal state, which isn't disclosed by producing the hash.
Furthermore, Keccak's Sponge design derives security by only allowing inputs to directly influence a subset of the internal state bits. Like MD hashes, the inputs are chunked into blocks and fed to the hash algorithm. But each block fed to the hash affects only that block's worth of internal state bits; the remainder of the hash's state (called the "capacity bits") are mixed in with the "outer" bits during the application of the hash core. Here's a picture that may tell the story better:
http://tinyurl.com/cryptosponge
Notice how the "M" bits of the input hit only the "r" bits of "outer" state in the hash, while the "c" bits of "capacity" are used only during the "f" function.
[+] [-] cdavidcash|13 years ago|reply
Also, your explanation of the sponge structure omits the real difference between it and MD: It is a transform that turns a non-compressing function (f in that diagram) into a compressing function. MD, on the other hand, starts with a fixed-input-size compressing function and extends its domain.
By the way, what do you mean by "Furthermore, Keccak's Sponge design derives security by only allowing inputs to directly influence a subset of the internal state bits."? That's as true for an MD-type construction as it is for a sponge construction. In fact, it's a crucial fact that allows us to build a reduction from, say, the collision-resistance of MD[f] to the collision-resistance of f.
[+] [-] B-Con|13 years ago|reply
I don't think so. From NIST's announcement: http://www.nist.gov/itl/csd/sha-100212.cfm
> Keccak (pronounced “catch-ack”)
[+] [-] andreyf|13 years ago|reply
[+] [-] xal|13 years ago|reply
[+] [-] apawloski|13 years ago|reply
"We preferred to be conservative about security, and in some cases did not select algorithms with exceptional performance, largely because something about them made us “nervous,” even though we knew of no clear attack against the full algorithm."
http://csrc.nist.gov/groups/ST/hash/sha-3/Round3/documents/E... [PDF]
[+] [-] zokier|13 years ago|reply
[+] [-] tveita|13 years ago|reply
Going by the eBASH benchmarks, software implementations of Keccak are mostly the same speed as SHA-2, and for some CPUs it's significantly slower.
It does have better throughput/area ratio in hardware, but chip manufacturers are not early adapters.
With that in mind I can't see any major push for general SHA-3 adoption unless the SHA-2 family is broken.
Still, I'm looking forward to the reading the report.
[+] [-] Zenst|13 years ago|reply
I don't think there will be any issue with regards to some early adopters. Maybe even a concideration with regards to common hardware usage like smartcards at some level.
But whilst your right about SHA-2 not being broken and making the rush to SHA-3 moot, it does not stop marketing types getting sales over SHA-2 only complient. In that it will be marketing and drive for sales and standing out that will drive it into market with all things being as they stand. Besides, everybody likes a plan B to fall back upon.
[+] [-] ChuckMcM|13 years ago|reply
[+] [-] lifeisstillgood|13 years ago|reply
I am not saying this isn't good stuff, but as someone struggling to implement a user login properly it would be much nicer to have a template to follow. (and he waits for someone to point to the template ...)
[+] [-] pjscott|13 years ago|reply
http://www.mindrot.org/projects/py-bcrypt/
Store the hashed password that bcrypt gives you in a database somewhere. Make sure your login goes over HTTPS. I think that should cover the basics; do you have any specific questions?
[+] [-] sp332|13 years ago|reply
[+] [-] tptacek|13 years ago|reply
There is value simply in diversifying the gene pool of hash functions.
[+] [-] kbd|13 years ago|reply
http://www.schneier.com/blog/archives/2012/10/keccak_is_sha-...
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] B-Con|13 years ago|reply
[+] [-] dchest|13 years ago|reply
[+] [-] gourneau|13 years ago|reply
[deleted]
[+] [-] unknown|13 years ago|reply
[deleted]