I've never really understood the difference between a KDF, hashing function, password hashing function, though it's relevant to know when writing reports (I work for a security company). We recommend Argon2/scrypt/bcrypt for password storage of course, but we call them KDFs and I'm not sure if it's correct. From my understanding, a KDF can be fast, but a PBKDF must be slow. Could you elaborate or do you know a good resource (short of a whole book on low level crypto details)?
I don't know of a good source beyond textbooks or papers which focus precisely on the low level crypto details you (rightly) want to avoid. There just isn't much of a need for that kind of nuance most of the time. I also (gently) reject the premise; as far as practical security is concerned, if your team is recommending Argon2/scrypt/bcrypt to developers then that's far more important than being able to explain the difference between key derivation and keyless password hashing.
It's essentially like rectangles versus squares. You can create a key derivation function out of anything which passes all the criteria of a password hashing function. But it won't be a particularly performant or useful key derivation function. Likewise you can create a password hashing algorithm out of a dedicated key derivation function, but that's insufficient on its own.
There's no need to get bogged down in the details, just continue recommending a reputable implementation of these algorithms. On the other hand, if you'd like to learn more out of intellectual curiosity, Boneh & Shoup's textbook is good (work in progress) [1]. Galbraith's textbook includes chapters which cover the topic to a depth that's beyond what you're looking for, but you'll learn whatever it is you want to know [2].
Finally, more accessible, informal answers that get the basic idea across are [3], [4].
A Key derivation function (KDF) is a basic and essential component of cryptographic systems: Its goal is to take a source of initial keying material, usually containing some good amount of randomness, but not distributed uniformly or for which an attacker has some partial knowledge, and derive from it one or more cryptographically strong secret keys.
Not all KDFs are hash functions, or hash-function based. There are block-cipher based KDFs, stream-cipher based KDFs, etc.
Hash functions take arbitrary length input (well, up to some very large maximum size) and provide fixed-length output.
eXtensible Output Functions (XOFs)take arbitrary length input (up to some very large max) and provide arbitrary length output (up to some very large max).
Password Hashing Functions take (at least) three inputs: a unique salt, a secret password, and a tuning parameter (or set of parameters). They use the tuning parameter(s) to change the amount of work needed to compute their outputs. For any set of inputs they produce a deterministic output. The output may or may not be directly suitable for use as a cryptographic key, and may or may not be variable length.
Some password hashing functions are KDFs, taking effectively arbitrary input length and producing effectively arbitrary output length. PBKDF2 and Argon2 are KDFs.
Some password hashing functions are NOT KDFs, having limits on their input & output sizes. Bcrypt is not a KDF and not a hash function: it has a maximum 56-byte input (55 bytes if taking a null-terminated string, 72 bytes max in newer implementations) and a 60-byte output. It's suitable for logins where the password is hashed and compared to the stored hash, but not for directly deriving key material. And it's not necessarily suitable for non-ASCII passwords/passphrases, due to the short input.
When talking about passwords specifically in our reports (I work at a security company too), I tell our team to use the same language as NIST 800-63B, since its the best "standard" for passwords and authentication I can find.
The relevant bit here is this:
Verifiers SHALL store memorized secrets in a form that is resistant to offline attacks. Memorized secrets SHALL be salted and hashed using a suitable one-way key derivation function. Key derivation functions take a password, a salt, and a cost factor as inputs then generate a password hash.
They are specific about the type of KDF required: "one-way key derivation function".
You are right. I still think we should call "Password Hashing Algorithms". "One-way password storage" maybe. Or something. Just get rid of the word hashing.
Edit:
NIST uses "one-way key derivation function" in their requirements, which I like, but that perhaps unfairly excludes other potential functions.
lucb1e|6 years ago
throwawaymath|6 years ago
It's essentially like rectangles versus squares. You can create a key derivation function out of anything which passes all the criteria of a password hashing function. But it won't be a particularly performant or useful key derivation function. Likewise you can create a password hashing algorithm out of a dedicated key derivation function, but that's insufficient on its own.
There's no need to get bogged down in the details, just continue recommending a reputable implementation of these algorithms. On the other hand, if you'd like to learn more out of intellectual curiosity, Boneh & Shoup's textbook is good (work in progress) [1]. Galbraith's textbook includes chapters which cover the topic to a depth that's beyond what you're looking for, but you'll learn whatever it is you want to know [2].
Finally, more accessible, informal answers that get the basic idea across are [3], [4].
1. https://toc.cryptobook.us/
2. https://www.math.auckland.ac.nz/~sgal018/crypto-book/main.pd...
3. https://security.stackexchange.com/questions/95410/what-is-t...
4. https://crypto.stackexchange.com/questions/70716/key-derivat...
SAI_Peregrinus|6 years ago
Hash functions take arbitrary length input (well, up to some very large maximum size) and provide fixed-length output.
eXtensible Output Functions (XOFs)take arbitrary length input (up to some very large max) and provide arbitrary length output (up to some very large max).
Password Hashing Functions take (at least) three inputs: a unique salt, a secret password, and a tuning parameter (or set of parameters). They use the tuning parameter(s) to change the amount of work needed to compute their outputs. For any set of inputs they produce a deterministic output. The output may or may not be directly suitable for use as a cryptographic key, and may or may not be variable length.
Some password hashing functions are KDFs, taking effectively arbitrary input length and producing effectively arbitrary output length. PBKDF2 and Argon2 are KDFs.
Some password hashing functions are NOT KDFs, having limits on their input & output sizes. Bcrypt is not a KDF and not a hash function: it has a maximum 56-byte input (55 bytes if taking a null-terminated string, 72 bytes max in newer implementations) and a 60-byte output. It's suitable for logins where the password is hashed and compared to the stored hash, but not for directly deriving key material. And it's not necessarily suitable for non-ASCII passwords/passphrases, due to the short input.
[1] https://eprint.iacr.org/2010/264
amlozano|6 years ago
The relevant bit here is this:
Verifiers SHALL store memorized secrets in a form that is resistant to offline attacks. Memorized secrets SHALL be salted and hashed using a suitable one-way key derivation function. Key derivation functions take a password, a salt, and a cost factor as inputs then generate a password hash.
They are specific about the type of KDF required: "one-way key derivation function".
The examples given later are PBKDF2 and Balloon.
amlozano|6 years ago
Edit: NIST uses "one-way key derivation function" in their requirements, which I like, but that perhaps unfairly excludes other potential functions.