The comments on this point to another question of which this is an exact duplicate. Interested parties should definitely read the top answer there, as it was written by a member of the IEEE-754 committee.
Strangely, I was just adding some more material to this answer as I’ve never really been completely satisfied with it; then I flip over to hacker news and there it is. Anyway, if anyone has other IEEE-754 questions, I’ll try to answer.
This is what I would expect from the underlying mathematics. NaN roughly means the output of a function is undefined, and it does not make sense to say functions are equal at some value just because they are both undefined for that value.
For example, it's probably more sensible to say that 2x/x != 3x/x for all values of x, rather than 2x/x != 3x/x for all x except 0, wherein both functions are undefined and therefore equal to each other.
I agree it really doesn't make sense to say two NaNs are definitely equal, but it doesn't seem necessarily true that they aren't equal, either. Semantically, I would say NaN == NaN should return undefined (assuming JS here). Of course, it would be weird for == not to return true or false, so in practice that probably isn't a good idea. So I agree with you false is the least bad option.
>This is what I would expect from the underlying mathematics. NaN roughly means the output of a function is undefined,
That's not true, NaN means "not a number". Take for example f(x) = 1/x. What is f(0)? Infinity? Well, you can't divide by zero. In fact, I tricked you. When you talk about a function in mathematics, you need more information that just the formula, you also need to know over what domain I am allowed to evaluate such a function. What I should have said was, define f: [1, 2] -> R s.t. f(x) = 1/x. Now you are not allowed to ask the question, "what is f(0)?" because I didn't include zero in the domain. You might then ask, "so how do you define f at zero?" That question mentions 'f', but the 'f' it is referring to is now different. Such an 'f' you are allowed to ask this question for would be the following:
f:[0, 2] -> R
f(x) = 1/x when x > 0
f(0) = 1
Now, when you ask me what f(0) is, I can tell you. It is 1. "That's cheating!" you might add. No it isn't. It is a different function with different properties. Just because the formula is the same does not mean you can evaluate it wherever you want.
Note: you are not allowed to divide by zero. The result would be meaningless if you could. And 'NaN' is only something the computer spits out when it encounters an operation that doesn't return a number. Usually, a computer will give you 'Inf' when you try to divide by zero numerically. Mathematically, you cannot do it. So don't try.
NaN means 'not a number', and '=' is an infix operator to compare numbers. You can't compare for equality on things that are not numbers.
You can compare infinities, however. "I thought you said dividing by zero was meaningless!" It is. I'm not going to divide by zero, but I can still talk about infinity. Take for example the following function:
f: (0, 1] -> R
f(x) = 1/x
f is defined on every number strictly greater then zero, but less than or equal to one. Zero is not in the domain, so I can't evaluate f there. However, I can still study what happens near zero. Take a really small number bigger than zero. The reciprocal of that is huge. Now take a smaller number, the reciprocal becomes even bigger. In fact, we can keep doing this: construct any positive sequence that converges to zero (but never attains it). Now evaluate f at each element in that sequence, you will notice that the result becomes larger and larger. We call the limit of this 'infinity', and we have notation for this. Notice I did NOT evaluate f at zero, I merely studied what happens as I approach zero. Infinity is not a real number, and I cannot have f ever evaluate to infinity, but I can talk about limiting properties. Computers do this too, but they have a much lower standard for doing so. Essentially, anything bigger than the maximum value of a double (about 10^300, I think?) is called 'Inf' on a computer.
Edit: I forgot to address the second paragraph of the parent comment.
>For example, it's probably more sensible to say that 2x/x != 3x/x for all values of x, rather than 2x/x != 3x/x for all x except 0, wherein both functions are undefined and therefore equal to each other.
This is not a sensible thing to do.
Let me elaborate. I'm going to ignore any cancellation issue and say it is definitely true that 2/x != 3/x for x != 0. Let's also say that each of these functions has domain (0, 1]. These two functions are not equal are x = 0 because you cannot evaluate them at x = 0. It is forbidden. However, behaviour near to zero is very similar. See my discussion above. In fact, you can make a stronger statement. You can say that as x approaches zero (from the positive side), in the limit, 2/x and 3/x are equal. That is, they have the same limit as you approach zero. This does not make them equal at zero, because I cannot evaluate them at zero. To make this point absolutely clear, I will add zero into the domain and define two different functions:
f:[0,1] -> R
f(x) = 2/x for x > 0
f(0) = 1
and
g:[0,1] -> R
g(x) = 3/x for x > 0
g(0) = 2
These functions have the same limit as you approach zero, but they are not equal at zero. I can evaluate them at zero this time, because I defined what it meant to do so. And indeed f(0) = 1 != 2 = g(0).
It seems like the desired result is to make it determinable when a NaN value is produced in a language that doesn't have an is_nan function (such as an assembly language lacking such a predicate) so that a simple boolean statement does the trick, for example:
def is_nan(val):
return not (val == val)
This absolutely bizarre statement of non-identity gives us a surefire test. It is from the perspective of higher level languages that the non-identity of NaN with itself seems bizarre.
There's the obvious warning that one shall not compare floating point numbers for equality...
But then, it's indeed quite surprizing. It get worse on pure tanguages where the expressions (0 / 0) and (0 / 0) are equal before evaluation, but different afterwards. Also, it trashes hash tables.
Yet, NANs being different is the sanest mathematical definition. Maybe people should have opted for the ugliest choice because of real world concerns (it wouldn't be the first time), but this time, they didn't.
Some of the links I posted to stack overflow; the top-voted answer there (as of this writing) was wildly incorrect, claiming it to be a mistake. It is not.
Any invalid calculation produces NaN as opposed to raising an exception/signal because that would be non-portable (especially in the early 80s). The goal was that a correct algorithm on one platform or in one language would translate and be correct somewhere else. Further, some algorithms depend on being able to probe a function to find its bounds, requiring them to be able to locate the out of bounds condition without halting execution; doing that in a safe, portable way is actually quite hard unless you do it in software as part of the spec, hence NaN.
If NaN==NaN, then (NaN/NaN)==1, etc etc and you still end up in a world of nonsense as far as naive algebra is concerned. This is just some of my fellow programmers whining and crying about having to think about floating point, the same sort of complaining I hear about Unicode (and full of just as much self-sure awful advice that produces wildly incorrect results for everything except the complainer's specific use case).
It's not just "having to think about floating-point"--it's that a number of other things break. Easy function memoization doesn't work with NaNs because NaN!=NaN, so the function arguments look different, so the saved memoized value is never used.
The comments give reason to believe this isn't the cause, but my first instinct was that NaN cannot be equal to NaN as it isn't clear what the object represents. As Mike C states in his answer: "Another way to look at it is that NaN just gives you information about what something ISN'T, not what it is."
But is this an effective way of looking at the question? In reality what you're trying to do is not compare a NaN to another NaN, you're actually attempting to check if both objects are NaN. This hurts my brain.
As a C/C++ and graphics programmer, I love NaN. It allows saving space by not having to carry an extra is_valid boolean for uninitialized values or regions of data that are masked.
For textures, they are great as they don't interpolate w/ nearby neighbors and properly discard fragments & vertices without the aforementioned cost of an extra boolean attribute.
That was a very interesting read. I always thought NaN != NaN was because it's a class of values (there are many binary representations that are NaN, as it just requires an all-ones exponent and a mantissa that isn't all-zeroes) but I guess I was considering it too simplistically.
This might actually explain a very intermittent bug that I spent some hours on and never solved. A sort that would never terminate, and the best I could come up with was that for reasons I couldn't understand the comparison function was returning false when an item was compared to itself ... TIL.
Not a number just tells you what something isn't, not what it is. A NaN could be a string, or a potato, or an airplane, or the prime minister of Djibouti. Is a potato equal to an airplane?
Throwing exceptions for everything isn't a catch-all solution for writing better software—take a look at the Ariane rocket failure, which was caused by bounded numeric types throwing exceptions aggressively. Some modern programming styles tend to avoid enthusiastic non-local returns anyway in favor of things like monadic composition, such as Maybe in Haskell (which is more like NaN than it is like an exception).
The nice thing about NaN is that you can just do a calculation as normal, and check for NaN at the very end. This means you don't have to do an expensive test/branch after every arithmetic or other numeric operation. The hardware is much, much easier to design if you don't have to make it branch, and the code is much faster without the compiler inserting branches. People who work with floating point numbers every day care very deeply about performance.
If you don't care as much about performance, why not write your code in a language that does throw exceptions? Python, for example.
Those of us that do use numerics love NaN, love signed zeros, and can live with NaN ≠ NaN even though it's kind of dumb.
Because having the hardware throw exceptions all over the place can get INCREDIBLY tedious if you want to ignore them and solider on. It means having "try/catch" around every single arithmetic operation everywhere. So it's tons of extra code and it might slow things down 100%-500% or more.
What if you're doing some kind of matrix math and you just need the results of the diagonal and a single NaN creeps in somewhere but doesn't destroy your matrix. Would you accept your program needing 50% more LOC and taking 4x as long for this result?
[+] [-] benjamincburns|11 years ago|reply
http://stackoverflow.com/a/1573715/203705
[+] [-] stephencanon|11 years ago|reply
Strangely, I was just adding some more material to this answer as I’ve never really been completely satisfied with it; then I flip over to hacker news and there it is. Anyway, if anyone has other IEEE-754 questions, I’ll try to answer.
[+] [-] dang|11 years ago|reply
[+] [-] karamazov|11 years ago|reply
For example, it's probably more sensible to say that 2x/x != 3x/x for all values of x, rather than 2x/x != 3x/x for all x except 0, wherein both functions are undefined and therefore equal to each other.
[+] [-] Fishkins|11 years ago|reply
[+] [-] coherentpony|11 years ago|reply
That's not true, NaN means "not a number". Take for example f(x) = 1/x. What is f(0)? Infinity? Well, you can't divide by zero. In fact, I tricked you. When you talk about a function in mathematics, you need more information that just the formula, you also need to know over what domain I am allowed to evaluate such a function. What I should have said was, define f: [1, 2] -> R s.t. f(x) = 1/x. Now you are not allowed to ask the question, "what is f(0)?" because I didn't include zero in the domain. You might then ask, "so how do you define f at zero?" That question mentions 'f', but the 'f' it is referring to is now different. Such an 'f' you are allowed to ask this question for would be the following:
f:[0, 2] -> R
f(x) = 1/x when x > 0
f(0) = 1
Now, when you ask me what f(0) is, I can tell you. It is 1. "That's cheating!" you might add. No it isn't. It is a different function with different properties. Just because the formula is the same does not mean you can evaluate it wherever you want.
Note: you are not allowed to divide by zero. The result would be meaningless if you could. And 'NaN' is only something the computer spits out when it encounters an operation that doesn't return a number. Usually, a computer will give you 'Inf' when you try to divide by zero numerically. Mathematically, you cannot do it. So don't try.
NaN means 'not a number', and '=' is an infix operator to compare numbers. You can't compare for equality on things that are not numbers.
You can compare infinities, however. "I thought you said dividing by zero was meaningless!" It is. I'm not going to divide by zero, but I can still talk about infinity. Take for example the following function:
f: (0, 1] -> R
f(x) = 1/x
f is defined on every number strictly greater then zero, but less than or equal to one. Zero is not in the domain, so I can't evaluate f there. However, I can still study what happens near zero. Take a really small number bigger than zero. The reciprocal of that is huge. Now take a smaller number, the reciprocal becomes even bigger. In fact, we can keep doing this: construct any positive sequence that converges to zero (but never attains it). Now evaluate f at each element in that sequence, you will notice that the result becomes larger and larger. We call the limit of this 'infinity', and we have notation for this. Notice I did NOT evaluate f at zero, I merely studied what happens as I approach zero. Infinity is not a real number, and I cannot have f ever evaluate to infinity, but I can talk about limiting properties. Computers do this too, but they have a much lower standard for doing so. Essentially, anything bigger than the maximum value of a double (about 10^300, I think?) is called 'Inf' on a computer.
Edit: I forgot to address the second paragraph of the parent comment.
>For example, it's probably more sensible to say that 2x/x != 3x/x for all values of x, rather than 2x/x != 3x/x for all x except 0, wherein both functions are undefined and therefore equal to each other.
This is not a sensible thing to do.
Let me elaborate. I'm going to ignore any cancellation issue and say it is definitely true that 2/x != 3/x for x != 0. Let's also say that each of these functions has domain (0, 1]. These two functions are not equal are x = 0 because you cannot evaluate them at x = 0. It is forbidden. However, behaviour near to zero is very similar. See my discussion above. In fact, you can make a stronger statement. You can say that as x approaches zero (from the positive side), in the limit, 2/x and 3/x are equal. That is, they have the same limit as you approach zero. This does not make them equal at zero, because I cannot evaluate them at zero. To make this point absolutely clear, I will add zero into the domain and define two different functions:
f:[0,1] -> R
f(x) = 2/x for x > 0
f(0) = 1
and
g:[0,1] -> R
g(x) = 3/x for x > 0
g(0) = 2
These functions have the same limit as you approach zero, but they are not equal at zero. I can evaluate them at zero this time, because I defined what it meant to do so. And indeed f(0) = 1 != 2 = g(0).
[+] [-] Dewie|11 years ago|reply
> undefined == undefined
In Haskell? Exception: Prelude [...]
[+] [-] zenogais|11 years ago|reply
It seems like the desired result is to make it determinable when a NaN value is produced in a language that doesn't have an is_nan function (such as an assembly language lacking such a predicate) so that a simple boolean statement does the trick, for example:
def is_nan(val): return not (val == val)
This absolutely bizarre statement of non-identity gives us a surefire test. It is from the perspective of higher level languages that the non-identity of NaN with itself seems bizarre.
[+] [-] dllthomas|11 years ago|reply
def is_nan(val): return (val == NaN)
It's true that it can't just give NaN or you'd be stuck.
[+] [-] marcosdumay|11 years ago|reply
But then, it's indeed quite surprizing. It get worse on pure tanguages where the expressions (0 / 0) and (0 / 0) are equal before evaluation, but different afterwards. Also, it trashes hash tables.
Yet, NANs being different is the sanest mathematical definition. Maybe people should have opted for the ugliest choice because of real world concerns (it wouldn't be the first time), but this time, they didn't.
[+] [-] xenadu02|11 years ago|reply
Any invalid calculation produces NaN as opposed to raising an exception/signal because that would be non-portable (especially in the early 80s). The goal was that a correct algorithm on one platform or in one language would translate and be correct somewhere else. Further, some algorithms depend on being able to probe a function to find its bounds, requiring them to be able to locate the out of bounds condition without halting execution; doing that in a safe, portable way is actually quite hard unless you do it in software as part of the spec, hence NaN.
If NaN==NaN, then (NaN/NaN)==1, etc etc and you still end up in a world of nonsense as far as naive algebra is concerned. This is just some of my fellow programmers whining and crying about having to think about floating point, the same sort of complaining I hear about Unicode (and full of just as much self-sure awful advice that produces wildly incorrect results for everything except the complainer's specific use case).
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.ht...
http://www.eecs.berkeley.edu/~wkahan/ieee754status/754story....
http://cnx.org/content/m32770/latest/
[+] [-] lbarrett|11 years ago|reply
[+] [-] shalmanese|11 years ago|reply
[+] [-] shawnz|11 years ago|reply
[+] [-] tannerc|11 years ago|reply
But is this an effective way of looking at the question? In reality what you're trying to do is not compare a NaN to another NaN, you're actually attempting to check if both objects are NaN. This hurts my brain.
[+] [-] malkia|11 years ago|reply
[+] [-] dllthomas|11 years ago|reply
Much better, in terms of early failure, would be signalling NaN or something like...
Of course, there are other dimensions in which this is not better, but I don't like silent propagation if it might stretch over a lot of code.[+] [-] letney|11 years ago|reply
For textures, they are great as they don't interpolate w/ nearby neighbors and properly discard fragments & vertices without the aforementioned cost of an extra boolean attribute.
[+] [-] quasque|11 years ago|reply
[+] [-] unknown|11 years ago|reply
[deleted]
[+] [-] zmmmmm|11 years ago|reply
[+] [-] jamesbrownuhh|11 years ago|reply
http://www.youtube.com/watch?v=HAw2jU4de2A
[+] [-] InclinedPlane|11 years ago|reply
[+] [-] wglb|11 years ago|reply
Most here don't remember the totally sad state of floating point before this came to be.
[+] [-] personjerry|11 years ago|reply
[+] [-] dsugarman|11 years ago|reply
[+] [-] leeoniya|11 years ago|reply
[+] [-] tybenz|11 years ago|reply
[+] [-] klodolph|11 years ago|reply
The nice thing about NaN is that you can just do a calculation as normal, and check for NaN at the very end. This means you don't have to do an expensive test/branch after every arithmetic or other numeric operation. The hardware is much, much easier to design if you don't have to make it branch, and the code is much faster without the compiler inserting branches. People who work with floating point numbers every day care very deeply about performance.
If you don't care as much about performance, why not write your code in a language that does throw exceptions? Python, for example.
Those of us that do use numerics love NaN, love signed zeros, and can live with NaN ≠ NaN even though it's kind of dumb.
[+] [-] msandford|11 years ago|reply
What if you're doing some kind of matrix math and you just need the results of the diagonal and a single NaN creeps in somewhere but doesn't destroy your matrix. Would you accept your program needing 50% more LOC and taking 4x as long for this result?