top | item 20186728

(no title)

civility | 6 years ago

The person I replied to claims to have looked at large* data sets and never seen a discrepancy in 6-8 significant digits. I thought I'd show them a small data set with 3 samples that retains no significant digits.

* Never mind that "millions" isn't large by current standards...

discuss

order

firethief|6 years ago

But you are observing all the digits, not just 6-8. It's implicit in the semantics of the operation, and that's something everyone who works with floating point should know.

civility|6 years ago

I think I see where you're hung up. If I change it to:

    x = 1e20 + 1 - 1e20
    y = 1e20 - 1e20 + 1
    assert x == y
There is only 1 digit, and it's wrong. You don't even need 6-8. I probably should've used this as my example in the first place.