top | item 44896061

(no title)

fiforpg | 6 months ago

Right, I did realize after posting that close to numbers of the form

1{hidden bit} + (1-2^-52){mantissa with all ones}

the relative accuracy — corresponding to the absolute accuracy of a single bit in mantissa — is about 2^-53. The hidden bit is easy to forget about...

discuss

order

No comments yet.