(no title)
fiforpg | 6 months ago
> the largest integer value that can be represented exactly is 2^53
— I am confused as to why it not 2^52, given that there are 52 bits of mantissa, so relative accuracy is 2^-52, which translates to absolute accuracy larger than 1 after 2^52. Compare this to the table there saying "Next value after 1 = 1 + 2^-52".
pclmulqdq|6 months ago
fiforpg|6 months ago
1{hidden bit} + (1-2^-52){mantissa with all ones}
the relative accuracy — corresponding to the absolute accuracy of a single bit in mantissa — is about 2^-53. The hidden bit is easy to forget about...
layer8|6 months ago