top | item 45913521

(no title)

mrspuratic | 3 months ago

It seems like it, but it can't be only that. A float64 representation of an int in the range 2^58 to 2^59 should be rounded to multiples of 2^6, i.e. 348555896224571968 as you found (3.48555896224571968E17) (the final digit 9 in the math.log2() expression was lost, it's 2^58 not 2^54) The unexpected output (according to the bugreport) arises from javascript, it does NOT round like everything else for reasons I don't understand. It seems to prefer rounding to arbitrary multiples of 10 in my limited testing.

discuss

order

No comments yet.