top | item 22586109

(no title)

bloomer | 6 years ago

Somewhat, in that it doesn’t use more precision than needed, but the real issue is you can’t just pick some arbitrarily large precision and round at the end. For some calculations, even 400 digits during intermediate steps would not be enough due to catastrophic cancellation and you would need to go even higher precision to get the right answer. It really is about solving an accuracy issue and not an optimization. And determining that you are using sufficient precision to get an accurate answer is an extra cost, so it is always more expensive than just plowing ahead and calculating an inaccurate answer.

discuss

order

nullc|6 years ago

Presumably there is a maximum precision or otherwise a seemingly innocuous calculation could run you out of memory.