WingNews logo WingNews
top | item 30945429

Why does Excel get 213,458,046,676,875 x 29 wrong?

2 points| christudor | 3 years ago

When I put =PRODUCT(213458046676875,29) into Excel, it gives me the answer 6,190,283,353,629,370.

The actual answer is 6,190,283,353,629,375.

What has happened?

2 comments

order

lavezza|3 years ago

Excel only stores 15 significant digits. Once you get to the 16th digit you'll just get a zero.

Example: 6,190,283,353,629,370 + 1 will still be 6,190,283,353,629,370. 6,190,283,353,629,370 + 10 will be 6,190,283,353,629,380

db48x|3 years ago

Floating–point numbers happened. Excel uses IEEE–754 floating–point arithmetic because it is fast and ubiquitous. One of the disadvantages of doing math this way is that as you get farther away from zero, the precision of the numbers goes down. It’s generally fine because most people don’t do arithmetic on 6 quadrillion and change, they do arithmetic on smaller, more approachable numbers.

Look for a way to solve your problem using software that uses unlimited–precision arithmetic instead.