[cdavid]

Maybe I am too mathematically enclined, but this was not easy to understand.

The ELI5 explanation of floating point: they approximately give you the same accuracy (in terms of bits) independently of the scale. Whether your number if much below 1, around 1, or much above 1, you can expect to have as much precision in the leading bits.

This is the key property, but internalizing it is difficult.

[ekelsen]

I like "between each power of 2, there are the same number of numbers."

So between 1/2 and 1 there are the same number of numbers as between 1024 and 2048. If you have 1024 numbers between each power of 2, then each interval is 1/2048 in the first case and 1 in the second case.

I reality there are usually:

bfloat16: 128 numbers between each power of 2

float16: 1024 numbers between each power of 2

float32: 2*23 numbers (~8 million) between each power of 2

float64: 2*52 numbers (~4.5 quadrillion) between each power of 2

[hnuser123456]

Or, say, you can write any number you want, but it has to be a whole number from 0 to 9, and you can only make the number bigger or smaller by moving the decimal point, and you can only move the decimal point up to 10 spaces. And you can add or remove a negative sign in front.