top | item 46953548

(no title)

alisonkisk | 21 days ago

Here's an intuitive description of the entropy, [log(log(n)) -sum(log(p_i) log(log(p_i)))]:

The entropy of a random integer N is the volume of the gap between how much space N takes up and how much space its internal components take up.

This can be visualized as The City of N, in base 2. (OP used log_e, but that's too hard to draw.)

1. The Foundation (The Factors)

Take a random number N and break it into its prime factors. We write these prime factors in binary, side-by-side, along the bottom of a page.

The total width of this baseline is roughly log_2(N)(the number of bits in N).

2. The Cloud Ceiling (The Potential)

We write down the length of (N written down in base 2) in base 2. (If N = 46 = 101110 (base 2), its length is ~6 = 110 (base 2),

We write that number vertically (110) to set the Maximum Ceiling Height.

Finally, we look at the number N itself.

3. The Buildings (The Structure):

Above each prime factor, we construct a building.

* The Width: The width of the building is simply the length of that prime factor in bits.

* The Height: To determine how tall the building is, we look at its width and write that number down vertically in binary.

To normalize, we zoom our camera so the length (log) of N fills the view.

The Entropy (The Visible Sky):

The Sky: This is the empty space between the tops of the buildings and the top of the picture (cloud ceiling).

The Entropy of N is exactly the total area of the visible sky.

If N is prime, the building is as wide and tall as the whole city and touches the cloud ceiling. No Sky. Zero Entropy.

If N is a random integer, it usually has one wide building (the largest prime) that is almost as tall as the ceiling, and a few tiny huts (small primes) that leave a massive gap of blue sky above them.

Here is the visualization for N = 46. (Binary 101110, length ~6).

          (46)
      1 0 1 1 1 0  (length ~5.5)
     ---------------------------
         |1 1 1 1 1|
         |0 0 0 0 0|  
     |1 1|1 1 1 1 1|
     +---+---------+
     |1 0|1 0 1 1 1|
      (2)    (23)

(Visualization not exact due to rounding of logarithms, and because)

Interpretation:

Building 23 is tall. It reaches Level 3 (101 is length 5). It touches the ceiling (Level 3). There is zero sky above it.

Building 2 is short. It only reaches Level 2 (10 is length 2). There is one unit of sky visible above it.

Total Entropy: The total empty area above the buildings is small (just that gap above factor 2), which matches the math: 46 is "low entropy" because it is dominated by the large factor 23.

A number with High Entropy would look like a row of low, equal-height huts, leaving a massive amount of open sky above the entire city.

discuss

order

No comments yet.