konstruction's comments

Looks like a nice article, but there are some rendering issues for me.

In the section "We also need to define a metric for pruning." the math afterwards (and in all other places) has a dark background, and black font. Not legible. The code seems to have a weird shadow effect that goes away if I mark the code. That doesn't seem to be intended.

I miss mention of Parkinson's law stated (1955) by the naval historian Cyril Northcote Parkinson in "The Economist":

"work expands so as to fill the time available for its completion"

https://en.wikipedia.org/wiki/Parkinson%27s_law

You can already use the embeddings as features (as input) to another model that is then trained only on the embedding vectors. In this sense, they are exchangeable.

This goes even further, as a model sophisticated enough to capture a probability distribution will produce embeddings that encode this distribution (to some extent) so that any two models of that kind produce "equivalent" embeddings that can be transformed into each other. This is an area of active research (in fact, I've just been to a seminar talk about that).

So the answer to the "How can we .." would be: by capturing the distribution, by making the embedding big enough and the training task difficult enough.

Examples of embeddings that are re-used are variants of word2vec, CLIP and CLAP.

As others have already mentioned: the hash analogy would be correct if you think about non-cryptographic hashes, but I doubt that this clarifies anything.

by Oliver Bräunling. The Leibniz formula for Pi gets more digits right when summing the first 5000 terms than when summing 6000. This happens again, e.g., summing 50,000 vs 60,000 terms.

Such unreasonably many correct digits, which then fade away again when summing more terms (only to resurface much later again), can be observed experimentally, but there is also a theoretical explanation.

The video is a homage to the paper 'Borwein, Borwein, Dilcher - Pi, Euler Numbers and Asymptotic Expansions, The American Mathematical Monthly, 1989, Vol 96, 681-687'. (for the specialists: By sweeping the error terms for the asymptotic expansion under the rug, I hope to give a more accessible and less technical explanation why these correct digits occur.)

A game playing in the Bobiverse. I would love to upgrade my von Neumann probes, clone myself and land on a megastructure to go swimming and fighting with Quinlans on the look for Bender .. You get it.

Hilarious :-)