[qorrect]

I think they are talking about the word embeddings, where context is embedded into high geometric dimensions (one dimension might capture how 'feminine' a word is, or how 'blue' it is).

[cubefox]

Which word embeddings get their own dimension though, and which don't? ("feminine" and "blue" are words like any other)

[willy_k]

My extremely naive understanding is that the more useful ones, which also tend to be structures of language like gender or color, get their own dimensions, and other embedding are represented with combinations.

A weak illustration of this is this site[1], from an HN post a few months ago[2].

[1] https://neal.fun/infinite-craft/

[2] https://news.ycombinator.com/item?id=39205020

[exe34]

maybe it's like a PCA/Huffman deal, where the more regularly useful ones get to be the eigenvectors.

[exe34]

ooooh what if qualia is just embedding? some philosophers would get their toga in a twist!

[darby_nine]

Not Wittgenstein :cool: