[lsy]

The example given for inverting an embedding back to text doesn't help the idea that this effect is reflecting some "shared statistical model of reality": What would be the plausible whalesong mapping of "Mage (foaled April 18, 2020) is an American Thoroughbred racehorse who won the 2023 Kentucky Derby"?

There isn't anything core to reality about Kentucky, its Derby, the Gregorian calendar, America, horse breeds, etc. These are all cultural inventions that happen to have particular importance in global human culture because of accidents of history, and are well-attested in training sets. At best we are seeing some statistical convergence on training sets because everyone is training on the same pile and scraping the barrel for any differences.

[jychang]

I fail to see how that matters. You're implying that all reality is cultural, but that seems irrelevant. The same thing would apply to scientific facts, but whales not having a word for science doesn't make it not real.

If we somehow discover LLMs right after Newton discovered the theory of gravity, and then a while later Einstein discovers General Relativity, then GR would not be in the training set of the neural net. That doesn't make GR any less of a description of reality! You also can't convert General Relativity into whalesong!

But you CAN explain General Relativity in English, or in Chinese to a person in china. So the fact that we can create a mapping from the concept of General Relativity in the neural network of the brain of a human in the USA using english, to someone in china using chinese, to a ML model, is what makes it a shared statistical model of reality.

You also can't convert General Relativity to the language of "infant babble", does it make general relativity any less real?

[zer00eyz]

> You're implying that all reality is cultural,

Let's look at two examples of cultural reality:

Fan death in South Korea. Where people believe that a fan running while you sleep can kill you.

The book "Pure, White and Deadly". Where we discredited the author and his findings and spent decades blaming fat, while packing on the pounds with high fructose corn syrup.

An LLM isn't going to find some intrinsic truth, that we are ignoring, in its data set. An LLM isn't going to find issues in the reproducibility / replication crisis. I have not seen one discredit a scientific paper with its own findings.

To be clear LLM's can be amazing tools, but garbage in garbage out still applies.

[indymike]

> That doesn't make GR any less of a description of reality!

When you layer the concept of awareness into the mix it does alter reality for an individual or llm. Awareness creates interesting blind spots into our statistical models of reality.

[cainxinth]

It doesn’t matter whether the Kentucky Derby is core to reality. The point is it is part of reality. If you want to model reality with 100% accuracy, you need to know about the Kentucky Derby. The author is arguing that models are converging on something close to the platonic ideal representation. So, a perfect model with perfect translatability would in fact be able to communicate the concept of a four legged land animal (named after a being capable of impossible feats) that attempts to be faster than other animals to win a reward for a rider on its back. Whether the platonic representation hypothesis is correct or not and whether our models will ever actually get that good are different questions.

[cmiles74]

Agreed. They aren't converging on a statistical model of reality, they are converging on a statistical model of their training data. In the case of LLMs and the size of the training data it's possible they are also converging on some commonality between all text. I doubt this reveals a core truth but maybe it will give us some insight into what we all agree certain chunks of text represent (when I use this idiom, everyone understand I mean this).

[benlivengood]

You also can't translate "Mage (foaled April 18, 2020) is an American Thoroughbred racehorse who won the 2023 Kentucky Derby" into Hellenistic Greek or some modern indigenous languages because there isn't enough shared context; you'd need to give humans speaking those languages a glossary for any of the translation to make sense, or allow them to interrogate an LLM to act as the glossary.

I'd say our current largest LLMs probably contain sufficient detail to explain a concept like a named race horse starting from QCD+gravity and ending up at cultural human events, given a foothold of some common ground to translate into a new unknown language. In a sense, that's what a model of reality is. I think it's possible because LLMs figure out translation between human languages by default with enough pretraining.

[waldrews]

Your point holds, but the example of Hellenistic Greek seems ill-chosen to make that case - they had horse races and calendars, and mythological mage equivalents that would be reasonable to name a horse after, so the only thing left to map is 'American' as a geographic proper name and 'an important race in America' - which is about as translatable as it gets. Maybe if we pick one of the many cultures that never had horses, their translation would have to throw in so much context that the corresponding text is structurally different?

[cwmoore]

Man"splain a concept like...given a foothold of some common ground to translate into a new unknown language. In a sense, that's what a model of reality is" not. It is a story with motivated characters.

[edwardbernays]

Why QCD? Quantum chromodynamics, the quantized theory of the nuclear strong force? There is also QED, quantum electrodynamics, which is the quantized field theory for electrodynamics, and then also QFD (quantum flavordynamics) for the weak force. Do you seriously mean to imply that the quantum field theory corresponding to ONLY the strong force, plus gravity, explains every emergent phenomena from there to culture? Fully half of the fundamental forces we account for, in two disparate theoretical frameworks?

[photonthug]

> You also can't translate .. into Hellenistic Greek or some modern indigenous languages because there isn't enough shared context; you'd need to give humans speaking those languages a glossary for any of the translation to make sense

What? By substitution, this means you can translate it. As long as we're assuming a large enough basis of concept vectors of course it works.

> I'd say our current largest LLMs probably contain sufficient detail to explain a concept like a named race horse starting from QCD+gravity and ending up at cultural human events

What? I'm curious how you'd propose to move from gravity to culture. This is like TFAs assertion that the M+B game might be as expressive as 20 questions / universal. M or B is just (bad,sentient) or (good,object). Sure, entangling a few concepts is slightly more expressive than playing with a completely flattened world of 1/0 due to some increase in dimensionality. But trying to pinpoint anything like (neutral,concept) fails because the basis set isn't fundamentally large enough. Any explanation of how this could work will have to cheat, like when TFA smuggles new information in by communicating details about distance-from-basis. For example to really get to the word or concept of "neutral" from inferred good/bad dichotomy of bread/mussolini, you would have to answer "Hmmmmmm, closer to bread I guess" in one iteration and then "Umm.. closer to Mussolini I guess" when asked again, and then have the interrogator notice the uncertainty/hesitation/contradiction and then infer neutrality. This is just the simple case.. physics to culture seems much harder

[Invictus0]

You could settle this fairly easily by training two small models on highly disparate datasets, maybe historical Chinese texts and historical greek texts, in their native languages, and seeing if the same similarities recur.