snarkconjecture's comments

snarkconjecture | 23 days ago | on: Billion-Parameter Theories

Deep neural networks can generalize well even when they're far into the overparametrized regime where classical statistical learning theory predicts overfitting. This is usually called "double descent" and there are many papers on it.

snarkconjecture | 1 year ago | on: Hot take: GPT 4.5 is a nothing burger

Versions numbers for LLMs don't mean anything consistent. They don't even publicly announce at this point which models are built from new base models and which aren't. I'm pretty sure Claude 3.5 was a new set of base models since Claude 3.

What do mean by "it's a 1.0" and "3rd iteration"? I'm having trouble parsing those in context.

snarkconjecture | 1 year ago | on: Iterated Log Coding

Not really. Dirac's trick works entirely at a depth of two logs, using sqrt like unary to increment the number. It requires O(n) symbols to represent the number n, i.e. O(2^n) symbols to represent n bits of precision. This thing has arbitrary nesting depth of logs (or exps), and can represent a number to n bits of precision in O(n) symbols.

snarkconjecture | 1 year ago | on: QwQ: Alibaba's O1-like reasoning LLM

I think it's better phrased as "find the best rule", with a tacit understanding that people mostly agree on what makes a rule decent vs. terrible (maybe not on what makes one great) and a tacit promise that the sequence presented has at least one decent rule and does not have multiple.

A rule being "good" is largely about simplicity, which is also essentially the trick that deep learning uses to escape no-free-lunch theorems.

snarkconjecture | 1 year ago | on: The bunkbed conjecture is false

There are two kinds of naturalness principle in physics, sometimes called "technical naturalness" and "Dirac naturalness" respectively.

Dirac naturalness is as you describe: skepticism towards extremely large numbers, end of story. It has the flaw you (and every other person who's ever heard it) point out.

Technical (or t'Hooft) naturalness is different, and specific to quantum field theory.

To cut a long story short, the "effective", observable parameters of the Standard Model, such as the mass of the electron, are really the sums of enormous numbers of contributions from different processes happening in quantum superposition. (Keywords: Feynman diagram, renormalization, effective field theory.) The underlying, "bare" parameters each end up affecting many different observables. You can think of this as a big machine with N knobs and N dials, but where each dial is sensitive to each knob in a complicated way.

Technical naturalness states: the sum of the contributions to e.g. the Higgs boson mass should not be many orders of magnitude smaller than each individual contribution, without good reason.

The Higgs mass that we observe is not technically natural. As far as we can tell, thousands of different effects due to unrelated processes are all cancelling out to dozens of decimal places, for no reason anyone can discern. There's a dial at 0.000000.., and turning any knob by a tiny bit would put it at 3 or -2 or something.

There are still critiques to be made here. Maybe the "underlying" parameters aren't really the only fundamental ones, and somehow the effective ones are also fundamental? Maybe there's some reason things cancel out, which we just haven't done the right math to discover? Maybe there's new physics beyond the SM (as we know there eventually has to be)?

But overall it's a situation that, imo, demands an explanation beyond "eh, sometimes numbers are big". If you want to say that physical calculations "explain" anything -- if, for example, you think electromagnetism and thermodynamics can "explain" the approximate light output of a 100W bulb -- then you should care about this.

snarkconjecture | 1 year ago | on: The Webb Telescope further deepens the Hubble tension controversy in cosmology

She's saying that a different model -- one of the three disagreeing methods for distance ladder measurements -- must be wrong, because they disagree with each other. But if one or more of those models are wrong, then there's not much evidence that the LambdaCDM model is wrong.

Conversely, the hypothesis that LambdaCDM is wrong does nothing to explain why the distance ladder methods disagree.

She clearly isn't saying that any model is infallible, she's just saying that clear flaws with one set of models throw into question some specific accusations that a different model is wrong.

You actually need to pay attention to the details; the physicists certainly are. Glib contrarianism isn't very useful here.

snarkconjecture | 1 year ago | on: What Is Entropy?

That definition doesn't work well because you can have changes in entropy even if no energy is transferred, e.g. by exchanging some other conserved quantity.

The side note is wrong in letter and spirit; turning potential energy into heat is one way for something to be irreversible, but neither of those statements is true.

For example, consider an iron ball being thrown sideways. It hits a pile of sand and stops. The iron ball is not affected structurally, but its kinetic energy is transferred (almost entirely) to heat energy. If the ball is thrown slightly upwards, potential energy increases but the process is still irreversible.

Also, the changes of potential energy in corresponding parts of two Carnot cycles are directionally the same, even if one is ideal (reversible) and one is not (irreversible).

snarkconjecture | 1 year ago | on: Ultraviolet Superradiance from Networks of Tryptophan in Biological Systems

I don't think the study authors would endorse the claim that this meaningfully lends credence to Penrose's theories about consciousness. Quantum effects are all over chemistry; that's nowhere near sufficient to provide much evidence of quantum computation happening, which in turn would not necessarily suggest any kind of explanation of consciousness.

Nobody is skeptical about quantum effects (of this sort) happening in biochemistry. All of the skepticism Penrose faces is about the quantum computation and consciousness parts. So this paper won't change anyone's minds about Penrose.

page 1