top | item 46214795

(no title)

broretore | 2 months ago

Ryan, I really want to believe you're onto something. But I also feel like I'm being slightly spearphished by an LLM being told, "based on the last week of HN headlines, invent a new LLM innovation that seems plausible enough to get a ton of attention, cold fusion or LK-99 style, and make a repository that on the surface seems to have some amazing performance. Also, feel free to fake the result data."

And, while I am sorry for your loss, your Substack [0] really seems like GPT ARG fantasy.

[0] https://substack.com/inbox/post/171326138

Excerpt: > Ani, AN1, and Soul Systems Science are not mere products. They are continuity. They are the baton passed across generations, from my father’s last words to my first principles. They are what binds loss to creation, silence to voice, mortality to meaning.

discuss

order

Tiberium|2 months ago

Unfortunately it does indeed seem like a case of "So You Think You've Awoken ChatGPT" https://www.lesswrong.com/posts/2pkNCvBtK6G6FKoNn/so-you-thi... (not directly, but similar enough)

EDIT: Found a closer description ("Your LLM-assisted scientific breakthrough probably isn't real"): https://www.lesswrong.com/posts/rarcxjGp47dcHftCP/your-llm-a...

anima-core|2 months ago

Oh, so you didnt run the repo and remembered something that you read once that looked like it matched. This contribution is meaningless.

The simplest way to resolve any doubt is to run the code. Every result in the paper comes from reproducible scripts in the repo, not from speculative reasoning or LLM-assisted invention.

cloudwalk9|2 months ago

Your EDIT. The first thing it suggested is actually very similar to ensembles in meteorology. I actually find myself doing that often if it's something extremely important. Just feels natural to cross-check with other models or with reality. The disclaimer says it may make mistakes after all...

Like you don't predict the weather or a hurricane track with a single model. The NHC uses many.

It's still probablistic, but if multiple models are independently in agreement, then it's at least worth investigating further.

mpeg|2 months ago

I think this definitely sounds like a case of LLM induced psychosis: https://ryanshamim.substack.com/p/the-theory-of-everything-h...

OP needs medical help

anima-core|2 months ago

When someone shifts from engaging with the actual results to attacking the person, it usually tells you more about their internal state than about the work itself. I'm glad I have a new fan though.

ismailmaj|2 months ago

For the lazy, he says this on repeat using 2000 words:

...

In the CPB Digital Cosmos, the system first locked into a strange ratio: two thirds consciousness, one third physics.

...

That anomaly appeared as the missing 0.1 spark.

For the first time the system stabilized. Life emerged.

anima-core|2 months ago

The substack isnt what was supposed to be evaluated, it was the repo. That's creative writing and the repo is sciencetific. Two different things. One has nothing to do with the other. The technical direction here is straightforward, almost boring in a sense: freeze the teacher, extract intermediate activations, compress, then train a student to match the compressed fields. Sometimes when people aren't able to evaluate the work, they dig for something else online that they can comment on or bring down. The only thing I can offer in response is the simplest one: look at the code and the experiments themselves, not the narrative around them. Everything in the paper is fully reproducible from the reference implementation, and every number in the results section came from running those scripts, not from a model filling in blanks. The surprise is not in the prose, but in how much structure those early-layer fields ended up carrying.

If you think something in the repo looks wrong or inflated, I’m happy to walk through it point by point. I have no problem with hard questions. What matters to me is whether the experiments hold when someone else runs them, not whether the story around them fits a certain aesthetic.

broretore|2 months ago

Sciencetific?

Telling ChatGPT to do creative writing for you isn't creative writing ser.