[Version467]

So they trained LLM's on a bunch of junk and then notice that it got worse? I don't understand how that's a surprising, or even interesting result?

[nazgul17]

They also tried to heal the damage, to partial avail. Besides, it's science: you need to test your hypotheses empirically. Also, to draw attention to the issue among researchers, performing a study and sharing your results is possibly the best way.

[Version467]

Yeah I mean I get that, but surely we have research like this already. "Garbage in, garbage out" is basically the catchphrase of the entire ml field. I guess the contribution here is that "brainrot"-like text is garbage which, even though it seems obvious, does warrant scientific investigation. But then that's what the paper should focus on. Not that "LLMs can get 'brain rot'".

I guess I don't actually have an issue with this research paper existing, but I do have an issue with its clickbait-y title that gets it a bunch of attention, even though the actual research is really not that interesting.

[yieldcrv]

I don’t understand, so this is just about training an LLM with bad data and just having a bad LLM?

just use a different model?

dont train it with bad data and just start a new session if your RAG muffins went off the rails?

what am I missing here

[Sxubas]

Sometimes the simplest of experiments/observations can lead to useful results: You can't do science without challenging your beliefs.

And while this result isn't extraordinary, it definitely creates knowledge and could close the gap to more interesting observations.

[Perz1val]

I seen claims that you can train the models with anything, so it would be a research to check that