ResearchAgent: Iterative Research Idea Generation Using LLMs

[pedalpete]

I've found where LLMs can be useful in this context is around free-associations. Because they don't really "know" about things, they regularly grasp at straws or misconstrue intended meaning. This, along with the volume of language (let's not call it knowledge) result in the LLMs occasionally bringing in a new element which can be useful.

[gotts]

Can you list some examples where free-associations from LLM were useful to you?

[robwwilliams]

This approach is already useful in functional genomics. A common type of question requires analysis of hundreds of potentially functional sequence variants.

Hybrid LLM+ approaches are beginning to improve efficiency of ranking candidates and even proposing tests and soon I hope—higher order non-linear interactions among DNA variants.

[deegles]

I like thinking of LLMs as "word calculators." Which I think really encapsulates how they aren't "intelligent" as the marketing would have you believe but also show how important the inputs are.

[KhoomeiK]

A group of PhD students at Stanford recently wanted to take AI/ML research ideas generated by LLMs like this and have teams of engineers execute on them at a hackathon. We were getting things prepared at AGI House SF to host the hackathon with them when we learned that the study did not pass ethical review.

I think automating science is an important research direction nonetheless.

[srcreigh]

That’s pretty wild. What was the reason behind failing ethics review?

[brigadier132]

I don't think LLMs are the right approach for this. Coordinated science would basically be a search problem where we verify different facts using experiments and use what we learn to determine what experiment to do next.

[barathr]

This strikes me as similar to Cargo Cult Science.

https://calteches.library.caltech.edu/51/2/CargoCult.htm

https://metarationality.com/upgrade-your-cargo-cult

[UncleOxidant]

The ideas aren't the hard part.

[tokai]

This. Any researcher should, over a lunch, be able to generate more idea than can be tackled in a life time.

[fpgamlirfanboy]

Tell that to PhD advisor that took credit for all my work because they were his ideas (at least so he claimed).

[SubiculumCode]

In some fields of research, the amount of literature out there is stupendous, and with little hope of a human reading, much less understanding the whole literature. Its becoming a major problem in some fields, and I think, in some ways, approaches that can combine knowledge algorithmically are needed, perhaps llms.

[wizzwizz4]

Traditionally, that's what meta-analyses and published reviews of the literature have been for.

[deegles]

It would be fun to pair this with an automated lab that could run experiments and feed the results into generating the next set of ideas.

[imranq]

Check out: https://www.insitro.com/

They have an automated robotics powered research lab

[geraneum]

What would an automated lab look like?

[not-chatgpt]

Cool idea. Never gonna work. LLMs are still generative models that spits out training data, incapable of highly abstract creative tasks like research.

I still remember all the GPT-2 based startup idea generators that spits out pseudo-feasible startups.

[bigyikes]

Ignoring the “spits out training data” bit which is at best misleading, it’s interesting that you use the word “abstract” here.

I recently followed Karpathy’s GPT-from-scratch tutorial and was fascinated with how clearly you could see the models improving.

With no training, the model spits out uniformly random text. With a bit of training, the model starts generating gibberish. With further training, the model starts recognizing simple character patterns, like putting a consonant after a vowel. Then it learns syllables, and then words, and then sentences. With enough training (and data and parameters, of course) you eventually yield a model like GPT-4 that can write better code than many programmers.

It’s not always that clear cut, but you can clearly observe it moving up the chain of abstraction as the training loss decreases.

What happens when you go even bigger than GPT-4? We have every reason to believe that the models will be able to think more abstractly.

Your “never gonna work” comment flies in the face of exponential curve we find ourselves on.

[ramraj07]

I have asked chat GPT to generate hypotheses on my PhD topic that I know every single piece of existing literature about and it actually threw out some very interesting ideas that do not exist out there yet (this was before they lobotomized it).

[growthwtf]

I think that ship has sailed, if you believe the paper (which I do).

LLMs are already super-human at some highly abstract creative tasks, including research.

There are numerous examples of LLMs solving problems that couldn't be found in the training data. They can also be improved by using reasoning methods like truth tables or causal language. See Orca from Microsoft for example.

[CuriouslyC]

they don't just spit out training data, they generalize from training data. They can look at an existing situation and suggest lines of experimentation or analysis that might lead to interesting results based on similar contexts in other sciences or previous research. They're undertrained on bleeding edge science so they're going to falter there but they can apply methodology just fine.

[llm_trw]

They just need to be better at it than humans, which is a rather low bar when you go beyond two unrelated fields.

[krageon]

When you're this confident and making blanket statements that are this unilateral, that should tell you you need to take a step back and question yourself.