Nvidia Trains LLM on Chip Design

[falcor84]

"""Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind... Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control. It is curious that this point is made so seldom outside of science fiction. It is sometimes worthwhile to take science fiction seriously. """

I. J. Good, in 1965 - https://en.wikipedia.org/wiki/I._J._Good

[arbuge]

> provided that the machine is docile enough to tell us how to keep it under control.

That part of his statement wasn't accurate.

Should be that the machine is docile enough for that, AND its descendants are too, and their descendants, and so on down the line as long as new and improved generations keep getting created.

[omneity]

Keeping in mind, this holds true in a runaway fashion if the only bottleneck to more intelligence is further intelligence.

I suspect physical limitations similar to how many runaway processes in the universe are more logistical than exponential in nature.

[m3kw9]

How does it magically run away? What’s the process, we all talk about it “running away” leaving us “behind”, the exact practical process of that happening has not been laid out other than people hand wavingly copying apocalyptic movie scripts.

Most ai experts just say it could end us, but suspiciously never gives a detailed plausible process and people suspicious just say oh yeah, it could, and there is a bubble over their head thinking about Terminator or Hal9000 something something

[bee_rider]

We’ve seen the sort of output that LLMs produce, it can be good but also it just makes things up. So, this might produce good designs but ones that still need to be checked by a human in the end. This sort of thing just makes humans better, we’re still at the wheel.

Or maybe it could be used as a heuristic to speed up something tedious like routing and layout (which, I don’t work in the space, but I’m under the impression that it is already pretty automated). Blah, who cares, human minds shouldn’t be subjected to that kind of thing.

[cedws]

I feel like this intelligence explosion idea is foolish but I don't really have the language to explain why.

There are underlying limits to the universe, some of which we still have to discover. A machine intelligent to improve itself may only be able to do so extremely slowly in minute increments. It might also be too overspecialised, so it can improve itself but not do anything of use to us.

I think we will eventually discover reasons we cannot achieve a simultaneously performant, controllable, and generally intelligent machine. We might only be able to have one or two of these traits in a single system.

[ethanbond]

Don’t worry, it’ll be good because it’s trained on human stories, in which usually the good guy wins.

[dekhn]

I was going to comment I hadn't seen that quote before, but I just went back and checked and it's in The Coming Technology Singularity by Vinge. https://edoras.sdsu.edu/~vinge/misc/singularity.html

[ChatGTP]

What I don't get about the idea of an intelligence "explosion". In what direction should the intelligence explosion decide to go in and to what end? I mean you could say "in any direction", but that would seem kind of "stupid"? If you had infinite possibilities to choose from, which path would you explode towards?

At some stage does "something" become so smart, that it doesn't make sense, even to itself. Imagine an ultra complex system changing at light speed, at what point does it trip itself up? I've worked with people like this, people who were ultra smart but couldn't slow down and actually achieve much.

I love these thought experiments because personally, they make me realize that what we think about intelligence, consciousness and the self might be wrong. For me personally, they seem to have a Zen Koan type impact.

[dharma1]

More detail on how he saw some of the outcomes https://gizmodo.com/why-a-superintelligent-machine-may-be-th...

[carabiner]

In a panic, we try to pull the plug.

[codethief]

> an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,'

Anyone who knows anything about convergence & asymptotic behavior would beg to disagree (based on the given assumptions).

[throwaway4good]

The singularity where machines become so smart that they will run the planet for us and we can just relax and enjoy.

My guess is that in 50 years technologists will be fantasizing about the same thing.

[make3]

I hate the term science fiction, because it encompasses pretty serious science based studies of possible futures (like the book Hail Mary or Aldous Huxley's Brave New World) with complete star-wars-like nonsense, which make the average person think of sci fi as teenager nonsense.

Similarly, here, scifi oversimplifies the situation quite a bit, anthropomorphizing a machine's intelligence, assuming that an intelligent machine would be intelligent in the same way a human would be, in an equally spread out way as a human would, and would have goals & rebel in a similar way a human would

[hasbot]

The title is a bit misleading as the first sentence says "to help chip designers with tasks related to chip design, including answering general questions about chip design, summarizing bug documentation, and writing scripts for EDA tools."

Still pretty cool though.

[gumby]

Isn’t that what chip design is?

[xnx]

Google has been using machine learning for chip design since at least 2021: https://www.nature.com/articles/s41586-021-03544-w

Hasn't brought about the singularity yet.

[atx_ml_guy]

While I have no doubt that Google is working on machine learning applications for chip design, there have been a number of concerns raised with that paper:

https://retractionwatch.com/2023/09/26/nature-flags-doubts-o...

[scrlk]

DEC did it in the 1980s: https://en.wikipedia.org/wiki/VAX_9000#SID_Scalar_and_Vector...

[passion__desire]

Google’s Chip Designing AI - https://www.youtube.com/watch?v=zR9IusOpEzk

Analog Chip Design is an Art. Can AI Help? - https://www.youtube.com/watch?v=lNypq1XuZRo

[taneq]

That’s the best thing about a singularity, you often can’t tell when you cross the event horizon.

[thunkshift1]

This is one single step out of many in chip design for which they used machine learning. It will not produce anything revolutionary.

[DesiLurker]

there is a rock hurling directly towards earth but how bad could it be because Hey we are not dead yet.

[ballenf]

The first chip they give it to design should be an ML chip that is optimized for ML chip design.

[I_Am_Nous]

It's turtles all the way down :P

[WesSouza]

"Nvidia Trains LLM on Chip Design" + " documentation"

[andy_ppp]

When I see the headline "LLM trains Nvidia on Chip Design" I'll start to worry :-/

[jstummbillig]

I can't be alone in just assuming that everyone with a few millions to spare is training LLMs to help with their problem right now.

[einpoklum]

Well, at least everyone with too much money on their hands and no useful notion of how to spend it.

Hey, I have an idea! Let me train an LLM to make suggestions on how to use LLMs! That can't fail, can it?

[lucubratory]

Limiting factor actually isn't money right now, it's compute. Anyone with the compute to spare is doing it though, yes.

[kumarski]

https://motivo.ai

https://www.silimate.com/

https://www.astrus.ai

[dddrh]

Interesting concept that raised the question for me: What is the primary limiting factor right now that prevents LLM’s or any other AI model to go “end to end” on programming a full software solution or full design/engineering solution?

Is it token limitations or accuracy the further you get into the solution?

[thechao]

LLM's can't gut a fish in the cube when they get to their limits.

On a more serious note: I think the high-level structuring of the architecture, and then the breakdown into tactical solutions — weaving the whole program together — is a fundamental limitation. It's akin to theorem-proving, which is just hard. Maybe it's just a scale issue; I'm bullish on AGI, so that's my preferred opinion.

[drsopp]

I guess this would be the context window size in the case of LLMs.

Edit: On second thought, maybe at a certain minimum context window size it is possible to cajole the instructions in such a way that you at any point in the process make the LLM work at a suitable level of abstraction more like humans do.

[meiraleal]

Memory and finetuning. If it was easy to insert a framework/documentation into GPT4 (the only model capable of complex software development so far in my experience), it would be easy to create big complex software. The problem is that currently the memory/context management needs to be done all by the side of the LLM interaction (RAG). If it was easy to offload part of this context management on each interaction to a global state/memory, it would be trivial to create quality software with tens of thousands of LoCs.

[einpoklum]

It is the fact that LLM's can't and don't try to write valid programs. They try to write something which reads like a reply to your question, using their corpus of articles, exchanges etc. That's not remotely the same thing, and it's not at all about "accuracy" or "tokens".

[anon291]

The issue with transformers is the context length. Compute wise, we can figure out the long context window (in terms of figuring out the attention matrix and doing the calculations). The issue is training. The weights are specialized to deal with contexts only of a certain size. As far as I know, there's no surefire solution that can overcome this. But theoretically, if you were okay with the quadratic explosion (and had a good dataset, another point...) you could spend money and train it for much longer context lengths. I think for a full project you'd need millions of tokens.

[einpoklum]

NVIDIA trains press on clickbait and gimmicks. Result: Stellar success!

[antimatter15]

The code generation tool better be called "Tcl me NeMo"

[moistoreos]

> “If we even got a couple percent improvement in productivity, this would be worth it. And our goals are actually to do quite a bit better than that.”

So your engineers will get a productivity salary increase, right? RIGHT?!

[chii]

The engineers get paid to do the work, but won't partake in the profits of the company (except by prior agreement like a bonus target reached etc). I suppose equity grants straddle the fence regarding this, since such productivity gains would translate to higher prices for equity.

[4b11b4]

reminding me of flux.ai for PCBs

[carabiner]

No! LLM's must be destroyed!