Nvidia Trains LLM on Chip Design – EE Times
Nvidia has trained its NeMo large language model (LLM) on internal data to help chip designers with tasks related to chip design, including answering general questions about chip design, summarizing bug documentation, and writing scripts for EDA tools. Nvidia’s chief scientist, Bill Dally, presented the LLM, dubbed ChipNeMo, in his keynote presentation at the International Conference on Computer-Aided Design today.
“The goal here is to make our designers more productive,” Dally told EE Times in an interview prior to the event. “If we even got a couple percent improvement in productivity, this would be worth it. And our goals are actually to do quite a bit better than that.”
Foundation models like NeMo are generally trained on many trillions of words scraped from the internet to allow them to gain a good general understanding of language. LLMs can then be further pre-trained on domain-specific data to gain additional context in a particular field, and they can be fine-tuned with example question-answer pairs.
Nvidia pre-trained ChipNeMo using a single data set scraped from internal repositories of code and text, including architecture documents, design documents and Nvidia’s code base, Dally said, and the company then fine-tuned on a subset of that data.
“Pre-training on our design data makes a smaller model perform like a larger model,” he said. “Eventually, we want to run this on very large models. For the experimental case, we’re trying to learn a lot very quickly. We use the small models because we can train them more quickly, but ultimately we want to run it on very on some of our biggest models—and then we think it’s going to become even more useful because the results will get better.”
ChipNeMo is a single model that can take on three types of tasks, all of which are concerned with designer productivity and helping to use EDA tools more efficiently rather than actually performing chip design. The 43-billion-parameter ChipNeMo runs on a single Nvidia A100 GPU in “a few seconds,” Dally said.
ChipNeMo can answer questions about chip design, particularly from junior designers.
“It turns out our senior designers spend a lot of time answering questions from junior designers,” Dally said. “If the first thing the junior designer can do is to go to ChipNeMo and say, ‘What does this signal coming out of the memory unit do?’—and if they get a possible answer that saves the senior designer’s time, the tool is well worth it.”
To avoid hallucination, Nvidia used a technique called retrieval augmented generation (RAG).
“We use the initial prompt to query a database and pull up a bunch of source documents that are relevant to this particular query,” Dally said. “We can append it to the prompt and feed it to ChipNeMo so we can ground that response to particular source documents, which reduces the tendency to hallucinate and makes things more explainable.”
ChipNeMo can also summarize bugs that are already documented. Typical bug documentation needs to be exhaustive and can result in designers having to read long documents–even to get the basics of what the bug does.
“Bug summarization is probably the lowest-hanging fruit in getting productivity,” Dally said. “When there’s a bug filed, people throw all sorts of stuff into [our bug system]…the tool is pretty good at summarizing a bug down to a concise paragraph and then saying, here’s who should go try to fix it.”
Chip NeMo can also write short scripts (circa 20 lines of code is typical, Dally said) in Tcl, the industry standard scripting language used for EDA tools. Scripting is a typical way to interface with CAD tools.
Could EDA Tool vendors eventually use this technology as a higher level of abstraction for chip design? They could if they had access to the training data, Dally said.
“I could imagine that [EDA tool vendors] would be very interested in having [an LLM] be a more approachable human interface to the tool,” he said. “Writing scripts for a lot of these tools is an art, and if they could at least get a set of scripts for a particular tool and fine tune on that, then they could make it much easier for designers to use the tools because you could write in human language: ‘Here’s what I’d like the tool to do,’ and it would then generate the script that would direct the tool to do that.”
ChipNeMo is intended for Nvidia internal use only and will not be commercialized, though other chip companies may find success training LLMs on their own internal data, Dally said.
“[ChipNeMo] is fairly specialized to how Nvidia does things,” he said. “For example, we have a particular way of writing scripts and all the scripts that it’s seen are NVIDIA scripts, but that’s what we want, this is for internal use.”
While it likely learned a lot of generic chip-design concepts from being trained on GPU design, this would be broadly applicable to any type of digital chip design but with the particular style that Nvidia has developed over the years, Dally said.
While this technology is still currently a research project, it is being tested inside Nvidia to gather feedback from team members designing Nvidia’s future products. ChipNeMo will probably be applied to more use cases across the chip-design process in the future, Dally said.
“This tool can be applied to many stages of chip design, [perhaps] writing scripts to run logic simulations or test benches early in the process, or it could be scripts to do macro placement later on, or timing verification and rule checking in the final stages,” he said. “We’d like to do this because what limits us on designing chips is human resources: We want to give our designers ‘superpowers’ so that we can design more and better chips with the same designers, and that needs to apply throughout all the stages of the design process.”
Sally Ward-Foxton covers AI for EETimes.com and EETimes Europe magazine. Sally has spent the last 18 years writing about the electronics industry from London. She has written for Electronic Design, ECN, Electronic Specifier: Design, Components in Electronics, and many more news publications. She holds a Masters’ degree in Electrical and Electronic Engineering from the University of Cambridge. Follow Sally on LinkedIn
Using a LLM for mentoring just feels wrong somehow.
I think it would be a good teaching aid, for example to teach about AI algorithms, as part of an organized and supervised course. It needs feedback to stay on track.
You must Register or Login to post a comment.
This site uses Akismet to reduce spam. Learn how your comment data is processed.
Advertisement
