Chip Pioneers to Battle it out in Modern RISC-V AI Chips – HPCwire
Since 1987 – Covering the Fastest Computers in the World and the People Who Run Them
Since 1987 – Covering the Fastest Computers in the World and the People Who Run Them
August 25, 2022
Some chip pioneers from the 1980s are raising the ante in modern chip design with new opportunities provided by artificial intelligence and the open-source RISC-V architecture.
Untether AI, which was co-founded by an analog and mixed signal chip pioneer Martin Snelgrove, released a new AI inferencing chip called Boqueria, which has more than 1,400 optimized RISC-V processors.
That chip will compete with Esperanto’s ET-SoC-1 AI inferencing chip, which has over 1,088 cores, and started sampling earlier this year. Esperanto’s co-founder is Dave Ditzel, who is known for pioneering work on RISC chips, which includes the development of Sun’s SPARC processors.
Tenstorrent is also building a RISC-V chip for AI under the guidance of CTO Jim Keller, who is known for his breakthroughs in mobile and PC chips through stints with Apple, AMD and Intel.
These companies are among many chip startups focusing on AI. The RISC-V instruction set architecture, which is free to license and customize, has spawned low-cost chip development opportunities for a host of new applications that include AI.
Untether announced the Boqueria chip at the Hot Chips conference this week. The company claimed the performance crown in AI inferencing performance, with 2 petaflops of performance on the FP8 data type. The chip also performs 30 teraflops per watt.
That’s an overall improvement from the company’s predecessor AI chip, RunAI, which was released in 2020 and delivered 500 teraflops of performance, and an efficiency of 8 teraflops per watt.
“In inference, you want to make sure that you’re doing things as efficiently as possible and using the right data types to ensure the necessary accuracy, but at the same standpoint, providing energy efficiency,” said Robert Beachler, vice president of product and hardware engineering, during a presentation at Hot Chips.
The company said its benchmarks on performance and performance-per-watt outperformed comparable chips on MLPerf, which is emerging as an industry standard AI benchmark.
But the company hasn’t opened its benchmarks to MLPerf, and plans to do so when it has the resources available.
Untether AI took a new design approach to ensure efficiency in data movement by putting more compute resources where data exists. For example, a computing element is directly attached to memory cells.
“When we sat down to architect for AI inference acceleration, the first thing we saw is that 90% of the energy in doing neural network computing is in movement of data, either from external memory or on internal caches. And only 10% is really done in the computer,” Beachler said.
The chip has 729 memory banks, with two RISC-V CPU cores running at 1.3GHz for each memory bank.
“Because the processing elements have their own dedicated memory, we have tremendous memory bandwidth on the order of a petabyte of memory bandwidth,” Beachler said.
Boqueria supports the PCI-Express interconnect. The company can plug in up to six chips in a single PCI-Express card to run large natural language processing networks. A six-chip PCI card will consume 400 watts of power, but typical workloads will run at the 200- to 300-watt range.
Boqueria can also be scaled down to fewer memory banks for customers who don’t run large-scale inferencing applications. The family of AI will scale from sub-one watt all the way up to the new chip described at Hot Chips.
“This allows us to address multiple different price performance points and form factors,” which includes integration in chiplets with direct die-to-die interconnects to other chips in a system-on-chip package, Beachler said.
The chip supports many data types, from INT4 to FP16, for better accuracy. The company researched different data types, and concluded that the 16-bit data types, like BF16, provide as much as accuracy as FP32, but using lesser power and compute resources.
“We’ve also found that we’ve developed a new FP8 data type that quadruples the efficiency from BF16, but at the same time preserves the accuracy necessary for today’s AI workloads,” Beachler said.
The RISC-V CPU in Boqueria is a standard architecture with more than twenty custom instructions to take advantage of the design and inference workloads.
More Off The Wire
Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!
July 8, 2024
Generative AI will take on a larger chunk of the world’s power consumption to keep up with the hefty hardware requirements to run applications. “AI chips represent 1.5% of electricity use over the next five years, a subs Read more…
July 5, 2024
Senate leader, Chuck Schumer, and three colleagues want the US government to spend at least $32 billion annually by 2026 for non-defense related AI systems. This includes spending on government and private-sector resea Read more…
July 3, 2024
A funny thing happened on the way to the AI promised land: People realized they need data. In fact, they realized they need large quantities of a wide variety of data, and that it would be better if it was fresh, trusted Read more…
July 3, 2024
Recently, an interesting paper appeared on Arvix called Use Cases for High-Performance Research Desktops. To be clear, the term desktop in this context does not refer to a machine but rather a computing desktop environme Read more…
July 2, 2024
Trapped ion quantum computing specialist Quantinuum and University of Colorado (Boulder) researchers reported yesterday they had implemented nonlocal qLDPC codes for the first time and exceeded the breakeven point (error Read more…
July 2, 2024
IonQ, the trapped ion quantum computing specialist, delivered a progress report last week firming up 2024/25 product goals and reviewing its technology roadmap. Next up on the product roadmap is Forte Enterprise, intende Read more…
July 8, 2024
Generative AI will take on a larger chunk of the world’s power consumption to keep up with the hefty hardware requirements to run applications. “AI chips repres Read more…
July 5, 2024
Senate leader, Chuck Schumer, and three colleagues want the US government to spend at least $32 billion annually by 2026 for non-defense related AI systems. T Read more…
July 3, 2024
Recently, an interesting paper appeared on Arvix called Use Cases for High-Performance Research Desktops. To be clear, the term desktop in this context does not Read more…
July 2, 2024
IonQ, the trapped ion quantum computing specialist, delivered a progress report last week firming up 2024/25 product goals and reviewing its technology roadmap. Read more…
June 30, 2024
Nvidia is saying that companies could make $5 to $7 for every $1 invested in GPUs over a four-year period. Customers are investing billions in new Nvidia hardwa Read more…
June 27, 2024
Everyone is aware of the inflationary model of the early universe in which the volume of space expands exponentially then slows down. AI-augmented HPC (AHPC for Read more…
June 25, 2024
Last week, the Defense Advanced Research Projects Agency (DARPA) issued an interim progress update from the second phase of its Quantum Benchmark (QB) program. Read more…
June 21, 2024
As AI continues to reach into every domain of life, the question remains as to what kind of software these tools will run on. The choice in software stacks – Read more…
May 21, 2024
Atos – via its subsidiary Eviden – is the second major supercomputer maker outside of HPE, while others have largely dropped out. The lack of integrators and Atos’ financial turmoil have the HPC market worried. If Atos goes under, HPE will be the only major option for building large-scale systems. Read more…
October 30, 2023
With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…
May 30, 2024
Consider the GPU. An island of SIMD greatness that makes light work of matrix math. Originally designed to rapidly paint dots on a computer monitor, it was then Read more…
March 18, 2024
Nvidia’s latest and fastest GPU, codenamed Blackwell, is here and will underpin the company’s AI plans this year. The chip offers performance improvements from Read more…
June 30, 2024
Nvidia is saying that companies could make $5 to $7 for every $1 invested in GPUs over a four-year period. Customers are investing billions in new Nvidia hardwa Read more…
August 17, 2023
The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…
June 10, 2024
Nvidia had an explosive 2023 in data-center GPU shipments, which totaled roughly 3.76 million units, according to a study conducted by semiconductor analyst fir Read more…
May 15, 2024
The makers of the Aurora supercomputer, which is housed at the Argonne National Laboratory, gave some reasons why the system didn’t make the top spot on the Top Read more…
December 11, 2023
Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…
June 3, 2024
In the world of AI, there’s a desperate search for an alternative to Nvidia’s GPUs, and AMD is stepping up to the plate. AMD detailed its updated GPU roadmap, w Read more…
April 30, 2024
It’s a long wait for customers hanging on for Intel’s next-generation GPU, Falcon Shores, which will be released in late 2025. “Then we have a rich, a very Read more…
May 17, 2024
On Tuesday May 14th, Google announced its sixth-generation TPU (tensor processing unit) called Trillium. The chip, essentially a TPU v6, is the company’s l Read more…
February 8, 2024
Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…
May 7, 2024
We have all thought about it. No one has done it, but now, thanks to HPC, we see what it looks like. Hold on to your feet because NASA has released videos of wh Read more…
March 27, 2024
Pictures of Nvidia’s new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing po Read more…
July 2, 2024
IonQ, the trapped ion quantum computing specialist, delivered a progress report last week firming up 2024/25 product goals and reviewing its technology roadmap. Read more…
© 2024 HPCwire. All Rights Reserved. A Tabor Communications Publication
HPCwire is a registered trademark of Tabor Communications, Inc. Use of this site is governed by our Terms of Use and Privacy Policy.
Reproduction in whole or in part in any form or medium without express written permission of Tabor Communications, Inc. is prohibited.
