Meta’s new AI chips run faster than before – The Verge
By Emilia David, a reporter who covers AI. Prior to joining The Verge, she covered the intersection between technology, finance, and the economy.
Meta promises the next generation of its custom AI chips will be more powerful and able to train its ranking models much faster.
The Meta Training and Inference Accelerator (MTIA) is designed to work best with Meta’s ranking and recommendation models. The chips can help make training more efficient and inference — aka the actual reasoning task — easier.
The company said in a blog post that MTIA is a big piece of its long-term plan to build infrastructure around how it uses AI in its services. It wants to design its chips to work with its current technology infrastructure and future advancements in GPUs.
“Meeting our ambitions for our custom silicon means investing not only in compute silicon but also in memory bandwidth, networking, and capacity as well as other next-generation hardware systems,” Meta said in its post.
Meta announced MTIA v1 in May 2023, focusing on providing these chips to data centers. The next-generation MTIA chip will likely also target data centers. MTIA v1 was not expected to be released until 2025, but Meta said both MTIA chips are now in production.
Right now, MTIA mainly trains ranking and recommendation algorithms, but Meta said the goal is to eventually expand the chip’s capabilities to begin training generative AI like its Llama language models.
Meta said the new MTIA chip “is fundamentally focused on providing the right balance of compute, memory bandwidth, and memory capacity.” This chip will have 256MB memory on-chip with 1.3GHz compared to the v1’s 128MB and 800GHz. Early test results from Meta showed the new chip performs three times better than the first-generation version across four models the company evaluated.
Meta has been working on the MTIA v2 for a while. The project was internally called Artemis and was previously reported to focus only on inference.
Other AI companies have been looking into making their own chips as the demand for compute power increases along with AI use. Google released its new TPU chips in 2017, while Microsoft announced its Maia 100 chips. Amazon also has its Trainium 2 chip, which trains foundation models four times faster than the previous version.
The competition to buy powerful chips underscored the need to have custom chips to run AI models. Demand for chips has grown so much that Nvidia, which dominates the AI chip market right now, is valued at $2 trillion.
Correction April 10th, 1:21PM ET: The story previously stated Artemis was a different chip project from Meta focused on inference. Artemis is an internal name for MTIA v2 and is not a separate chip. We regret the error.
/ Sign up for Verge Deals to get deals on products we’ve tested sent to your inbox weekly.
The Verge is a vox media network
© 2024 Vox Media, LLC. All Rights Reserved
