Ironwood is Google's Latest Tensor Processor
Nvidia's position as the dominant supplier of artificial intelligence chips could be threatened by a custom chip pioneered by Google. like Meta And anthropic plan to spend billions on Google tensor processors.
What is TPU?
The success of the artificial intelligence industry is largely based on GPUs (GPU), a kind of computer chip that can perform many parallel calculations simultaneously, rather than one after the other, like the computer processors (CPUs) that run most computers.
GPUs were originally designed for computer graphics, as the name suggests, and for gaming. “If I have a lot of pixels in space and I need to rotate them to calculate a new camera image, this operation can be done in parallel for many different pixels,” says Francesco Conti at the University of Bologna in Italy.
This ability to perform calculations in parallel has proven useful for training and running AI models, which often use calculations involving large grids of numbers performed simultaneously, called matrix multiplication. “GPUs are a very general architecture, but they are ideal for applications that exhibit high parallelism,” says Conti.
However, because they were not originally designed with AI in mind, the ways in which GPUs transform the computations performed on chips may not be efficient. Tensor processing units (TPUs), which were originally developed by Google in 2016, are instead designed exclusively for matrix multiplication, says Conti, which is the core computation needed to train and run large AI models.
This year Google released seventh generation TPU called Ironwoodwhich powers many of the company's artificial intelligence models, such as Twins and protein modeling AlphaFold.
Are TPUs much better than GPUs for AI?
From a technology standpoint, TPUs are more of a subset of GPUs than a completely different chip, he said. Simon Mackintosh-Smith at the University of Bristol, UK. “They focus on the things that GPUs do, more specifically aimed at training and inference for AI, but they're actually more like GPUs in some ways than you think.” But because TPUs are designed with specific AI applications in mind, they can be much more efficient for those tasks and potentially save tens or hundreds of millions of dollars, he says.
However, this specialization also has its downsides and can make TPUs inflexible if AI models change significantly from generation to generation, Conti says. “If you don't have flexibility in your [TPU]you have to do [calculations] on the processor of your node in the data center, and that will slow you down significantly,” says Conti.
One of the benefits that Nvidia GPUs have traditionally had is the availability of simple software that can help AI developers run their code on Nvidia chips. This wasn't the case with TPUs when they first came out, but the chips are now at a point where they're easier to use, Conti says. “Now with TPU you can do the same [as GPUs]”,” he says. “Now that you've done that, it's clear that affordability is becoming a major factor.”
Who builds the transport hub?
While Google was the first to launch TPUs, many of the largest artificial intelligence companies (known as hyperscalers) as well as smaller startups have now begun developing their own custom TPUs, including Amazon, which uses its own Trainium chips to train its AI models.
“Most hyperscalers have their own internal programs, and that's partly because GPUs have become so expensive because demand has outstripped supply, and it might have been cheaper to design and build your own,” McIntosh-Smith says.
How will TPUs impact the artificial intelligence industry?
Google has been developing its TPUs for over a decade, but mainly uses these chips for its own artificial intelligence models. What's changing now is that other large companies like Meta and Anthropic are making significant purchases of computing power from Google's TPUs. “What we haven't heard about is large clients moving on, and maybe that's what's starting to happen now,” McIntosh-Smith says. “They are old enough and they are enough.”
In addition to creating more choice for larger companies, diversification could make good financial sense for them, he said. “It might even mean you'll get a better deal from Nvidia in the future,” he says.
Topics:
