Semiconductor Chip Making Machine
David Talukdar / Alami
The latest commodity in demand in the artificial intelligence industry is computer memory, and the sector is signing billions of dollars' worth of deals directly with chip makers – the same chips that consumers use in smartphones, laptops and game consoles. At best, this will lead to higher prices, and at worst, it will lead to shortages that limit production.
Why does AI need so much memory?
AI models are very, very large. You can think of them as grids of billions or even trillions of parameters—numbers stored in memory—on which extremely repetitive, but at scale, demanding calculations are performed. This is how a large language model takes input and generates output.
Transferring this amount of data back and forth to processors from cheap but slow hard drives—what we commonly call computer storage—creates ridiculous bottlenecks. To avoid this, huge amounts of much faster RAM – what we commonly call computer memory – are used instead.
And there is another factor: the models that artificial intelligence companies create operate on a huge scale. This means they require computers that can handle hundreds, thousands, or millions of copies of these models so that large numbers of clients can use them simultaneously.
Take an extremely compute-intensive task, scale it across a large number of users, remove expansion limits, add virtually endless investment money, and you have an insatiable demand for hardware. A company that produces several million laptops a year simply can't match it.
Why can't chipmakers just make more chips?
This is easier said than done. Semiconductor fabs have limited capacity, and building a new one requires huge investments and often takes several years.
There are also signs that producers are reluctant to end the drought. Korean media reports that Samsung Electronics and SK Hynix, which together produce about 70 percent of these chips, doesn't want to increase the offer too much in case there is a downturn in the AI ​​industry and they are left with idle and costly factories to make new chips and a shortage of orders.
And given the current growth in demand and the fact that Samsung is in a comfortable position, having the opportunity raise prices by as much as 60 percentWhy would a company rock the boat? The numbers show that the 32GB chip that Samsung sold for $149 in September was on sale for $239 in November.
Have we seen shortages like this before?
Again and again. For years, the artificial intelligence boom has seen companies vacuum everything GPU (GPU) computer chips they can build huge data centers capable of training and running ever larger models. It's this unrelenting demand that has sent chipmaker Nvidia's share price soaring from $13 at the start of 2021 to peaking at more than $200 in recent months.
In 2021 we had shortage of all types of computer chips due to a perfect storm of factors including a global pandemic, trade war, fires, drought and snow storms. It affected the production of everything from pickup trucks to microwave ovens.
That same year, we even experienced a hard drive shortage when a new cryptocurrency called Chia emerged that ran on disk space rather than computer power. gained popularity.
In short, technology is moving fast. Sometimes much faster than global supply chains.
When might the shortage end?
Not soon. OpenAI has signed agreements with Samsung and SK Hynix, under which it will receive approximately 40 percent of the world's memory capacity. And this is just one artificial intelligence company, albeit one of the giants. Microsoft, Google and ByteDance, among others, are also buying every possible chip.
One way to end shortages – and possibly quickly lead to gluts – is if Bust AI what economists, bankers and even the head of OpenAI are warning about is actually happening. But it would likely have devastating economic consequences, so it may not be a panacea.
If this bust does not arrive, it is estimated that it may be 2028 before things calm down Supply and demand are reaching equilibrium again, and some smaller firms are bringing new factories online.
Some suggest that this wait could become a problem for the entire manufacturing industry. Sanchit Vir Gogia, Industry Analyst, Greyhound Research, told Reuters that “memory shortages have now moved from a component-level problem to a macro-economic risk.”
Topics:
