New research from MIT suggests that the largest and most computationally intensive AI models may soon yield smaller returns compared to smaller models. By contrasting scaling laws with ever-increasing model efficiency, the researchers found that it may become more difficult to achieve leaps in the performance of giant models, while increasing efficiency could make models running on more modest hardware more capable over the next decade.
“Over the next five to 10 years, things will likely start to taper off,” says Neil Thompson, a computer scientist and professor at the Massachusetts Institute of Technology who participated in the study.
Efficiency jumps similar to those observed with Surprisingly cheap DeepSeek model in January have already served as a reality check for the artificial intelligence industry, which is accustomed to burning through huge amounts of computing power.
As it stands, a cutting-edge model from a company like OpenAI is currently much better than a model trained using a piece of computation in an academic lab. While the MIT team's prediction may not come true if, for example, new learning methods such as reinforcement learning produce surprising new results, they suggest that large artificial intelligence companies will have less of an advantage in the future.
Hans Gundlach, a postdoctoral fellow at MIT who led the analysis, became interested in the problem because of the cumbersome nature of working with advanced models. Together with Thompson and Jason Lynch, another MIT scientist, he mapped out the future performance of advanced models compared to models built using more modest computational means. Gundlach says the predicted trend is especially pronounced for currently fashionable reasoning models that rely more on additional calculations during inference.
Thompson says the results show the value of improving the algorithm as well as scaling up the computation. “If you're spending a lot of money training these models, then you should definitely spend some of it on developing better algorithms, because that can make a huge difference,” he adds.
The research is especially interesting given today's AI infrastructure boom (or should we say bubble?) that shows no signs of slowing down.
OpenAI and other US technology companies signed deals worth one hundred billion dollars to create artificial intelligence infrastructure in the United States. “The world needs a lot more computing,” said OpenAI President Greg Brockman. announced this week when he announced a partnership between OpenAI and Broadcom to create custom artificial intelligence chips.
A growing number of experts are questioning the validity of these deals. Rough 60 percent Part of the cost of building a data center is spent on GPUs, which tend to depreciate quickly. Partnerships between major players are also emerging. round and opaque.
