During the presentation of its new products, Qualcomm spends a lot of time talking about its Hexagon NPU processors. Keen observers may remember that the brand was reused across the company's line of digital signal processors (DSPs), and there's a good reason for that.
“Our journey in AI processing started probably 15 or 20 years ago, when our first entry point was signal processing,” said Vinesh Sukumar, head of AI products at Qualcomm. DSPs have a similar architecture to NPUs, but are much simpler and focused on processing audio (such as speech recognition) and modem signals.
The NPU is one of many components in modern SoCs.
1 credit
As the set of technologies we call “artificial intelligence” has matured, engineers have begun to use DSPs for more types of parallel processing, such as long short-term memory (LSTM). Sukumar explained that as the industry became fascinated with convolutional neural networks (CNNs), the technology behind applications such as computer vision, DSPs began to focus on matrix functions, which are also needed for generative AI processing.
While there is an architectural pedigree here, it wouldn't be entirely accurate to say that NPUs are just fancy DSPs. “If we talk about DSP in general, then yes, [an NPU] “is a digital signal processor,” said MediaTek assistant vice president Mark Odani. “But this has come a long way and is much more optimized for parallelism, running converters and storing a huge number of parameters for processing.”
Even though NPUs are so widely used in new chips, they are not strictly necessary to run AI workloads at the “edge,” a term that distinguishes local AI processing from cloud-based systems. Processors are slower than NPUs, but can handle small workloads without consuming as much power. Meanwhile, GPUs can often process more data than NPUs, but they use more power to do so. According to Qualcomm's Sukumar, sometimes you may need to do this. For example, running AI workloads while gaming can benefit the GPU.
