The NPU quietly became standard hardware

A few years ago, a neural processing unit was a line item you noticed mostly in phone chips, and it mostly handled camera tricks — face detection, scene recognition, the computational photography that made a tiny sensor look good. Now it is turning up as an expected block in laptop and desktop processors, listed alongside the CPU and GPU as a core part of the chip. Microsoft went as far as drawing a hard line: to wear the "Copilot+ PC" badge, a Windows machine needs an NPU rated at 40 TOPS or more. A hardware spec became a marketing tier. That shift happened without much fanfare, but it tells you a lot about what these companies expect software to start doing.

What an NPU is for#

An NPU is a block of silicon built to run one kind of math very efficiently: the dense matrix multiplication — millions of multiply-and-add operations — that sits at the heart of every neural network. A CPU can do that math, and a GPU can do it faster, but both pay a heavy power cost. The NPU is deliberately narrower. It runs those operations at low precision — usually 8-bit integers, sometimes lower — which inference tolerates well, and it does so at a fraction of the energy.

The win is not peak speed. A big GPU still wins there. The win is performance per watt. That matters most on devices that run on a battery, or that need to keep a feature running constantly without draining the system or spinning up a fan.

Think of the division of labor this way:

• CPU. General work, anything unpredictable, the glue that holds a program together.
• GPU. Heavy parallel work, including the largest and most demanding AI models, at a high power draw.
• NPU. Sustained, modest AI tasks at low power, the kind that can run quietly in the background.

That split is a useful simplification, not a hard wall — and, as we'll see, the line between the GPU and the NPU is already starting to blur.

Why it is becoming standard#

The move to put an NPU in almost every new processor is a bet, not a response to existing demand. Chip designers are wagering that a steady stream of features will start leaning on local AI: live transcription, noise removal, image cleanup, on-device assistants, and search that understands intent rather than keywords. None of these individually justifies the silicon. Together, and running constantly, they do.

Microsoft's 40-TOPS line is the clearest sign the bet is being placed in public. An entire class of Windows features now sits behind that threshold, which turned the NPU from an optional accelerator into a checkbox that decides whether a laptop qualifies for the software at all. Every major Windows supplier now ships parts built to clear it. Qualcomm rates the Hexagon NPU in its Snapdragon X Elite at 45 TOPS; Intel's Core Ultra line reached a third generation, Panther Lake, in commercial machines from March 2026; AMD's Ryzen AI parts are on the same list. Apple is not subject to a Windows badge, but it placed this bet earlier and harder than anyone, which is why its hardware is now the high-bandwidth option for running models locally. Across both camps the presence of a capable NPU is no longer a differentiator. Its absence is.

A CPU and GPU could already run these tasks. The NPU exists because running them all the time, on battery, without the fans spinning up, is a different problem. That is the bet: that ambient, always-available AI becomes normal.

There is also a hard economic reason, not just a privacy one. Pushing inference onto the device keeps data local and reduces dependence on a network connection — but it also shifts the running cost off the vendor's servers and onto hardware the customer already paid for. Every query answered on the NPU is a query that did not hit a data center. For both privacy and economics, on-device inference is attractive, and the NPU is what makes it practical.

Reading the TOPS number#

Because the NPU is now a number on a spec sheet, that number — TOPS, trillions of operations per second — has become a marketing figure, and it deserves suspicion. TOPS is a ceiling: the most low-precision operations the block can theoretically issue per second, under ideal conditions, at a precision the vendor picks. It says nothing about whether real software can keep the block fed.

Two things usually decide real-world usefulness more than the headline number:

• Memory bandwidth. Running a language model on-device is often limited not by compute but by how fast the chip can pull the model's weights out of memory. A high TOPS rating on a bandwidth-starved system is a fast engine with a thin fuel line. This is part of why designs with wide, unified memory can outperform parts that quote bigger numbers.
• Software support. An NPU only helps if applications and their frameworks actually target it, and that support has been uneven across vendors and operating systems. A capable block that no app addresses is dead silicon.

The direction of travel is telling. With its latest silicon generation, Apple stopped quoting a Neural Engine TOPS figure at all and began routing much of its on-device AI through neural accelerators built into the GPU cores. Whether the work runs on a dedicated NPU or on GPU cores tuned for the same math, the clean three-way split blurs at the edges. The label matters less than where the operations actually land — and how fast the memory can feed them.

What this signals#

The quiet standardization of the NPU is a clearer signal than any single product launch. Hardware leads software here. Companies do not add expensive silicon blocks speculatively unless they are confident the use cases are coming. When an accelerator goes from optional to assumed — and when a platform owner gates its flagship features behind a minimum spec — it means the baseline has moved. Local AI is being treated as part of the standard experience, not a premium add-on.

For anyone reading chip specs, the practical takeaways are simple:

1. Expect every new mainstream processor to ship with an NPU. Its absence is now the notable thing. Anchor on Microsoft's 40 TOPS, because it is the one figure here with a published threshold behind it; everything quoted above it is a number each vendor chose how to measure.
2. Treat the TOPS figure as a ceiling, not a promise. Memory bandwidth and software support decide how much of it you will ever actually feel.
3. Watch whether features really move onto the block. The hardware is ahead of the software, and the gap between them is where the real story will play out over the next few years.

The accelerator arrived before most of the software that needs it. That is unusual, and it is the most interesting part. The industry built the road first and is now waiting for the traffic.