Apple stood on a stage at WWDC two weeks ago and announced it as Apple Intelligence. Six days ago, the first Copilot+ laptops went on sale, and every single one of them runs a Qualcomm chip. Strip the marketing off both moments, and you land on the same thing: on-device AI, arriving everywhere at once. For Qualcomm, this isn’t a 2024 pivot. It’s the payoff of a bet placed back in 2018, when the Hexagon DSP in the Snapdragon 855 picked up its first dedicated tensor accelerator. What I want to dig into is how Qualcomm’s on-device AI quietly became a whole-portfolio strategy, phones and laptops running the same playbook, and why the TOPS number everyone keeps quoting is the least interesting part of the story.
Start with the phone, because that’s where the architecture got proven. The Snapdragon 8 Gen 3 was the first mobile platform Qualcomm built explicitly around generative AI, and the headline claim that made me sit up last October was the one about up to 10 billion parameters running entirely on the device. Not a 1B toy, not a cloud round trip dressed up as local. Ten billion. The demos ran a 7B Llama 2 at around 20 tokens per second, roughly the speed at which text generation stops feeling like a progress bar and starts feeling like a reply.
The way it pulls that off is the part worth slowing down on. There’s no single magic AI core. There’s a Hexagon NPU that Qualcomm rebuilt for transformers, sitting inside an AI Engine that hands work between the NPU, the Adreno GPU, and the Kryo CPU depending on what the job needs. And yes, Kryo, not Oryon. The custom Nuvia cores everyone got excited about live in the laptop chip; the 8 Gen 3 runs a conventional Arm cluster, a single Cortex-X4 prime core at 3.3GHz with A720 and A520 cores stacked underneath. The interesting silicon is the NPU, where Qualcomm claims 98% faster AI and 40% better performance per watt over the 8 Gen 2, with INT4 hardware acceleration that halves a model’s memory footprint compared to FP16. That last detail is what actually lets a 10B model fit inside a phone’s memory budget, and it’s the one nobody bothers to put on a slide.
Qualcomm wasn’t alone in making this play. MediaTek’s Dimensity 9300 landed within weeks, running the same Llama 2 demos on its APU 790, and it did it with an all-big-core CPU that drops efficiency cores entirely, a different bet altogether on what a flagship should prioritize. Apple’s A17 Pro, meanwhile, carries a 16-core Neural Engine rated at 35 TOPS, which on paper looks like it laps the Android field. It doesn’t, and the reason is the whole argument of this piece. Apple barely mentions TOPS. Apple Intelligence, the thing it just announced, runs on that Neural Engine with a Private Cloud Compute fallback, and you’re never meant to know which one answered. Qualcomm sells you the horsepower. Apple sells you the feature and hides the engine. I keep flip-flopping on which framing is smarter.
Then the strategy jumped platforms, and that’s when it got loud. The Snapdragon X Elite is the same on-device-AI thesis rebuilt for Windows, and here Qualcomm finally gets to deploy the Oryon cores: twelve of them, Nuvia-designed, the thing Cristiano Amon had been promising since the acquisition closed. Its NPU is rated at 45 TOPS, and that figure matters for one very specific reason. Microsoft drew a hardware line for its Copilot+ tier: 40 TOPS of NPU, 16GB of RAM, a 256GB SSD, Windows 11 24H2. Clear it, and you unlock the Copilot+ feature set, Recall, live captions with translation, Studio Effects, all running on the NPU instead of the cloud. Miss it, and you don’t get the badge. In June 2024, exactly one silicon vendor cleared that bar in a laptop you could actually buy.
The gap underneath that number is the part that surprised me.
| Chip | NPU TOPS | Clears Copilot+ 40-TOPS bar? |
|---|---|---|
| Snapdragon X Elite | 45 | Yes |
| Apple M4 | 38 | macOS, not applicable |
| AMD Ryzen 8040 “Hawk Point” | 16 | No |
| Intel Core Ultra “Meteor Lake” | 10 | No |
Intel’s shipping NPU that summer did 10 TOPS, a quarter of what Microsoft asked for. AMD’s Hawk Point managed 16. Both companies showed their real answers at Computex three weeks ago, Lunar Lake and Strix Point, each promising to clear 40 and then some, but demoing a chip and shipping one are different verbs, and neither sat on a shelf the day Copilot+ went live. So Qualcomm got a window where, if you wanted the full Windows AI feature set, you had to buy Arm. Full stop. How long that window stays open is the real question, and I don’t think the answer is very long.
Here’s what I wish more of the launch coverage had said plainly. The 45 TOPS figure tells you almost nothing about whether the X Elite runs a large language model at a speed you’d tolerate. The token-by-token generation you actually feel is bound by memory bandwidth, not raw compute, and the X Elite’s 136GB/s of LPDDR5x is what sets its real ceiling, somewhere near 30 tokens per second on an 8B model at 4-bit. Pile on more TOPS, and that number barely moves unless the memory bus widens with it. Where the NPU actually shines is the small, sustained, low-power work that runs all day without you noticing, recall indexing your screen history in the background or Studio Effects keeping your webcam usable, with the occasional quick prompt handed to Microsoft’s little Phi Silica model. It is not how you run a 70B model, and anyone who bought a Copilot+ machine expecting a local ChatGPT got set up for disappointment by the spec sheet itself. I made the on-device-versus-cloud case earlier this month, and it holds here, too.
The piece that never fits on a spec sheet is the software, and I think that’s where the actual lead sits. A fast NPU gets you nothing without developers writing apps that touch it. Qualcomm AI Hub, live since MWC in February, is a library of pre-optimized models you drop onto a Snapdragon target without hand-tuning the quantization yourself, and AIMET handles the shrinking work that turns a model trained in a datacenter into something a phone can hold. None of it is glamorous, and all of it is why the 10B claim is more than a benchmark stunt. You can’t just throw silicon at this problem, which is roughly the mistake Intel made: great fab, late toolchain.
I’m not going to litigate the x86 emulation question here, the will-my-apps-actually-run anxiety that has sunk every previous Windows-on-Arm attempt, because that’s its own post and the independent benchmarks are still landing. I’ll just flag it as the single biggest thing standing between Qualcomm’s NPU lead and Qualcomm actually winning in the PC market. A 45 TOPS NPU does nothing for you if the emulation layer is choking on the one legacy app your job depends on.
So, where does the bet stand? Qualcomm spent six years turning the NPU from a camera-assist block into the thing the whole industry now builds its roadmaps around, and for one strange month in mid-2024, it held both ends of the market at once: the only Windows chip Microsoft will bless, and a phone platform running models its laptop rivals can’t match. That’s an impressive place to be standing. It’s also the most exposed, because Apple is about to push Apple Intelligence onto hundreds of millions of phones this fall without ever quoting a TOPS figure, and Intel and AMD will have 40-plus-TOPS parts shipping before the year is out. The hardware lead is real enough; whether Qualcomm can hold it through 12 months of everyone catching up is what I’ll be watching, and I wouldn’t place the same bet on the phone as on the PC. Ask me again at CES.