Llama 3, AMD XDNA, and H100 SXM5: Three Bets on Where AI Actually Runs

Three announcements in the last few weeks reveal fundamentally different strategies for who controls AI infrastructure and where the compute actually runs. Meta released Llama 3 two days ago, an open-weight LLM in 8B and 70B parameter versions under a permissive license. AMD shipped Ryzen processors with integrated XDNA AI acceleration, putting NPU hardware into consumer PCs for the first time. And Nvidia’s H100 SXM5 continues to dominate data center deployments while the Grace CPU brings Arm architecture to the server rack. Each represents a different bet: open versus closed, edge versus cloud, Arm versus x86.

I care about this because the infrastructure question determines everything downstream. If all AI workloads stay in hyperscale data centers, companies rent intelligence by the token. If inference moves to edge devices and local servers, there is an entirely different opportunity landscape for hardware makers, developers, and regional infrastructure plays.

Llama 3: Meta’s open-weight distribution play

Meta releasing Llama 3 under a permissive license for research and commercial use is a direct challenge to OpenAI’s closed-API model. The 8B parameter version runs on consumer hardware. The 70B version competes with GPT-4 class models on reasoning and coding benchmarks. Performance is table stakes. The strategic move is the license.

Open-weight models shift bargaining power away from API vendors. Developers can run Llama 3 locally, fine-tune it on domain-specific data, and avoid per-token pricing entirely. For companies building AI products, this lowers the cost structure and eliminates vendor lock-in. Snapdragon Insider bias declared: if Llama 3 runs efficiently on the Snapdragon 8 Gen 3’s Hexagon NPU, you do not need to send every query to the cloud. Qualcomm’s entire on-device AI thesis depends on models like this existing.

The sovereignty angle is also real. European and Asian governments prefer open models they can audit and deploy domestically. Meta’s license terms allow this. OpenAI’s do not. From my FDI seat, I am already hearing European enterprises ask whether they can run Llama 3 on-premise to comply with data residency requirements. The answer is yes, and that changes the competitive calculus for every closed-model provider.

AMD’s XDNA NPU: AI acceleration at the edge

AMD’s integration of the XDNA AI accelerator directly into Ryzen mobile and desktop processors is a bet that AI compute moves to the PC. The NPU handles inference workloads locally without requiring a discrete GPU or a cloud round-trip. Intel has AMX instructions and is pushing the Meteor Lake NPU. Qualcomm has Hexagon in Snapdragon. AMD now has a comparable on-device AI story in the x86 world.

For developers, this changes deployment assumptions. If your target hardware includes XDNA-equipped PCs, you can assume AI acceleration is available and build for it: real-time transcription, local code assistants, on-device image generation, privacy-first workflows that never leave the machine. The argument that cloud-hosted AI models like Gemini 1.5 Pro are the only serious option gets weaker every time a consumer chip ships with an NPU.

Nvidia H100 SXM5 and Grace: doubling down on the data center

Nvidia’s H100 SXM5 is the power-optimized variant of the chip that trained GPT-4, Claude, and Gemini. The Grace CPU, built on Arm architecture, pairs with Hopper GPUs via NVLink-C2C to create a unified CPU-GPU system that shares memory without a PCIe bottleneck. This is Nvidia consolidating its dominance in cloud AI while simultaneously validating Arm for the data center.

The Grace CPU is strategically important because it is Arm-based, not x86. Nvidia is betting that data center economics favor Arm’s power efficiency over x86’s legacy compatibility. Amazon’s Graviton processors already proved this works for general compute. Nvidia is extending the thesis to AI-specific workloads. The Aurora supercomputer at Argonne runs Intel Ponte Vecchio, but the next generation of national lab AI infrastructure may well run on Grace Hopper Superchips.

For enterprise customers, the CUDA ecosystem remains the default for large-scale training and inference. The H100 SXM5’s power efficiency matters because electricity is a growing share of total cost of ownership. Data centers in Illinois and the Midwest, where power is cheaper than coastal markets, become more competitive for AI workloads as per-watt performance improves.

Three bets, one stack

Meta bets on open-weight distribution lowering barriers to AI adoption. AMD bets on edge compute replacing cloud APIs for privacy-sensitive and latency-critical workloads. Nvidia bets on hyperscale data centers staying dominant, with Arm CPUs handling the non-GPU orchestration. All three can be right simultaneously, because Llama 3 runs on Nvidia GPUs in data centers and on AMD CPUs at the edge. The question is which deployment pattern wins the most valuable use cases.

My read: privacy-sensitive and latency-critical workloads move to edge devices, favoring AMD and Qualcomm. Large-scale training and batch inference stay in data centers, favoring Nvidia. Open-weight models enable both, which is why Meta’s licensing strategy is smarter than OpenAI’s API-only approach. And the software-hardware split I wrote about with DeepMind and Intel is playing out in real time: the model layer and the hardware layer are separating, and the companies that own the intersection of both will capture the most value.

For Chicago specifically, the opportunity is in the tooling and services layer that sits on top of these three platforms. Developers need frameworks that abstract across edge and cloud deployment. Enterprises need guidance on when to use local inference versus API calls. Infrastructure providers need hybrid solutions. That is where regional tech companies compete, even if the silicon itself comes from California.