HP is bolting a discrete Hailo-10H edge AI chip into its Engage retail point-of-sale systems, shipping it as the HP AI Accelerator M.2 Card. A checkout terminal with a dedicated 40 TOPS vision chip is not where most people expected edge AI to land, and companies do not pay BOM premiums and surrender an M.2 slot for a rounding error. Years of edge AI marketing trained me to expect the asterisk: a chip rated at some impressive TOPS number, then fine print revealing it only hits that figure on a benchmark nobody runs in production. This time the money is going behind a discrete accelerator in the exact place the integrated-NPU crowd swore one would never be worth the bill of materials.
Hailo has chased the same idea out of Tel Aviv since 2017: map a neural network onto a fixed dataflow graph and kill the von Neumann bottleneck that makes GPUs and CPUs waste most of their energy shuffling weights in and out of DRAM. The thesis is not original. Graphcore and Cerebras and SambaNova all built businesses on versions of it, at wafer and datacenter scale. What separates Hailo is that they aimed the whole thing at the edge constraint set from day one: milliwatts to low single-digit watts, tiny die, M.2 and PCIe form factors, and the kind of automotive reliability paperwork that takes years to accumulate.
The Hailo-8 earned the credibility: 26 TOPS at roughly 2.5W on TSMC 16nm, AEC-Q100 qualified, sitting in Tier 1 ADAS reference designs. A cost-optimized Hailo-8L offered 13 TOPS and around 1.5W for anyone who wanted the architecture cheaper. The Hailo-10H lands in the 40 TOPS range at a 5W to 8W TDP, on what most analyst coverage pegs as TSMC 7nm, though Hailo has not confirmed that node publicly, so treat it as consensus guess rather than gospel. The bigger story is the target. The Hailo-8 lived in cars and industrial cameras; the 10H pushes into the commercial compute world that Qualcomm, Intel, and AMD have spent two years carving up with integrated NPUs, and HP’s first deployment lands somewhere none of them were aiming, the retail point of sale.
Hailo is selling a discrete accelerator into a world that spent two years convincing itself the NPU belongs on the SoC. The case for discrete comes down to performance density and the fact that you can drop an M.2 card into an existing platform without buying new silicon. The case against is real, and I am not going to pretend otherwise: you add a part and a cost line, and you give up the memory-sharing an SoC vendor gets for free.
Rather than executing matrix-multiply kernels on general-purpose SIMD cores the way a GPU does, Hailo’s Dataflow Architecture compiles the network topology into a static execution schedule. Each layer is assigned to specific processing elements, and intermediate activations flow between layers through an on-chip interconnect instead of bouncing out to external DRAM. Hailo likes to call it a neural network on a chip, and for once the marketing line is structurally accurate.
A GPU throws flexible SIMD cores at the workload and eats the overhead in thread scheduling, kernel-launch latency, and memory bandwidth. Qualcomm’s Hexagon and Cadence’s Tensilica use programmable VLIW vector engines, buying flexibility at a moderate efficiency cost. The systolic-array crowd, Google’s TPU and Apple’s Neural Engine, is brutally efficient for regular GEMM but awkward when the topology is irregular. Hailo sits with Graphcore and SambaNova in the dataflow camp, keeping data on-chip and routing it explicitly.
Inside the chip is an array of processing elements, each carrying a MAC unit that handles INT8 and INT16, local SRAM for weights and activations, and a routing interface into the on-chip network. The 40 TOPS figure is the aggregate MAC throughput at INT8 across that array at clock. The efficiency claim follows directly: because weights live in distributed local SRAM rather than getting fetched from DRAM, effective bandwidth utilization runs far ahead of GPU inference, as long as your network fits in the on-chip memory budget. FP16 support on the 10H specifically is something I would want confirmed against the datasheet before quoting it in a production context.
The Hailo-8 carried roughly 25MB of on-chip SRAM and nothing else, so the on-chip budget was the whole ballgame: a network either fit, giving you near-zero external memory traffic and the full TOPS-per-watt the architecture promises, or it did not, dropping you into tiling strategies that dragged DRAM back into the loop and bled the efficiency out. The 10H changes that math in one important way. It pairs a larger on-chip SRAM with an external LPDDR4 memory interface the Hailo-8 never had, which is precisely what lets it run models too big to sit fully on-chip, at a real efficiency cost, instead of refusing them outright. The on-chip budget is still the design’s center of gravity. It is no longer a hard wall.
MobileNetV3 and the EfficientNet-Lite family fit comfortably, even on Hailo-8-class memory. YOLOv8n and YOLOv8s fit with sensible quantization, and the heavier YOLOv8m and YOLOv8l that needed tiling on the Hailo-8 should land on-chip with the 10H. Whisper at small or medium size fits. Where the 10H breaks from its predecessor is the generative ceiling: Hailo pitched it from launch as an edge gen-AI part, and over that LPDDR4 path it will run a quantized INT4 Llama2-class 7B at a few tokens a second and small diffusion models in a handful of seconds, inside its power envelope. That is real, but it is the chip stretching, not the chip in its element. The on-chip sweet spot is discriminative work: classification, detection, segmentation, pose, small-model speech. Most production edge AI in cars, factories, and stores lives there anyway, which is the half of the workload space HP actually cares about.
On the HP Engage terminals it is all retail vision: real-time object detection on the checkout camera so cashierless and self-checkout work, plus item counting, barcode-free product recognition, and the one that pays for the whole card, shrinkage control, the polite industry word for catching theft at the register. A model like YOLOv11m chewing through 4K camera streams at the lane is exactly the load a weak or absent integrated NPU chokes on, and exactly the kind of frame-by-frame work you want running locally instead of paying a cloud round-trip for every frame. Every missed grab at self-checkout is a line on a shrinkage report, so the inference has to sit at the register, run constantly, and flag the item before it is in the bag.
Hailo’s Dataflow Compiler ingests ONNX as the primary format, with TensorFlow and TFLite coming through conversion and PyTorch arriving via ONNX export. The compiler fuses operator sequences like Conv plus BatchNorm plus ReLU into single PE operations, runs post-training quantization to INT8 with a calibration set, allocates weights and activations to specific SRAM banks for locality, builds the static schedule, and spits out a .hef binary you load at runtime. The runtime is HailoRT, a C/C++ library with Python bindings, and the piece that matters for anyone doing video is the GStreamer plugin plus TAPPAS, Hailo’s pipeline framework. That GStreamer lineage is not an accident, it is why Hailo already has traction in surveillance and industrial vision, and a big reason a retail-vision integrator will find the path familiar.
import hailo_platform as hp
from hailo_platform import HEF, VDevice, HailoStreamInterface, ConfigureParams, InputVStreamParams, OutputVStreamParams, InferVStreams
import numpy as np
# Load HEF (compiled model)
hef = HEF("yolov8s.hef")
# Create virtual device (abstracts physical Hailo device)
params = VDevice.create_params()
with VDevice(params) as vdevice:
configure_params = ConfigureParams.create_from_hef(
hef=hef,
interface=HailoStreamInterface.PCIe
)
network_groups = vdevice.configure(hef, configure_params)
network_group = network_groups[0]
network_group_params = network_group.create_params()
input_vstreams_params = InputVStreamParams.make(network_group)
output_vstreams_params = OutputVStreamParams.make(network_group)
with InferVStreams(network_group,
input_vstreams_params,
output_vstreams_params) as infer_pipeline:
input_data = {
"input_layer_name": np.random.randint(0, 256, (1, 640, 640, 3), dtype=np.uint8)
}
with network_group.activate(network_group_params):
infer_pipeline.infer(input_data)
output = infer_pipeline.get_output()That structure follows the public HailoRT documentation patterns. Check it against whatever API version ships before you trust it in production, because Hailo has been iterating the runtime fairly aggressively.
On the host side the 10H talks over PCIe, either Gen 3 x4 or Gen 4 x4, and in the HP boxes it almost certainly lands as an M.2 card in an M-key slot. People get nervous about the bus, so let me kill that worry with arithmetic. A YOLOv8s pass on a 640×640 RGB frame at INT8 needs about 1.2MB of input per frame. At 30 FPS that is roughly 36 MB/s. PCIe Gen 3 x4 gives you something like 3.9 GB/s. Even four 1080p streams at 30 FPS only ask for about 750 MB/s. The interface is nowhere near the bottleneck.
TOPS-per-watt is trivially gameable as a headline number. Peak TOPS gets measured under ideal dense INT8 GEMM. Real-world throughput depends entirely on your network and memory pattern. And TDP is a worst-case thermal figure with little to do with the power you actually draw mid-inference. With those caveats stapled on, here is the discrete edge accelerator field in the 5W to 10W band.
| Chip | TOPS (INT8) | TDP | TOPS/W | Form Factor | Notes |
|---|---|---|---|---|---|
| Hailo-10H | ~40 | ~5-8W | ~5-8 | M.2/PCIe | Discrete accelerator |
| Intel Movidius Myriad X | 4 | 1-4W | ~1-4 | M.2/USB | Older gen, thin software |
| Coral Edge TPU (M.2) | 4 | 2W | ~2 | M.2 | Limited model support |
| Qualcomm Cloud AI 100 Ultra | 400 | 75W | ~5.3 | PCIe | Different class entirely |
| NVIDIA Jetson Orin NX | 100 | 10-25W | ~4-10 | Module | Full SoC |
The integrated NPUs are a messier comparison because the NPU shares the SoC power budget and you cannot cleanly isolate its draw. For rough context, Qualcomm’s Snapdragon X Elite carries a 45 TOPS Hexagon NPU inside a 23W SoC, Intel’s Lunar Lake hits 48 TOPS, Apple’s M4 Neural Engine lands around 38, AMD’s Ryzen AI 300 claims 50, and Meteor Lake’s 11.5 is the laggard that never cleared the Copilot+ bar. Those are the flagship laptop parts, and they still publish a slide number nobody hits under a sustained 4K stream. The integrated silicon in a POS terminal sits well below even that, which is the whole reason HP reached for a card.
A discrete part adds to total system power, while an integrated NPU spends power the SoC is already burning, so the card only earns its slot when the host silicon cannot do the job. On a POS terminal that bar is low. The CPU’s integrated NPU, when there is one at all, tops out well short of 4K multi-stream object detection. The 40 TOPS card is not duplicating something the box already had, it is adding a capability the box never shipped with.
A discrete accelerator hands the retail-vision software dedicated, uncontested inference, on a GStreamer stack the surveillance and industrial-vision world already speaks, with an M.2 card that drops into a standardized Engage chassis instead of forcing a CPU generation jump across an entire store fleet. Retail hardware lives for five to seven years and gets bought by the thousand, so an upgrade path that does not touch the motherboard is worth real money to the people signing the PO. HP looked at the math and decided the premium was worth it.
Automotive parts live and die by two qualification frameworks, and that paperwork is half of Hailo’s real advantage over a faster die. AEC-Q100 is the component-level reliability standard, covering temperature grades (Grade 0 runs -40°C to +150°C, Grade 2 down to +105°C), ESD, electromigration, hot carrier injection, mechanical stress, accelerated life testing. The Hailo-8 cleared AEC-Q100 Grade 2. The 10H’s grade is something I would confirm against the datasheet before quoting, since it may still be in process.
Then there is ISO 26262, the functional-safety side, with its ASIL levels running A through D. Lane departure warning sits around ASIL B. Autonomous emergency braking and occupant monitoring climb toward ASIL D. For an AI inference chip that means ECC on memories, lockstep on critical paths, diagnostic coverage metrics, defined safe states, and stacks of FMEA documentation. The realistic pathway for the 10H is ASIL B with ASIL D decomposition, which is how this always works in practice: the AI chip handles perception, and a separate ASIL D-rated MCU owns the safety-critical actuation. Nobody is betting a braking decision on a neural net’s raw output.
With qualification plus 40 TOPS, the 10H slots neatly into in-cabin monitoring, where a MobileNetV3 or EfficientNet backbone running at 5 to 10 TOPS handles gaze, drowsiness, distraction, seatbelt and child-presence detection inside a sub-100ms budget. It runs surround-view and parking with YOLOv8s across multiple camera streams under a 200ms window, and manages traffic-sign recognition and lane-marking segmentation without breaking a sweat. What it does not do is Level 3 perception, which needs 200-plus TOPS and multi-chip platforms like NVIDIA’s DRIVE Orin at 254 TOPS. LiDAR point-cloud fusion is out too, it wants a different architecture entirely.
Rebellions out of Seoul is building its REBEL datacenter inference chip on Samsung’s 4nm process, going after transformer workloads with a memory-centric design that rhymes with Hailo’s philosophy but aims at far larger models. Different market entirely, but the same structural tide: dedicated inference silicon peeling specific workloads off general-purpose GPUs. I am not going to get into the datacenter inference race here, because that is a separate article and a separate set of tradeoffs.
The cloud-inference default was always partly a story we told ourselves because the edge silicon was not good enough to challenge it. A sub-8W card that a Fortune 100 OEM will pay to install in a cash register is where that story starts to crack. Perso, I am not convinced discrete wins the laptop, where the pressure to fold everything onto the SoC is relentless and the BOM math is unforgiving. But the register is not the laptop. Retail does not care which die runs the model, it cares whether self-checkout catches the steak sliding into the tote bag, and the integrated silicon in these terminals cannot, so HP bought one that can. Shrinkage gets measured in basis points of revenue, and a 40 TOPS M.2 card is cheap against that number.