On-Device AI Inference in React Native

Part 1 : On-Device AI Inference in React Native We open with the full runtime landscape: why on-device inference is compelling (privacy by default, free compute, sub-second latency), and why getting there is harder than it looks. We walk through the platform runtimes ; MLX and Core ML on iOS targeting the Apple Neural Engine, and the fragmented Android story where Qualcomm, MediaTek, and Samsung each ship different NPU hardware. We cover how ExecuTorch (Meta) solves the fragmentation problem with a C++ runtime that dispatches to whatever acceleration is available, how to find and quantize models on Hugging Face, and how GPU compute fits into the picture via PyTorch. By the end you'll have a clear mental model of the full stack ,from silicon to JavaScript. Part 2 : Harness Engineering: Making Small Models Dependable Getting the model running is only half the story. The second talk focuses on the systems and tooling around the model that turn its spiky, inconsistent output into something you can actually ship. When the model is small and the context window is tiny, the harness matters more than the model itself. We cover the levers that matter most: feeding the model the right context instead of the most context (write, select, compress, isolate); a self-verification loop that forces the model to check its own work before finishing; loop detection to break the doom loops small models fall into; and managing a reasoning budget when every token costs real battery and latency. Throughout, we show how to measure that the harness actually helped , because on-device, the model doesn't get smarter. The harness does.