For a decade the default answer to "where does the model run" was simple: in the cloud. You collected data on a device, shipped it to a data centre, ran inference on a powerful server, and sent the answer back. In 2026 that default is breaking down. A growing and important share of machine learning now runs on the device that produced the data — the phone in your pocket, the earbuds in your ears, the camera on the factory line, the microcontroller inside a thermostat. This is Edge AI, also called on-device intelligence, and this course teaches you how to build and ship it.
Educational note: This course is for learning. Any system you build with real data must respect data-protection law (the GDPR in the EU), device security, and the rights of the people whose data you process. Edge AI has real privacy advantages, but it is not automatically compliant. We treat these obligations as engineering requirements throughout.
What "edge" actually means
The edge is simply the far end of the network, close to where data originates and where actions are taken, as opposed to the cloud, which is centralised compute in a data centre. Edge AI means running the inference — and sometimes even training or adaptation — on hardware at that far end rather than shipping raw data to a server. The device might be a flagship phone with a neural accelerator, a single-board computer like a Raspberry Pi, an industrial gateway, or a microcontroller with a few hundred kilobytes of RAM. The unifying idea is that the intelligence comes to the data, instead of the data travelling to the intelligence.
It helps to see the edge not as one place but as a spectrum of tiers, because the engineering constraints change by orders of magnitude as you descend it:
| Tier | Typical hardware | Typical memory | What realistically runs there |
|---|---|---|---|
| Near edge / on-premises | Industrial gateway, edge server | Tens of GB | Multiple mid-size models, video analytics for many cameras |
| Rich device | Flagship phone, laptop, Jetson-class board | 8–16+ GB | Vision transformers, speech models, small language models |
| Constrained device | Budget phone, wearable, Raspberry Pi | 1–4 GB | Compact CNNs, keyword spotting, compressed classifiers |
| Deep embedded (TinyML) | Microcontroller (Cortex-M class) | KB to a few MB | Tiny quantized models: wake words, anomaly detection, simple vision |
Every technique in this course — quantization, pruning, distillation, runtime selection, accelerator delegation — exists to move a capability one or two tiers further down this table than it could otherwise go. When you read a claim like "this model runs at the edge", your first question should always be: which tier? A model that is trivial on a Jetson-class board may be impossible on a microcontroller, and a design that assumes a flagship NPU may crawl on the budget phones that dominate many markets.