Edge Computing Layer

Not every reading needs to reach the cloud. The edge computing layer decides what gets processed close to where it is generated and what actually needs to travel further, and getting that split wrong is one of the most expensive mistakes in an IoT architecture: too much sent to the cloud drives up bandwidth and latency, too little processed locally leaves a device unable to function the moment its connection drops.

What the edge computing layer actually does

Edge computing means processing data on or near the device that generates it, rather than sending everything to a centralised cloud first. In practice this ranges from a microcontroller filtering out noise before transmission, through a local gateway aggregating readings from several sensors, to a dedicated edge server running inference models on video or vibration data in real time.

The point is not to replace the cloud. It is to filter, aggregate and act on data locally, and send only the results or the exceptions onward, cutting unnecessary data transfer and reducing the delay between something happening and something being done about it (IoT For All, 2026).

Diagram of three reasons to process at the edge: latency for speed, bandwidth for cost, and resilience for uptime.

Why process data locally: latency, bandwidth and resilience

Three pressures push processing towards the edge:

Latency. Round-tripping data to a distant cloud region and back takes time that mission-critical applications cannot spare. Edge-native architectures are now delivering millisecond-level decisions for use cases from autonomous vehicles to factory quality control, where a cloud round trip would simply be too slow (IJOER, 2026).

Bandwidth. Sending raw sensor or video data continuously is expensive and often unnecessary. Pre-processing locally and forwarding only relevant readings or anomalies reduces the load on the network and the ongoing data transfer cost.

Resilience. A device that depends entirely on a live cloud connection stops working the moment that connection drops. Local processing lets a device keep sensing, deciding and acting through a network outage, syncing the record once connectivity returns.

Edge AI: what has actually changed

On-device inference is the area that has moved fastest. 2026 is broadly seen as the point where IoT manufacturers move from early pilots to marketing entire product portfolios as edge-AI-enabled, though the majority of the roughly 21 billion IoT devices already deployed still rely on external processing or simple rule-based logic rather than on-device AI (Mender, 2026).

The hardware has caught up with the ambition. High-performance edge system-on-chip designs now commonly deliver 15 to 30-plus TOPS (tera-operations per second) from an integrated neural processing unit, and NPUs typically use 10 to 20 times less power than a GPU for the same inference workload, which is why edge AI has become viable in battery-powered and thermally constrained devices rather than only in mains-powered equipment (Promwad, 2026).

None of this means every device needs an AI accelerator. Simple threshold-based filtering at the edge still solves most bandwidth and latency problems; inference hardware is worth it specifically when the decision itself, not just the data volume, needs to happen locally.

What to check before you commit

  • What genuinely needs to happen locally, versus what is processed at the edge out of habit rather than necessity. Start from the latency and resilience requirement, not the available hardware.
  • What happens during a connectivity outage. Confirm the device keeps operating safely and that data is queued and synced correctly once the connection returns, rather than lost.
  • Update and maintenance path for edge logic. Models and rules deployed to the edge still need a way to be updated in the field; plan this before deployment, not after the first false positive.
  • Power and thermal budget, particularly for anything running continuous on-device inference in a compact or battery-powered enclosure.
  • Total cost of the edge hardware itself, including the NPU or accelerator, against the bandwidth and cloud compute cost it is actually saving.

Related reading

About this page

Written by Mark Searle, founder of IoT Heart and an IoT connectivity professional with more than 20 years’ experience across network engineering, solution architecture and commercial connected services. This page is based on publicly available industry data and standards documentation, current as of August 2026; edge hardware and AI accelerator markets move quickly, so figures and component availability should be checked against current sources before a design decision.

Want the practical version of updates like this delivered twice a month? Join the IoT Heart Briefing.

Sources

Scroll to Top