When a manufacturing or industrial team decides to add AI to a process, one of the first architectural decisions is where the AI actually runs: in the cloud, or on hardware at the facility (the "edge"). This decision shapes cost, latency, security posture, and long-term maintenance — and it is often made too quickly, based on whatever the team already happens to know rather than what the application actually needs.
What Cloud AI Is Good At
Running inference in the cloud makes sense when:
- The workload is not time-critical — a report generated overnight or a batch analysis run weekly does not need sub-second response times
- The models are large and need frequent retraining with centralised data from multiple sites
- The facility already has reliable, high-bandwidth internet connectivity
- Centralised monitoring across many locations is a priority
The tradeoff is dependency on network connectivity and, for many industrial customers, discomfort with sending operational data — which can reveal proprietary processes — outside the facility.
What Edge AI Is Good At
Running inference locally, on hardware inside the facility, makes sense when:
- Latency matters — a quality-control camera that needs to reject a defective part before it moves to the next station cannot wait for a round trip to a remote server
- Connectivity is unreliable or the facility is in a low-bandwidth environment
- Data sensitivity is a hard requirement — some industries (defence, pharma, certain manufacturing clients) will not allow production footage or process data to leave the building under any circumstances
- The workload is well-defined and doesn't need constant retraining against a large centralised dataset
The tradeoff is that edge hardware has finite compute, so models need to be optimised (quantised, pruned, or otherwise made efficient) to run within that budget, and updates typically have to be pushed out to each site rather than updated once centrally.
A Simple Way to Decide
Rather than starting from a technology preference, start from three questions:
- How fast does the decision need to come back? Milliseconds → edge. Minutes or longer is acceptable → cloud is viable.
- How sensitive is the data? If it cannot leave the facility under any circumstances, edge is not optional — it is a requirement.
- How often does the model need updating from new data? If it needs frequent retraining against a large, growing dataset, cloud-centralised training with edge deployment of the resulting model is often the practical middle ground.
The Hybrid Reality
In practice, most mature industrial AI deployments end up hybrid: models are trained centrally in the cloud using aggregated data (sometimes anonymised or aggregated specifically so raw sensitive data never leaves the site), and the resulting trained model is then deployed to edge hardware for actual inference at the point of use. This gives the benefits of centralised learning without requiring live production data to leave the facility on an ongoing basis.
Getting this architecture decision right early avoids a costly rebuild later — teams that default to "cloud because it's easier to start with" often find themselves re-engineering for edge deployment once latency or data-sensitivity requirements surface in production.