Start With the Workload, Not the Platform 

Before deciding where an AI workload should run, organisations need to understand how it will behave. The term “AI workload” can encompass several fundamentally different patterns, each placing different demands on the underlying infrastructure. 

Experimentation and prototyping typically prioritise speed and flexibility. Teams need rapid access to tools and compute without making significant infrastructure commitments before they understand whether an idea will progress. Public cloud and managed services can support this uncertainty, allowing capacity to expand or disappear alongside demand. However, early convenience should not become a permanent architecture by default. Decisions made during experimentation can create dependencies that are difficult or expensive to unwind later. 

Training and fine-tuning present a different challenge. These workloads can require significant compute and access to large datasets, but they are often temporary or can be scheduled. Immediate proximity to the end user may matter less than accelerator availability, data-transfer requirements, power costs, and the ability to secure capacity when it is needed. Depending on these constraints, training might take place in a public cloud, a specialised cloud environment, or dedicated infrastructure located closer to the data or available power. 

Inference introduces another distinction. Batch inference, where data can be processed according to a schedule, gives organisations more freedom to optimise for throughput and cost. Real-time inference has far less flexibility. At Tech Show London, one discussion explored the infrastructure needed to analyse live 4K video, where even relatively small network delays could prevent insights from reaching users when required. In situations such as manufacturing, transport, or live events, running compute closer to the point of use may be essential rather than merely preferable. 

Sensitivity and regulation cut across all these categories. Training data, model outputs, and live inference can each contain information subject to data residency, jurisdictional or access-control. The question to answer is whether a workload is “regulated”, as well as which parts of the system must remain within a controlled environment, and which can safely operate elsewhere. 

A single AI system may therefore span several locations. A model could be trained using temporary cloud capacity, connected to data held in a private environment, and deployed for inference at the edge. Its placement may also change as demand becomes more predictable. Organisations should classify each component by its operational characteristics before deciding which platform should host it. 

Place Workloads by Constraint, Not Ideology 

Once a workload has been understood, placement should become an exercise in resolving its constraints rather than defending an organisation-wide technology policy. Cloud-first can be as limiting as an automatic preference for private infrastructure if it prevents teams from considering what the workload actually requires. 

Data provides one of the clearest starting points. Organisations need to understand where relevant data resides, how frequently it must move, and whether regulatory or commercial restrictions limit where it can be processed. Transferring large datasets between environments can introduce cost, delay, and security risk. In some cases, moving the compute closer to the data will be more practical than moving the data to a preferred platform. 

Latency and availability create a different set of boundaries. A workload that processes information overnight has far more geographical freedom than one making real-time decisions inside a factory, vehicle, or customer-facing service. Where an immediate response is required, network delay or loss of connectivity can turn a technically capable cloud service into an operational weakness. This is where regional or edge infrastructure may become necessary. 

Demand also shapes the decision. Uncertain or short-lived workloads benefit from infrastructure that can scale without leaving expensive capacity unused. As usage becomes more consistent, reserved, or dedicated capacity may provide greater cost predictability. Access matters alongside price. An environment that appears cheaper offers little advantage if the required accelerators, storage, or network capacity cannot be secured when needed. 

The financial assessment must extend beyond the headline price of compute. Data-transfer charges, storage, security tooling, specialist staff, unused capacity, and resilience all affect the true cost of running a workload. Organisations must also account for the cost of leaving. A workload may be technically portable but economically trapped if moving it requires extensive rearchitecture, data migration, or retraining of the teams responsible for it. 

Operational capability is therefore just as important as technical suitability. A genuinely hybrid environment requires consistent governance, security, and observability across its different platforms. Every additional location introduces another set of dependencies to manage. The theoretically ideal platform may still be the wrong choice if the organisation cannot operate it securely and reliably. 

Power availability, energy pricing, and regional connectivity are also beginning to influence where compute can be placed, particularly for workloads that are less sensitive to latency. These factors will not outweigh every other requirement, but they increasingly form part of the same decision. 

Workload placement is ultimately a trade-off between performance, control, flexibility, and cost. The right environment is the one that satisfies the workload’s most important constraints without creating unnecessary long-term risk. 

Placement Decisions Should Not Be Permanent 

The right environment for a workload today may not remain the right environment as its requirements change. An experimental system might begin in public cloud because demand is uncertain and rapid access matters. Once it enters production and utilisation becomes more predictable, reserved or dedicated capacity may offer greater control and cost certainty. If the system later needs to support real-time decisions, part of it may need to move closer to the point of use. 

Different components can also follow different paths. Training, data storage, and inference do not need to take place within the same environment. An organisation might train a model using temporary cloud capacity, retain sensitive data within a private environment, and deploy inference at the edge. The aim should not be to find one permanent home for the entire system, but to place each component according to its changing requirements. 

This does not mean every organisation needs to build a complex multi-cloud architecture. Portability has a cost of its own, and designing for every possible future environment can create more complexity than it removes. What matters is having a realistic exit path. Teams should understand the technical changes, data migration, contractual commitments, and new skills that would be required if the workload needed to move. 

A workload-placement policy can make these decisions more consistent. For every significant workload, organisations should be able to answer four questions: 

• What are its non-negotiable constraints? 

• Why was the current environment selected? 

• What changes would trigger a review? 

• What would it cost to move elsewhere? 

Reviews may be prompted by rising costs, changing demand, new regulation, capacity limitations, or repeated performance and reliability problems. Without these triggers, a temporary decision made during experimentation can quietly become a long-term dependency. 

The question is not as simple as: Where should an AI workload run? Other questions remain, such as: why is it there, what could make that decision change, and how would it respond when it does?