The AI infrastructure buildout of 2023 and 2024 prioritised training workloads. Hyperscalers and neocloud operators assembled large-scale GPU clusters to run sustained, high-intensity training jobs: weeks of continuous maximum-load operation across thousands of tightly coupled GPUs that generated heat at densities the industry had never previously engineered at scale. In response, operators adopted cooling infrastructure built around direct-to-chip liquid cooling at the rack level, high-capacity cooling distribution units, and chilled water loops sized to manage sustained thermal loads approaching one megawatt per rack. That architecture matched the demands of the workloads operators needed those systems to support.

Inference is now becoming the dominant AI workload, and it is not the same problem. Inference workloads are bursty, latency-sensitive, geographically distributed, and operate at rack densities that sit meaningfully below the frontier training tier. The cooling infrastructure that was purpose-built for training is not the wrong answer for inference, but it is frequently the oversized answer, and operators who apply training-optimised cooling assumptions to inference deployments are making design decisions that will compound into financial and operational problems over a five to ten year asset life.

Training and Inference Have Fundamentally Different Thermal Profiles

The thermal profile of an AI training cluster is defined by sustained maximum load. A training run does not pause for traffic fluctuations. It does not respond to user demand patterns. It runs at full GPU utilisation across all nodes simultaneously, for weeks at a stretch, generating heat continuously at the peak rate the hardware is capable of producing. Frontier training clusters running Blackwell NVL72 configurations draw around 120 kilowatts per rack sustained, with GB300 Ultra deployments approaching one megawatt. Cooling infrastructure for training must be sized and operated at that sustained maximum load, because that is what the workload delivers.

Inference operates differently at the physics level. Inference clusters serve real-time user requests, which means their thermal output tracks user traffic patterns rather than a fixed sustained maximum. Traffic spikes at peak hours and drops overnight. A large language model inference cluster serving enterprise customers in a single timezone may operate at 20 to 30 percent of peak thermal load for significant portions of each day. The cooling system serving that cluster must respond to those fluctuations instantly, because even brief thermal disruptions in an inference environment cause latency issues or service failures that are immediately visible to end users. The requirement is not maximum sustained cooling capacity. It is responsive, variable-capacity cooling with tight uptime guarantees.

The Density Gap Between Training and Inference Changes the Architecture

Training clusters at the frontier operate in the 120 kilowatt to one megawatt per rack range. At those densities, direct-to-chip liquid cooling is not an option but a requirement: air cooling’s physics fail to maintain safe GPU inlet temperatures above 35 kilowatts per rack, and, moreover, the energy cost differential between air and liquid at these densities makes the capital investment in liquid cooling infrastructure straightforward to justify. As a result, frontier training workloads dictate the cooling architecture through direct-to-chip liquid loops, cooling distribution units, and chilled water infrastructure engineered to support sustained peak load.

Inference workloads operate at a different density tier. Production inference clusters commonly run at 30 to 150 kilowatts per rack depending on the hardware generation and deployment configuration. That range spans three distinct cooling regimes: the lower end remains within the range where rear-door heat exchangers extend air cooling viability, the middle supports direct-to-chip liquid cooling at moderate capacity, and the upper end of next-generation inference deployments approaches training-tier densities that require a full liquid cooling stack. Applying a single cooling architecture to that entire range, as operators who build inference capacity by replicating their training facility design tend to do, results in either overcooling at lower densities or undercooling as inference hardware generations advance.

Inference Geography Creates Cooling Constraints That Training Does Not Face

Training clusters are sited for power access. Remote locations with cheap, reliable grid connections and few neighbours to compete for electrical infrastructure are commercially optimal for training. The operator building a frontier training campus can choose a greenfield site in a low-cost power market, design the facility from the ground up for the cooling architecture the workload requires, and optimise every element of the thermal management system without legacy infrastructure constraints.

Inference does not have that siting freedom. Inference must be close to users. Latency requirements below 100 milliseconds for production inference systems force operators to distribute inference capacity across major metropolitan markets instead of concentrating it in remote locations with abundant power. Operators frequently deploy inference capacity into colocation facilities, urban edge sites, and metro data centers whose original designs did not account for liquid cooling. They carry legacy air-cooled infrastructure, structural constraints that complicate the installation of cooling distribution units and chilled water runs, and floor loading limitations that affect heavy cooling equipment placement. The liquid cooling vendor market fragmenting and operators struggling to choose creates an additional constraint: operators retrofitting urban facilities for inference cooling are making vendor selections in a market without common standards, where equipment from different suppliers frequently cannot interoperate without custom integration work.

The Cooling Decision That Will Define Inference Economics for a Decade

The operators building inference capacity in 2026 are making cooling architecture decisions that will determine their unit economics for the next five to ten years. A facility designed with the right cooling infrastructure for inference density and traffic patterns will maintain competitive PUE, avoid thermal throttling at peak load, and carry capital costs appropriate to the actual workload.A facility whose designers copy the training cooling playbook will operate with excess cooling capacity at typical load, carry excess capital cost because designers sized the cooling infrastructure for thermal peaks the facility rarely reaches, and face the same retrofit economics as today’s air-cooled legacy facilities when future inference hardware generations exceed the density limits built into the original design.

The cooling architecture conversation the industry is having in 2026 is predominantly about training: megawatt-per-rack densities, direct-to-chip mandates, immersion versus direct-to-chip at the frontier. That conversation is necessary and correct for the facilities it addresses. What is missing is the equivalent conversation for inference, which will account for the majority of AI compute deployments by volume within the next few years. Inference cooling is not a simpler version of training cooling. It is a different engineering problem with different density requirements, different uptime characteristics, different geographic constraints, and different economic optimisation targets. The operators who understand that distinction before they break ground on their next inference facility will build assets whose economics hold up as the workload matures. The ones treating inference as a lower-power version of training will be managing retrofit debt within three to five years.