How are AI data centres cooled (and why have traditional cooling designs reached their limit?)

Key Insights:

• Traditional cooling systems are struggling to keep up with the increasing energy    demands of modern AI, making the introduction of new approaches vital.
• Liquid cooling has fast become the preferred solution for high-density AI, providing the efficiency and performance needed to maintain operational efficiency.
• There is no one cooling solution that suits all AI set ups. The right choice depends on rack quantity, pre-existing infrastructure, available budget, and plans for expansion.
• Successful AI infrastructure development requires a phased migration strategy that supports future growth while minimising the risk to operations and investments.

AI’s increasing workload is transforming how data centres are designed

High-performance AI workloads require a significant amount of power to function. As adoption accelerates across a range of sectors, the infrastructure that supports it is being stretched beyond existing capabilities, fundamentally shifting the way data centres have to be developed and managed.

There are solutions. Data centre cooling technologies once reserved for specialist environments are now readily available, and implementing these upgrades is a non-negotiable for data centres looking to future-proof operations.

In this guide, we’ll explore why traditional cooling designs are no longer viable for AI workloads, examine the best cooling practices for data centres, and explain how RED Engineering can help organisations confidently implement these upgrades.

Why are traditional data centre cooling methods not effective for AI workloads?

Traditional data centre cooling methods are increasingly unable to meet the demands of modern AI workloads.

AI clusters use densely packed GPU servers or accelerators that consume more power per node than standard CPU-based servers do. This consolidates the heat into a small concentrated space, placing greater demand - and importance - on the data centre’s cooling infrastructure.

This is where MEP engineering comes into play. If demand outgrows the existing cooling systems, equipment will simply throttle to protect itself, cutting processing speeds and even potentially causing equipment failure.

To retain a competitive edge in the AI industry, updates are a must, with cooling systems, power distribution, and mechanical systems operating as one integrated whole to support the larger energy demands.

Effective data centre cooling options

There are a range of effective cooling technologies available for data centres, each built to suit varying performance, efficiency, and operational requirements. The best practices for data centre cooling include:

Enhanced air cooling

Enhanced air cooling is achieved with cooler units placed in-between server racks. This is a great option for mixed environments, where greater flexibility is needed to host standard enterprise workloads alongside AI.

Requiring similar equipment to existing cooling methods, it also tends to be the most straightforward choice for legacy facilities looking to retrofit.

There are a few key details to consider. Ensuring aisle temperatures remain within the necessary ranges can be difficult to achieve without overcooling the whole room, increasing overall energy requirements.

Plus, with air-based cooling struggling to reach fine-grained hot spots, there is also a risk of local overheating on tightly packed GPU boards and memory modules. A robust monitoring system would be required.

Rear-door heat exchangers (RDHx)

A rear-door heat exchanger (RDHx) removes the majority of the heat before it even enters the room, contributing to a more comfortable working environment and reducing the strain on the centre's primary cooling system.

Able to integrate into existing chilled water loops with only minor upgrades, RDHx is also one of the more retro-fit friendly data centre cooling options.

It’s important to remember that, with water connections placed directly at the rack, RDHx requires robust leak monitoring and strong water-quality management. It also still relies on internal server fans to move air across components, meaning fan power and airflow tuning remain part of data centre cooling efficiency considerations.

Direct-to-chip liquid cooling

Direct-to-chip liquid cooling (DLC) is an efficient solution for handling high-performance AI systems, circulating coolant through cold plates to remove the heat at its source. As a lot of the heat is captured in liquid at elevated temperatures, it is able to run higher coolant setpoints, improving overall chiller efficiency.

The implementation of DLC does require introducing a facility-side liquid distribution network, as well as careful separation of facility water and IT equipment. Retrofitting can be a challenge, with feasibility depending on whether power and structural capacity can support densified AI islands without causing disruption to existing setups.

Immersion cooling

Immersion cooling submerges the servers in a non-conductive liquid that removes heat either by circulating the warmed fluid through a heat exchanger or by using a boiling and condensation cycle. The captured heat is then available for reuse or export.

Immersion provides consistent cooling across the various components, and is able to support extremely high densities within a compact space, perfect for facilitating AI clusters that would be impractical - or even impossible - to cool with just air alone.

Immersion can also considerably decrease water usage and, as the heat is captured in the liquid rather than lost to exhaust air, it is perfect for waste heat recovery schemes.

Feasibility can vary when it comes to implementing immersion cooling. It usually requires specialised hardware and service procedures, as well as new training for the centre operations teams. The coolant itself is a cost to consider, and operators will need to be taught safe handling and end‑of‑life management.

What are the wider implications of upgrading cooling systems?

Installing the best data centre cooling practices is important, but it’s only a small part of a wider picture; sustainability, efficiency and practicality should also be taken into account.

PUE and energy efficiency considerations

Upgrading cooling systems can significantly improve PUE and overall energy efficiency in data centres, reducing operational costs and supporting sustainability goals. Immersion-cooled facilities, for example, can in some documented studies reduce overall cooling energy by around 50%.

Structural capacity for new technologies

Installing new data centre cooling technologies is only part of the equation, with many legacy buildings lacking the structural capacity to accommodate additional equipment. Thus, upgrades may be required for the building itself. Immersion tanks and liquid-filled DLC racks, for example, require much higher floor loading than existing raised floors were designed for.

Improving sustainability

As data centre cooling methods evolve, organisations are increasingly placing greater importance on sustainability goals.

Traditional air-cooled data centres often rely heavily on evaporative cooling or water‑intensive chillers, which results in increased local water intake.

In contrast, liquid cooling, particularly with closed loops and dry coolers, can reduce water usage, supporting environmental commitments alongside improving overall data centre cooling efficiency.

How to approach retrofitting data centre cooling technologies?

Many data centres today were not built to contend with modern AI workloads - but that doesn’t necessarily mean a full rebuild is required.

With RED Engineering, each retrofit starts with a detailed engineering assessment of the existing facility, identifying both the opportunities and limitations of your facility.

• Key considerations will include:
• Available chiller and cooling plant capacity
• Existing chilled water distribution and pipe work routes
• Structural limitations
• Water quality management requirements
• Operational capability
• Acceptable levels of disruption during installation

Contact us today to find out how your data centre can be prepared for the future.