What is design engineering and how does it help us rethink racks for AI data centres

Key Insights:

• Rack density has evolved significantly, with a single next-generation rack drawing as much power as a full row of enterprise racks.
• The AI rack is not a container for servers, but rather the server itself, with power, cooling, weight, and interconnect all structurally different.
• Power distribution architecture, cooling systems, and physical layout limit what a facility can ever host, meaning coordinated design engineering is more important than ever.
• Future-facing decision making is key. The facilities that endure are those built around a rack that has long-term adaptability.

A 48U steel frame, a few kilowatts of IT load, and air cooling from a raised floor…

For over two decades, the data centre rack has needed few alterations. You could design data centre infrastructure in 2005, refresh the hardware multiple times, and avoid having to touch the shell even once during that time.

With the advent of AI, this era is now over. Just a single rack of current-generation AI hardware requires more power than an entire row of enterprise servers did ten years ago, and has now become the unit around which an entire data centre is developed.

Get it right, and you’ll have a set-up ready to tackle the next three generations of hardware. Get it wrong, and you risk falling behind in a competitive industry.

In this guide, we’ll examine what is shifting at rack and row level, and why the decisions made here can influence the rest of your data centre infrastructure.

What is rack density in data centre design?

Rack density is the amount of power delivered to - and heat removed from - a single rack of IT equipment. This figure is calculated by adding up the power draw of all components in the rack, from servers to supporting hardware.
Rack densities typically fall into four levels:

• Low to medium (5 to 10kW) - The traditional enterprise standard
• High (20 to 40kW) - Dense virtualisation and HPC - practical ceiling for air cooling
• AI platform (100 to 200kW) - Current GPU rack-scale systems, liquid cooling is mandatory
• Next generation (400kW to 1MW) - Rubin Ultra-era systems and any future developments

Rack density is significant as it identifies what a single position must support. It’s the figure that drives the overall design of data centre infrastructure.

Traditional enterprise racks vs AI racks

If you place the two side-by-side, there are few cosmetic differences; it’s primarily structural. Here’s how it differs:

• Power - AI racks have much higher power requirements and use busbar connections instead of rack-mounted PDUs.
• Cooling - A standard enterprise rack uses air cooling, which works up to 40 kW per rack. Any higher, and the air can’t remove heat, so direct-to-chip liquid cooling is required. This means CDUs are now needed around the rack row.
• Weight - A fully-loaded enterprise tends to weigh around 800kg, whilst a liquid-cooler AI rack falls between 1,500 to 2,000kg due to dense hardware
• Interconnect - AI racks connect all their GPUs together with high-speed links to create one centralised system, in contrast to Enterprise racks, which primarily connect to the external network.

For more information on the difference between enterprise and AI racks at a facility-level, read our guide.

Best practices for data centre rack layout for high density

As rack density increases, there are three primary changes seen within the AI data centre infrastructure:

• Coolant decides layout - With a traditional set-up, there are rows to accommodate the CRAC units and containment aisles. Now, AI data centre infrastructure is configured around the CDU-to-rack ratio and pipe runs.
• Cable management becomes more complex - Power busbar, fibre, and liquid coolant are now routed to the same rack, but each have different requirements. Power and data need to be separate to avoid electromagnetic interference, and strict pathways have to be enforced to avoid cable bulks blocking airflow routes.
• The role of airflow changes - Containment is still important, but it focuses on residual load instead of a full rack.

How infrastructure decisions shape what is possible?

Rack density is shaped by the three key components of data centre infrastructure:

Power distribution

The voltage architecture you decide on establishes what can be possible in the future. With the IEA projecting AI data centre electricity consumption will drastically increase by 2030, traditional 415V AC three-phase distribution will become obsolete far earlier than 600kW per rack.

800V DC architecture is now being standardised primarily because AC is impractical at Rubin Ultra densities. A facility wired for 120kW racks is unable to host 600kW racks without a complete distribution rebuild.

Cooling architecture

Again, the cooling systems you opt for tie you into a pipe network, a CDU strategy, a water plant sized in hundreds of kW per rack position, and a facility water temperature.

Rubin-era systems require 45°C liquid cooling, which means you’ll rely far less on standard mechanical chillers. By adjusting for warm water early on, you’re ensuring future efficiency for your AI data centre infrastructure.

Physical layout

Even the building itself can place limitations, with floor loading, room height, riser capacity, and row geometry all setting limitations on future capacity to handle higher densities.

Kyber-generation racks, in particular, are larger than anything that has been built in the enterprise era. If your structure cannot handle these loads, no amount of innovative mechanical design work can solve the problem.

What is design engineering, and why does it matter here?

If you’re asking ‘what is design engineering?’, it’s when performance requirements are transformed into efficient, operable systems. With data centre infrastructure, that means the mechanical, electrical, and public health design, which determines whether a facility can support the desired density.

With the margin for ill-equipped design growing ever smaller, this is more important than ever. At higher densities, incompatibility between electrical and cooling capacity is wasted capital and failing efficiency.

This whole-systems approach is precisely what RED brings to AI factory and HPC facility design.

Early rack-level choices drive long-term cost and flexibility

Hardware evolutions are growing increasingly regular, with Blackwell to Rubin to Rubin Ultra occurring during a span of roughly 36 months and with a fivefold rise in rack power.

The rack-level choices that offer data centres the greatest flexibility include:

• Oversized risers, pathways, and pipework - This approach takes into account space for future distribution
• Slab-first structural design - Accommodates next-generation rack weights
• A voltage strategy - Includes a viable route to high-voltage DC
• CDU and water plant architecture - Enables cooling capacity to monitor density without requiring a complete replumbing
• Row geometry with sidecar positions - Rack-adjacent power and cooling equipment now becomes a permanent fixture

These are easy ways to transform a potential future rebuild into merely a future upgrade, a small difference that will be felt in decreased expenses and in years of time-to-market.

The rack is now the design brief

With density now the primary force shaping the power and cooling architecture, the rack is now the item at the top of the agenda.

Design your facility around the rack you will require in five years, instead of the one you are deploying currently. The organisations that adhere to this approach will be operating Vera Rubin-era systems in buildings they have already future-proofed.

Contact us today to discover how you can develop racks built for the future.