AI in Data Centre Design

What is AI Computing and how is it driving the DC industry?

“Artificial Intelligence” (AI) is the field of computer science that aims to create machines and systems that can perform tasks that normally require human intelligence. Although the thought of the possibility of artificial life has long been in the minds of humans, it is only in the 20^th century that engineers and scientists began to make progress in developing a workable technology. The 1950s saw the birth of AI, with Alan Turing publishing “Computer Machinery and Intelligence” which proposed a test of machine intelligence called The Imitation Game. Five years later, in 1956, a computer scientist named John McCarthy held a conference where he first adopted the word “artificial intelligence” and for the first time, it was coined as an academic field of science[1].

Machine learning and neural networks are driving much of the commercial interest in AI today. Progress in these areas in the 20^th century was limited by the comparatively minimal processing power available at the time. The continued growth of computational power and the accumulation of large datasets has played a major role in revitalising progress in computational methods such as neural networks, which enable and in turn motivated significant investment within the field of AI.

2023 saw a boom in the interest around ‘AI’ technologies. This has broadly been driven by the availability of ‘trained’ large language models (LLMs) such as ChatGPT (amongst others), which are developing an increasing level of accuracy. With no signs of slowing down, the global AI market size was valued at USD 150.2 billion in 2023 and is expected to grow at a CAGR of 36.8% from 2023 to 2030[2]. Research undertaken by PwC, published in 2023, estimated that Global GDP could be up to 14% higher in 2030 as a result of AI – the equivalent of an additional $15.7trillion – making it the biggest commercial opportunity in today’s fast changing economy. AI is expected to revolutionise many industries and is set to persist as a powerful technological force for innovation, propelling many advancements for the foreseeable future.

AI systems require specialised computer hardware, and the industry’s high growth rate is pushing the need for this hardware to be faster, more robust and more efficient. Although this technology has been available through High Performance Compute (HPC) systems for many years, the adoption of AI workloads is now bringing this technology into the mainstream. New architectures are constantly being developed, with each new and improved equipment requiring more and more power. Many companies are implementing AI into their operations, with around one in six UK organisations (totalling 432,000) having embraced at least one form of AI technology[3]. When looking globally, there is an increasing number of businesses, about 35%, are using AI, and another 42% are exploring the technology. The development of generative AI, which uses powerful foundational models that train on large amounts of unlabelled data, can be adapted to new use cases and bring flexibility and scalability that is likely to accelerate the adoption of AI significantly due to its ‘time to value’ being up to 70% faster than traditional AI[4].

The applications of AI are still very general, with each application enabling numerous use cases for specific tasks and processes in all industries. Table 1 shows some examples, to give an idea of the wide scope and range of possibilities that AI offers.

[1] Harvard University

[2] MarketsandMarkets

[3] Gov.uk

[4] IBM

The AI market is influenced by many factors that are driving or restraining its growth, with opportunities and challenges that may shape the future evolution of the technology. These factors are constantly and rapidly changing, some examples can be seen in Figure 1.

There is currently a rush on the IT sector while the large cloud operators are fighting to be the first to bring their AI commercial offerings to the market, including: Google Bard, Microsoft Copilot / Bing Chat, AWS Lex & IBM Watson to name a few. The cloud providers, amongst others, are starting to offer a portfolio of products from out-of-the-box enterprise solutions to AIaaS (AI-as-a-Service) computational processing.

The growing need and use of AI technologies is putting data centres under pressure, demanding more and more power. The existing operational data centre infrastructure is not generally designed for high-performance computing (HPC), driving the need for new solutions. AI servers are currently only a small portion of the server market. As the applications of AI models in industry increase, it is expected that AI servers will take up an increasingly larger share of the overall revenue, with a report from DellOro group in January 2023 estimating this could account for almost a third of the server market revenue share by 2027.

This entails that the proportion of AI clusters in data centres compared to standard servers will increase. It is forecasted that future data centres will be a hybrid of new, high-performance computing, alongside the “normal” IT workloads seen today. This is why RED have been developing specific data centre design topologies for enabling HPC technologies for AI and Machine Learning applications.

In our next article in this series we will investigate AI computing in more detail, how this differs from traditional server workloads and to review what this means in terms of changes to the data centre designs of yesterday.

[5] Dell’Oro Group

Co-creators

• Alex Nock, Head of Engineering
• Luke Ortlieb, Graduate Mechanical Engineer