What Really Happens Inside an AI Data Centre

People often try to compare intelligence artificielle to a human brain. That’s understandable, because we tend to make sense of new things by comparing them to what we already know. But while human intelligence is found all in one place – a brain roughly the size of a big grapefruit with the consistency of thick custard – much of modern artificial intelligence is scattered across the globe inside vast buildings filled with specialised computing gear. These buildings are called data centres.

Data centres have been around for years and are essentially just buildings where most of the information that powers the internet and cloud computing is stored and processed. Emails, photos, apps, social media posts, streaming services and online games all rely on servers connected by wires, all housed in what are just big warehouses. When you open a website or stream a film, your device sends a request to one of these servers. The data centre processes that request and sends the information back almost instantly.

But the rise of artificial intelligence (AI) has led to a new type of facility called AI data centres. And while they may look similar on the outside, the technology inside is very different.

Think of an ordinary data centre like a library where information is held until you ask for it. When you ask for a book, the librarian gets it for you. Same with a data centre. When you ask for something like loading a website or opening an app, the system retrieves the data and gives it to you.

But an AI data centre is more like a classroom for AI than a library, because instead of just giving you something that’s stored, it uses data to train AI to produce something entirely new. That might be an answer to a question, a computer-generated image or a video showing your favourite actor playing football with Spider-Man, a polar bear and the Archbishop of Canterbury.

Think of it this way: a normal data centre can store a virtual book that already exists, but an AI data centre can write you an entirely new book, and that requires far more computational power.

Because they perform different tasks, the hardware inside these data centres is different too. Traditional data centres are largely built around chips called central processing units, or CPUs. These are the ordinary processors that power most computers. A CPU works a bit like an air-traffic controller, coordinating different computing tasks and responding to requests. When you click on a link, load a web page or open an app, CPUs help retrieve and organise the information needed to make that happen. Depending on the size of the data centre, there could be hundreds of thousands – or even millions – of CPUs running across racks of servers.

AI data centres rely heavily on a different kind of chip called a graphics processing unit, or GPU. These were originally designed to handle the complex calculations needed to render graphics in video games, but scientists realised they were also ideal for artificial intelligence as well.

The key difference is how they process information. CPUs are designed to handle a small number of tasks very quickly, but GPUs contain thousands of smaller processing ‘cores’ that can perform a whole load of calculations all at the same time. This kind of ‘parallel processing’ – where calculations are all done at the same time rather than one after the other – is what AI systems need, because training an AI model often involves analysing enormous datasets and performing billions, even trillions, of mathematical operations.

Some AI facilities also use specialised chips called tensor processing units (TPUs), developed by companies like Google, specifically to speed up machine-learning workloads.

Because they’re built around different technology, AI data centres can look a lot different inside to the ordinary data centres that have been around for decades. A traditional data centre often resembles something out of the original 1960s Star Trek, with banks of blinking lights stretching down long corridors. Servers are usually arranged in rows of racks, with aisles in between so technicians can move around and maintain the equipment.

AI data centres are designed differently. AI systems work best when thousands of GPUs can talk to each other very quickly. To make this possible, the hardware is organised into tightly packed ‘clusters’ connected by ultra-fast networking cables.

So instead of evenly spaced rows, there are these dense ‘islands’ of computing racks filled with GPUs linked together by thick bundles of high-speed cables. These racks can also be much heavier than standard server cabinets, which means AI facilities sometimes require reinforced floors to support the extra weight. It may not look elegant, but it is highly effective.

Keeping Things Cool Yet Powered

All of the AI computing power comes at a cost, and that’s the huge amount of electricity needed to make it all work. Finding that electrical power, then keeping everything cool when that power runs through it, is one of the biggest challenges facing these vast facilities.

Large AI data centres can require hundreds of megawatts of power, as much as a small city might need to run. According to the International Energy Agency, electricity demand from data centres is rising quickly, with artificial intelligence expected to be one of the reasons for growth.

Environmental campaigners have increasingly raised worries about the energy impact of this expansion. Many are pushing technology companies to invest more in renewable energy projects to help power them and minimise the use of fossil fuels.

Anyone who’s left a laptop running for too long knows that electricity generates heat. AI data centres face the same problem on a far larger scale. When thousands of powerful chips are working at full capacity, temperatures inside the servers can rise rapidly. If that heat isn’t removed, the hardware could slow down or even stop working altogether.

Traditional data centres often rely on powerful fans to circulate chilled air through the server racks. But because AI hardware is so tightly packed together, many newer facilities are turning to liquid cooling.

In these systems, coolant flows through pipes or metal plates that sit directly next to the chips, carrying heat away far more efficiently than air. Some experimental designs even submerge entire servers in special cooling liquids.

The Rise of AI

1956
Researchers formally introduced the term ‘artificial intelligence’ at the Dartmouth Summer Research Project.

1965
Intel’s Gordon Moore predicted the number of transistors on a microchip would double every year.

1997
IBM’s Deep Blue defeated world chess champion Garry Kasparov, demonstrating the power of AI.

2006
Researchers led by Geoffrey Hinton revitalised the field of neural networks by introducing deep belief networks.

2012
The AlexNet neural network dramatically improved image recognition accuracy, showcasing GPU-powered AI training in modern data centres.

2016
DeepMind’s AlphaGo defeated Go champion Lee Sedol, relying on massive computing infrastructure and advanced neural networks.

2020
OpenAI introduced GPT-3, a large language model requiring enormous datasets and powerful AI-focused data centre infrastructure.

2022
Generative AI entered the mainstream as ChatGPT attracted millions of users, increasing demand for specialised AI computing clusters.

2023
Technology companies rapidly expanded AI-optimised data centres using GPUs, specialised chips and advanced cooling systems.

2025
Global investment continued as demand sped up. Firms deployed dedicated AI data centres with liquid cooling and specialised chips.