“The most important thing about AI is the underlying infrastructure”
This quote, emphasized by SK Telecom's Yoo Young-young at the SK AI Summit 2024, captures the essence of the current AI industry. ChatGPT and AI image generation are just the tip of the iceberg, and there is a huge infrastructure that supports them. According to Precedence Research, a global market research firm, the global AI market was worth $538.1 billion (approx. KRW 708 trillion) in 2023 and is expected to grow to $2.75 trillion by 2032. That's more than 1.5 times the size of South Korea's 2023 GDP (about KRW 2,200 trillion).
AI infrastructure is like the human body. High-performance computers process complex computations like the brain, data centers store massive amounts of data like muscles, and communication networks deliver information like the nervous system. These three elements must work in perfect harmony for AI to function properly. For example, to produce a single response from one of our favorite AI chatbots, thousands of GPUs work simultaneously, hundreds of terabytes of data are processed, and all of this happens in less than a second.
For telcos in particular, AI infrastructure is a new growth engine. Traditional telecom network and data center operations experience has become an even more valuable asset in the AI era. The nation's densely connected telecommunications network serves as the “digital highway” for AI services. If the general Internet is a city street, AI needs a dedicated highway, a “dedicated line. This infrastructure, owned by telcos, is a key factor in the quality and reliability of AI services.
Moreover, AI infrastructure is no longer just a technology foundation, but a key element of national competitiveness. Just as social overhead capital (SOC) such as roads, ports, and railways were the foundation for national development during the industrialization era, data centers and telecommunications networks will play a role in the AI era. This is why Yoo emphasized that “building a solid infrastructure is essential for South Korea to enter the G3 AI powerhouse.”
Data Center Evolution: From Digital Hotel to AI Powerhouse
The 100 megawatt data center operated by SK Telecom through SK Broadband has been operated in a 'co-location' manner. Co-location is simply a 'digital hotel' service. The carrier provides the building, electricity, air conditioning, and security, and companies rent as much space as they need and set up their servers. Just like a hotel guest books a room and brings their own belongings, companies rent space in a data center to run their servers. This was an economical choice to avoid having to build their own data centers, which can cost tens of billions of dollars.
However, with the advent of the AI era, data centers are now evolving from mere “digital hotels” to “digital power plants”. The biggest change is power consumption. Whereas a rack of servers (the cabinets you plug them into) used to consume 4-7 kW of power, AI servers require at least 15 kW and as much as 30 kVA. This dramatic increase in power demand creates new technical challenges.
Cooling systems need to be completely different. Traditional air cooling cannot keep AI servers cool, requiring advanced cooling technologies such as rear door heat exchangers or direct chip liquid cooling. It's like how a regular air conditioner can't cool a semiconductor factory.
Of particular note is the location strategy of data centers. Cloud service providers want to cluster data centers within a 50-kilometer radius for quality of service, which is directly related to data processing speed. The role of telcos is also changing. They need to evolve beyond just providing space and power to become AI infrastructure specialists.
The GPU revolution and the democratization of computing
Mark Adams, president of Penguin Solutions, declared that “2023 was the year of GPU sales.” GPUs (Graphics Processing Units) are semiconductors originally developed to process graphics for gaming, but they are now the “brains” of AI. It's like the automotive industry's shift from internal combustion engines to electric motors, with batteries becoming a key component. In fact, GPU market leader NVIDIA's data center revenue grew 279% year-over-year in 2023 to $18.5 billion.
The importance of GPUs is changing the way AI is developed. The more computing power used to train AI models, the higher the quality and efficiency of the models. “More companies should be able to utilize AI technology without breaking the bank,” says Stephen Gallivan, President of Lambda Labs. This is where GPU as a Service comes in.
GPU as a Service is a way to subscribe to GPU resources and use them only as needed, just like watching a movie on Netflix. In the past, purchasing a server with a high-performance GPU required an upfront investment of hundreds of millions of dollars. But now you can pay by the hour or by the job. It's like using car sharing instead of buying a car.
“We see a huge shortage of data center capacity for AI in the next three to five years,” Adams said. The demand for GPUs is currently exploding as companies around the world dive into AI development. GPU supply, on the other hand, is limited. There are only a few companies capable of producing high-performance GPUs, and it will take years to ramp up production facilities.
This makes efficient utilization of GPUs even more important. SK Telecom's GPU service in collaboration with Lambda Labs aims to utilize limited GPU resources as efficiently as possible. This is done by appropriately distributing AI training tasks that use GPUs around the clock, and inference tasks that use GPUs only when needed, to maximize resource utilization.
Most notably, these changes are leading to the democratization of AI technology. AI development that was once only available to giants like Google and Meta is now available to SMEs and startups. This is expected to enable new innovations in the AI industry, just as cloud computing lowered the barriers to entry in the IT industry.
Challenges and issues: power and environmental dilemmas
The biggest challenge facing AI data centers is power. Mr. Yoo clearly presented the cost structure of operating AI infrastructure. 70% of the total capital expenditure (CapEx) is GPU-related, and 70% of the operating expenditure (OpEx) is power-related. This is similar to the electric vehicle industry, where the cost of batteries and charging account for the majority of the total cost.
South Korea in particular faces three serious challenges.
The first is the physical limitations of power supply. As Yoo says, “We can't afford to power data centers in the metropolitan area anymore,” and the power supply in the capital is already at its limit. Cloud service providers want to keep their data centers within a 50km radius of the center of Seoul to ensure quality of service, but this is no longer physically possible.
Second is the issue of power prices. South Korea's industrial power prices are higher than competitors such as India, Malaysia, and Australia. AI data centers will eventually have to compete in the global market, and these high power prices are a major weakness. It's like how high costs in manufacturing can make you less competitive. The recent 10% increase in industrial electricity prices adds to this concern.
Third is environmental concerns. Global big tech companies have pledged to reduce their carbon emissions to “zero” by 2050. But in the age of AI, that's a big challenge. “It's a paradox,” says Yoo. As the demand for AI grows, power consumption and carbon emissions inevitably increase.
Efforts are being made to address these challenges in various ways. First, the relocation of data centers to rural areas is being considered. Since latency is less important for AI model training than for real-time services, it is possible to relocate data centers to rural areas where electricity supply is plentiful. This could help revitalize local economies.
Energy solutions are also diversifying. Renewable energy sources such as solar and wind are increasingly being utilized, while new energy sources such as small modular reactors (SMRs) are also being explored. In the Middle East, special zones for data centers have been established to reduce electricity prices, a policy that Korea can learn from.
There are also ongoing technological innovations to make AI servers more energy efficient. With the introduction of new cooling technologies such as liquid cooling and direct chip cooling, research is also underway to make the AI chips themselves more energy efficient.
Preparing for the future: an era of collaboration and innovation
Building AI infrastructure is no longer a task that any one company or country can solve alone. “SK Telecom is active in collaborating with global partners in the AI data center business, and these partnerships are the strength of SK Group.” Mr. Yoo's words illustrate the new competitive paradigm in the AI era. Just as automobile companies collaborated with battery companies and software companies in the era of electric vehicles, collaboration in various fields has become essential in the AI era.
Three key factors are necessary for the success of the future AI infrastructure market.
The first is technological innovation. Penguin Solutions has already begun work on its next-generation AI accelerator board, RISC-V, Adams said. “We need to innovate two to three years out,” he says, so we need to start preparing for future technologies now.
The second is policy support. Comprehensive policy support is needed, including designating special data center zones, improving power supply systems, and fostering human resources. In particular, policies to reduce electricity prices through special zones, such as those in the Middle East, need to be reviewed to ensure global competitiveness.
The third is to build an ecosystem. It is important to create an environment where various companies, from large corporations to startups, can participate. In particular, the development of new services utilizing existing infrastructure, such as submarine cables, is also a noteworthy area.
AI infrastructure is now becoming the infrastructure of a new era, just like electricity and roads in the early 20th century. It is more important than ever for the government, businesses, and society at large to collaborate and innovate in order for Korea to become a leading player in this new era.
Comments