2023-12-26 08:57:00
Nvidia has had exponential success with sales of its H100 GPUs, with an average consumption of up to 700 W, in the USA. As these GPUs are adopted at scale for AI workloads, aggregate power consumption is predicted to reach extraordinary levels, rivaling that of entire American cities and even surpassing that of some smaller European countries.
Data indicates that the total energy consumption of data centers dedicated to AI applications already resembles that of a country like Cyprus, according to estimates from Schneider Electric in October. However, the most intriguing question is the specific impact of Nvidia’s most popular GPUs, the H100 and A100, on the energy landscape.
Paul Churnock, senior electrical engineer for technical governance and data center strategy at Microsoft, projects that by the end of 2024, when millions of these GPUs are in operation, Nvidia’s H100s will consume more energy than all the homes in Phoenix, Arizona . Despite this, total consumption will still fall short of large cities such as Houston, Texas.
“This is Nvidia’s H100 GPU; it has a maximum power consumption of 700 W. With an annual usage of 61%, it is equivalent to the power consumption of the average American household occupant,” it says in a post on Linkedin.
He estimates that, with the projected sales of 1.5 to 2 million H100 GPUs in 2024these processors will rank as the fifth largest energy consumer, behind Houston, Texas, but surpassing Phoenix, Arizona.
To provide numerical context, an H100 GPU, operating at 61% annual utilization, would consume approximately 3,740 kilowatt-hours (kWh) of electricity annually. Projecting these numbers to 1.5 to 2 million GPUs in 2023 and 2024, respectively, would result in a total consumption of approximately 13,092 gigawatt hours (GWh) per annum.
While this number is comparable to the annual consumption of some countries, such as Georgia, Lithuania or Guatemala, it is vital to highlight that AI and GPU HPC efficiency is constantly improving, signaling advances that could offset these energy consumption numbers in the future.
1703602782
#Nvidia #reach #mark #million #H100 #GPUs #consumption #700W #deployed #USA