Spearhead AI consulting

Meta and Microsoft Lead Demand for NVIDIA’s Powerful H100 AI Chips

Only a select set of companies have a large share of NVIDIA‘s H100 chip orders.

H100 is the ninth generation of Nvidia’s data center GPU. The device is equipped with more Tensor and CUDA cores, and at higher clock speeds, than the A100. There’s 50MB of Level 2 cache and 80GB of familiar HBM3 memory, but at twice the bandwidth of the predecessor – reaching 3 TB/sec.

H100 is the premier AI chip in the market and it is highly coveted for its performance: it promises up to 9x faster AI training and up to 30x faster AI inference in popular machine learning models over the previous generation’s A100, released just two years ago. The lead time of H100-based servers is from 36 to 52 weeks.

Omdia, a market tracking company, believes that Meta and Microsoft are the largest purchasers of Nvidia’s H100 GPUs. They procured as many as 150,000 H100 GPUs each, considerably more than the number of H100 processors purchased by Google, Amazon, Oracle, and Tencent (50,000 each). It is noteworthy that the majority of server GPUs are supplied to hyperscale cloud service providers. Server OEMs (Dell Technologies, Lenovo, Hewlett Packard Enterprise) cannot get enough AI and HPC GPUs to fulfill their server orders yet.

That being said, most hyperscalers are now creating their own chips: custom silicon for AI, High Performance Compute, and video workloads. The customized server market is estimated to be $196B by 2027.

This inclination towards custom, application-optimized server configurations is set to become the norm as the cost-efficiency of building specialized processors is realized, with media and AI being the current front runners and other sectors like database management and web services expected to join the movement.

What are your thoughts on companies’ appetite for AI chips like H100?

#Nvidia #AIInfrastructure #ServerInnovation #ProprietaryTechnology #FutureOfComputing #h100 #generativeai

Data: Omdia, Data Center Knowledge, Yahoo Finance.

Related Posts

Generative AI Transformation: Make The Trend Your Friend

Old myth: The internet is a fad. New myth: Generative AI is a fad.

Klarna’s AI ROI Case Study: faster customer resolution and $40M in efficiency in just 1 month.

Klarna's ROI on AI: it is doing the work of 700 full time agents and resolving issues in under 2 minutes compared to legacy 11 minutes resolution time.

The ROI of AI: How Generative AI generates Return on Investment around Productivity, Efficiency, and Revenue.

The ROI of AI

Elevating Excellence: The Impact of High Agency in Talent Recruitment

OpenAI’s Sora: Redefining Visual Storytelling through AI-Generated Videos

OpenAI is changing the world. Again.

Drive Smart: Electric Vehicles Outshine Gas Cars in Efficiency

Here is the cagematch to watch: efficiency of EVs (Electric Vehicles) vs Internal Combustion Engine Vehicles (ICE).
Scroll to Top