Google Cloud launches new custom AI chips to supplement Nvidia

On Wednesday, Google Cloud announced its eighth generation of custom-built AI chips, which are also known as tensor processing units (TPUs). For this new generation, the company is splitting the hardware into two distinct models to handle different parts of the artificial intelligence workflow:

• TPU 8t: A chip geared specifically for AI model training.
• TPU 8i: A chip aimed at inference, which is defined as the ongoing usage of models after users submit prompts.

According to Google Cloud, these new chips deliver significant performance and efficiency improvements over previous generations. The specifications include:

• Up to 3x faster AI model training.
• 80% better performance per dollar.
• The ability to cluster 1 million+ TPUs to work together in a single cluster.

Despite these custom hardware developments, Google is not attempting to replace Nvidia. Instead, the company is using its own chips to supplement the Nvidia-based systems it offers within its cloud infrastructure. For example, Google promises that its cloud will have Nvidia’s latest chip, Vera Rubin, available later this year, ensuring customers have access to Nvidia’s hardware alongside Google’s own TPUs.

This strategy of supplementing Nvidia hardware rather than replacing it is common among hyperscalers—a term for large cloud providers building their own AI chips, which also includes Microsoft and Amazon. While these large cloud providers continue to expand their custom silicon capabilities, they still rely on Nvidia’s hardware to support their cloud infrastructure.

In addition to hosting Nvidia hardware, Google is collaborating directly with the chipmaker. The two companies are working to improve Falcon, a software-based networking technology. Google originally created Falcon and open sourced it in 2023 under the Open Compute Project, which is an open-source data center hardware organization. This joint effort focuses on optimizing networking technology for systems running in the cloud.

Google’s custom silicon efforts are not a recent development; the company launched its first TPU back in 2016. However, Nvidia currently holds a nearly $5 trillion market cap, making it a formidable competitor. Chip market analyst Patrick Moorhead noted that betting against Nvidia has historically proven difficult, even as large cloud providers continue to build their own hardware.

Why it matters

Google is balancing its own silicon ambitions with Nvidia’s massive market presence. By offering both its own TPUs and Nvidia’s latest chips, Google ensures its cloud remains a competitive destination for AI workloads regardless of the underlying hardware.