Google Cloud made a significant announcement at its annual user conference, Cloud Next, stating that it is launching the fifth generation of its tensor processing units (TPUs) for AI training and inferencing. This release follows the introduction of the fourth version back in 2021, which was made available to developers just last year.
The focus of this latest iteration of the chip is on efficiency, aiming to deliver substantial improvements in both training and inferencing performance per dollar. Compared to the previous generation, the fifth generation TPU promises a 2x enhancement in training performance per dollar and a 2.5x boost in inferencing performance per dollar.
During a press conference preceding the announcement, Mark Lohmeyer, the VP and GM for compute and ML infrastructure at Google Cloud, highlighted the significance of the new chip.
Key Takeaway
The fifth generation of Google’s custom TPUs offers improved efficiency, providing a 2x increase in training performance per dollar and a 2.5x increase in inferencing performance per dollar. These upgrades make it the most cost-efficient and accessible cloud TPU to date.
Lohmeyer also emphasized that users would now have the ability to scale their TPU clusters beyond previous limits. By enabling large AI workloads to span multiple physical TPU clusters, Google Cloud allows for scaling to tens of thousands of chips. Importantly, this scaling capability is achieved in a highly cost-effective manner, granting customers exceptional choice, flexibility, and options to meet the demands of diverse AI workloads.
Alongside the launch of the new TPUs, Google also announced that it would make Nvidia’s H100 GPUs generally available to developers in the upcoming month as part of its A3 series of virtual machines.
Overall, Google’s advancements in custom TPUs and its commitment to offering a wide array of AI workload optimization options reflect the company’s unwavering dedication to empowering developers and organizations with cutting-edge technology.
