Google adds TPU 8i Max and 8tx chips to boost AI training and secure inference

What happened

Google announced the expansion of its eighth-generation TPU family with two specialised chips: TPU 8i Max and TPU 8tx. The 8i Max is built to overcome memory bottlenecks, offering double the compute, double the SRAM and double the high-bandwidth memory of the earlier TPU 8i. It also fits twice the silicon in the same rack footprint, lowering floor-space and power per square foot. The 8tx chip is focused on secure, confidential inference, embedding a hardware enclave that keeps sensitive data and model weights protected during processing. Both chips share the same architecture as the TPU 8t and use custom AMD central processing unit hosts.

Why it matters

The new TPUs address two pressing challenges for large-scale AI deployments. First, ultra-large context windows, such as the 1 million-token window used by Google’s Gemini frontier model, require substantial on-chip memory to avoid costly data swapping. By doubling memory and compute, the 8i Max lets models handle longer documents, conversations, audio, video or code without latency spikes. Second, regulated industries need to run proprietary models on confidential data. The 8tx’s built-in secure enclave lets organisations fine-tune, distil and run agents on the same hardware without moving data between separate clusters, reducing exposure to leaks.

What this means for enterprise IT teams

IT teams can expect tighter rack utilisation and lower power consumption when deploying 8i Max, freeing capacity for additional workloads. The enhanced memory bandwidth means fewer data-movement bottlenecks, translating to faster training cycles for large language models and multimodal AI. With 8tx, teams handling regulated workloads gain a single-system solution for both training-grade compute and secure inference, simplifying architecture and compliance reporting. The chips also align with the industry push toward larger context windows, meaning future model upgrades will face fewer hardware constraints.

In the words of Google: "double the compute, double the SRAM and double the high-bandwidth memory of TPU 8i"

The bottom line

Google’s TPU 8i Max and 8tx chips double key performance metrics while adding security features, directly tackling the compute and memory limits that hinder today’s massive AI models. For enterprises, the new silicon promises faster training, reduced data-centre footprint and a more secure path for confidential AI inference.

Sources

• siliconangle.com