What happened

Google Cloud announced a redesign of its AI infrastructure that puts storage ahead of compute. Ronen Schwartz told SiliconANGLE that storage, which has traditionally sat at the backend, is now a critical part of accelerator execution. The company is promoting its Rapid object storage and Rapid Bucket technology. Schwartz cited performance numbers of more than 15 terabytes per second throughput on Rapid object storage and the ability to handle 20 million queries per second at sub-millisecond latency. He also noted that Google Cloud managed clusters can deliver 10 terabytes per second, figures he described as industry-leading. The architecture places storage in the same zone as GPU and TPU clusters, reducing the distance data travels before reaching the accelerator.

Why it matters

For the past two years, enterprise AI projects have focused on securing GPU allocations, negotiating hardware deals and building machine-learning platforms. Many organisations now find that the biggest constraint is not the compute itself but the data-delivery path. Schwartz said customers often discover the storage bottleneck only after GPU utilisation falls short of targets and training timelines stretch. Training workloads require massive sequential reads and frequent checkpoint writes; a checkpoint every 15 minutes can be several terabytes in size. Slow storage freezes the entire GPU cluster, wasting expensive accelerator time. Inference workloads need real-time, low-latency responses, while agentic workloads add requirements for shared workspaces, state persistence and the ability to hibernate and reactivate on demand. By moving storage closer to the accelerator and using purpose-built Rapid technology, Google aims to keep accelerators fed with data, reducing idle time and the associated cost.

What this means for enterprise IT teams

IT teams will need to rethink AI budget allocations. Instead of directing the bulk of spend to GPUs alone, they must budget for high-throughput, low-latency storage that sits near compute resources. The zonal colocation model means moving away from default multi-zone or multi-region storage strategies for AI workloads. Teams should audit their current object-storage performance, especially checkpoint-write speeds, and compare them against the three-times-faster write and five-times-faster restore claims made for Rapid storage. Evaluating whether existing storage can meet the latency requirements of inference and the state-persistence needs of agentic workloads will become a priority. Selecting a storage solution that can deliver 15 TB/s throughput and support millions of queries per second will help maintain GPU utilisation targets and control overall AI spend.

The bottom line

Google Cloud is positioning storage as a first-class component of AI infrastructure to address the growing GPU utilisation ceiling. By delivering faster checkpoint writes, higher throughput and sub-millisecond query latency, the new architecture promises to keep accelerators busy and reduce wasted spend. Enterprises that adapt their storage strategy now will be better placed to meet AI performance targets, shorten training cycles and control costs.

Sources

  • siliconangle.com