Insights from Google Cloud’s COO on AI Security Challenges

In a recent discussion with Francis de Souza, the COO of Google Cloud, at a gathering in Los Angeles, he elucidated key strategies for firms navigating the evolving landscape of AI security. Speaking in a composed manner, much like a university professor, de Souza emphasized the inevitability of a transitional phase before reaching an improved state regarding security measures.

While he wasn’t directly addressing Google, it’s evident that the tech giant is still finding its footing.

De Souza’s primary message aligns with years of advice from security experts urging businesses to stop treating security as an afterthought. “As organizations embark on their AI journeys, they need to adopt a platform-oriented strategy,” he stated. “Security cannot be merely added later, nor can it be solely the responsibility of individual employees.” He specifically cautioned against “shadow AI,” where employees resort to consumer-grade tools without oversight, insisting that firms must enforce security, governance, and audit processes from the very beginning. “An AI strategy is intertwined with both a data strategy and a security strategy. They must develop in conjunction,” de Souza asserted.

It’s noteworthy that his comments were not solely a promotion for Google Cloud. When he sensed that his advice sounded like a marketing pitch, he clarified that Google embraces a multicloud strategy, highlighting that companies operating under the assumption of a single cloud often utilize various SaaS applications and collaborate with partners on different clouds. “Even within a single-cloud framework, the reality is they engage with multiple environments. It’s crucial for businesses to maintain a cohesive security posture across all platforms,” he explained.

De Souza further remarked on the dramatically altered threat landscape, asserting that traditional defensive approaches can no longer keep pace. He indicated that the timeframe between an initial breach and the next phase of an attack has shrunk from eight hours to just 22 seconds, as the attack surface has expanded beyond conventional network boundaries. “Beyond your standard environment, you now have models, data pipelines for training, agents, and prompts that also require protection,” he noted.

He highlighted a less-discussed threat: agents traversing an organization’s internal systems can uncover neglected data repositories that may have been forgotten. “Many organizations have outdated SharePoint servers and access controls that have not been updated, simply because no one remembered they existed. Agents within your enterprise will locate those data assets, putting them at risk,” he said.

In his view, the solution lies in synchronizing machine response with machine speed. “We are witnessing the rise of an AI-native, fully agentic defense whereby organizations can deploy agents to handle their own defenses,” he remarked. “Rather than having a defense managed solely by humans, you can allow humans to supervise a fully autonomous defensive mechanism.” He added that this issue has escalated to a top-level leadership priority. “This is essential for board members and executive teams, not just the security departments.”

However, as AI assumes more responsibility within cybersecurity, the pool of qualified individuals available to manage it is diminishing. The vulnerabilities introduced by AI are growing more rapidly than security teams can remediate. “We are going to face a ‘bug-pocalypse,’” LinkedIn’s chief information security officer Lea Kissner stated recently to the New York Times, stressing that the industry won’t fully grasp AI security in a sustainable manner for years to come.

This situation leads back to the platform providers. Recently, The Register reported a surge in incidents where Google Cloud developers received hefty bills due to unauthorized API calls to Gemini models. Many of these individuals had not actively used or enabled these services. The alarming trend involved API keys initially set up for Google Maps that could unexpectedly access Gemini after Google changed their parameters without adequate notification.

Rod Danan, the CEO of interview-prep platform Prentus, reported seeing charges of $10,138 in just 30 minutes after attackers exploited his compromised API key. Similarly, Isuru Fonseka, a developer in Sydney, found himself liable for around AUD $17,000 despite believing he had a $250 spending limit. Neither was aware that Google had automatically adjusted their billing tiers based on history, which could escalate limits up to $100,000 without prior consent.

Following initial reports by The Register, Google issued refunds to both individuals. Nevertheless, Google stated it does not intend to modify its automatic tier-upgrade policy, emphasizing service continuity over enforcing users’ budgetary constraints.

There’s also the issue of what occurs when a developer attempts to deactivate a service. The Register recently reported findings from the security firm Aikido, revealing that even when developers immediately delete a compromised key, attackers can still utilize that key for up to 23 minutes due to Google’s gradual key revocation process. Aikido researcher Joseph Leon noted that in some instances, over 90% of requests remained authenticated during this timeframe, allowing attackers the opportunity to extract files and conversation data from Gemini.

Leon observed that Google’s newer credential formats do not face the same issue; service account API credentials can be revoked in about five seconds, while the newer AQ-prefixed key format takes roughly a minute. “Both run effectively at Google’s scale,” he explained in Aikido’s paper. “This suggests that the 23-minute delay is not a technical limitation but rather a matter of company priorities.”

These considerations should be kept in mind when reflecting on de Souza’s guidelines, which are indeed valuable and merit serious attention. He has valid points, yet there exists a disconnect between the security practices he advocates and the speed at which the platforms are adapting, making it essential to remain vigilant.