What It Really Takes to Unlock AI Adoption: Lessons from the Field

AI adoption has moved fast — faster than most security teams, compliance frameworks, or vendor questionnaires have been able to keep up with. I recently presented a webinar with Ravi Nori, Head of Cybersecurity at Gopuff, and Max Pollard, Co-Founder and CEO of Cotool, to dig into the realities of integrating AI at scale: the productivity wins, the security trade-offs, and the hard-won lessons that don't make it into the press releases.

Here's what stood out.

The Gap Between Policy and Reality

One of the first things Ravi shared was something many security leaders quietly know but rarely admit out loud. His team had officially approved five AI tools, yet discovered over 100 in active use across the organization. He himself was caught using an unapproved tool during an internal scan.

This isn't unique to Gopuff. It's the nature of a fast-moving field where the productivity gains are real and immediate, and the guardrails are still being built. The lesson isn't to lock everything down, because that kills momentum. The lesson is that monitoring has to come before restriction. You can't govern what you don't understand, and you can't understand what you haven't mapped.

Ravi's framework is sequential and practical:

1. Discover what's in your environment and who's using it
2. Build alerting around behaviors you want to catch.
3. Only then move toward prevention. Jumping straight to restriction without that foundation creates friction without security.

Data Privacy Is Harder Than It Looks

Both Max and Ravi emphasized that contractual guarantees around data retention are the beginning — not the end — of a privacy strategy.

Max described how Cotool negotiated zero-day data retention agreements with all major model providers before writing a line of product code. That's the right foundation, but Ravi illustrated exactly why it's insufficient on its own. When his team audited how their customer-facing chatbot was actually handling data, they found that a third-party agent provider's logs contained unredacted credit card numbers and personal information that customers had shared in frustration during order delays. The contract said the right things. The implementation didn't follow through.

The practical implication is trust but verify. Contracts establish intent, but technical audits establish reality. If an AI agent you don't control is processing sensitive customer data, you need to be watching what it actually does with that data, not just what its vendor promises.

Max also raised a point worth sitting with. The explosion of sub-processors and third-party model integrations means that even well-governed organizations now have data flowing through systems they have limited visibility into. Mapping that data lineage is increasingly difficult and increasingly important.

The Shadow AI Problem Is the New Shadow IT

Anyone who lived through the early cloud migration era will recognize this dynamic. Shadow IT was the phenomenon of employees using unapproved tools like Dropbox, Google Drive, or personal email to get work done faster. Security teams spent years playing whack-a-mole trying to contain it.

Shadow AI is the same pattern, accelerated. Employees are using personal accounts with AI tools, pasting code snippets containing secrets into consumer chatbots, and uploading internal documents for analysis because the enterprise-approved tool their company has given them doesn't have the feature they need.

Ravi explained that you can't just invest in more documentation and expect employees to read and internalize every policy. Most won't. The effective answer lies in automated controls that make the risky path harder and the compliant path easier. The goal is not to block everything, but to get enough visibility and implement selective enforcement to catch meaningful data exposure without grinding productivity to a halt.

Human-in-the-Loop Is a Systems Design Decision

Max made a sharp observation when discussing AI agents in security operations. He used the framing of determinism versus non-determinism. The power of modern AI agents is precisely their ability to reason across contexts and take action without being pre-scripted. But that same quality is what makes them dangerous if left fully autonomous.

The example he gave was instructive. A Cotool detection agent identified that a set of endpoints were making unusual API calls, traced them back to devices that had been stolen from a delivery truck, and was prepared to remotely wipe them. That's a legitimate, valuable action. But the same reasoning applied to a miscategorized host could take down a payment processing system and cause major business consequences.

Adding a human checkpoint to every action defeats the purpose. Instead, security teams must be intentional about which actions get a human gate. For most triage and investigation, let the agent run. For irreversible or high-blast-radius actions like network isolation, account termination, and system wipes, build in a mandatory human review. Teams need to think more about good systems design and less about caution for its own sake.

Transparency Is What Makes AI Decisions Acceptable

A theme that came up repeatedly was transparency as a practical tool for building internal and external trust, not just a compliance checkbox.

At Material, we've found that customers are far more willing to accept an AI-driven decision, even a wrong one, if they can see the reasoning behind it. A phishing email flagged incorrectly is frustrating. A phishing email flagged incorrectly with no explanation is infuriating. When we can show a customer that the system flagged a message because of a newly registered domain, a novel sender, and urgency language — even if we got it wrong — the conversation becomes constructive rather than adversarial.

Max described building this out for Cotool. They started with full agent trace logs (complete but overwhelming), then invested heavily in structured summaries that distill the key findings while keeping the full trace accessible for deeper investigation. Neither extreme worked alone. The right answer was layered explainability, providing a quick summary for everyday use plus a detailed audit trail for when something goes wrong.

The Risks People Overestimate (and Underestimate)

We unpacked a pointed question: what AI security risk do organizations overestimate, and what are they actually underestimating?

On overestimation, Max pushed back on the degree of anxiety around internal employees feeding sensitive data to public AI models. His argument was that if a major model provider wanted proprietary data, they have far more sophisticated ways of obtaining it than waiting for an employee to paste something into a chat window. The energy spent on enterprise-wide DLP for AI inputs might, in many cases, be better deployed elsewhere.

On underestimation, he pointed to prompt injection, and particularly the "lethal trifecta" of customer-facing agents that have access to private data, are exposed to untrusted internet content, and have the ability to exfiltrate information. When all three conditions are present simultaneously, it is, in his words, logically impossible to guarantee that injection attacks can't succeed. Ravi confirmed his team was already seeing sophisticated prompt injection attempts against Gopuff's customer chatbot at meaningful scale.

Ravi's point on the underestimation side was about institutional knowledge gaps. Many security teams are using AI every day without deeply understanding how it works. That gap makes it hard to ask the right questions, protect the right things, or evaluate vendor claims accurately. The recommendation was straightforward — security leaders need to invest in their own AI literacy, not delegate understanding to vendors.

The Upside Is Real, and It's Getting Bigger

After an hour of security risks and governance challenges, we made sure to end where the story actually begins, with the value.

Ravi described AI as a genuine game-changer for his security work. AI is enabling faster analyses, better triage of alerts, sanity-checking vendor remediation recommendations, and handling reporting tasks that used to consume hours. Max talked about operations that once took 20-30 minutes now taking seconds, with human review required only on a small fraction of outputs. Across both companies, the cumulative time savings translate to meaningful capacity in hours per week, per person, that’s now redirected toward higher-judgment work.

There's also a defensive imperative here that neither of them shied away from. As the cost of conducting attacks with AI approaches zero, the cost of defending without AI becomes existential. The organizations that build genuine AI fluency now — using it, understanding its failure modes, and designing workflows around its strengths — will have a meaningful advantage in the years ahead.

The risks are real. The governance work is hard. But sitting on the sidelines while competitors and adversaries move forward isn't a neutral choice.

‍