As enterprises shift from AI copilots to autonomous agents, a constraint that is often underestimated is the data the agents access themselves. Eléonore Hébrard, a solutions engineer at Box, offered a concise assessment during a discussion jointly hosted by Sifted and Box: agents can hallucinate and fail when fed disorganized data or encountering duplicate or outdated files—the problem lies not with the AI, but with the content. She further noted that an agent's speed and capacity make the blast radius of errors far greater than human operational mistakes; therefore, data governance is not optional but a prerequisite.
Leo Rosales, co-founder and CTO of Ralio, approached the issue from another angle: maintaining skepticism toward third-party AI models themselves. He described his company's approach as assuming the model has been compromised and building control layers around that assumption. They define what an agent can and cannot do, directly blocking it when it attempts to execute actions beyond its authorized scope. This is not a localized validation of model output, but a fundamental distrust of the entire model's reliability—here, 'zero trust' means a default hostile assumption toward the model itself.
Mick McNeil, founder and CEO of Deliverance AI, compared agent deployment to new employee onboarding. He would not grant a new salesperson full access to all company systems on day one; instead, he defines their role scope, gradually opens system access, and has managers continuously monitor performance. An agent's autonomy should be earned over time, not granted in one lump sum. He emphasized that while giving agents knowledge, you must also provide context—just as you define role boundaries for humans, you must instruct the agent on how to think and act when it encounters data.
Regarding specific permission management, Hébrard stressed that agents should not have broad system access; permissions must be task-specific, time-bound, and context-aware. Hanah-Marie Darley, co-founder and Chief AI Officer at Geordie AI, pointed out two overlooked risks. The first is silent failure: the agent is running, but the organization lacks visibility into its real-time behavior, meaning problems may occur without triggering any alerts. The second is the dilution of human oversight: many teams fail to explicitly designate who the human is and what information they need to make judgments, resulting in supervision that appears present but actually reduces effectiveness.
The common premise behind these recommendations is that agents are shifting from generating text to executing operations; once connected to databases, the consequences of errors escalate sharply. McNeil pointed out that humans must always remain accountable—humans grant agents the capability to execute, not the other way around. Hébrard requires organizations to enforce dynamic security controls at the content layer, logging every prompt and reasoning step to ensure that 100% of events can be explained during audits.
Boundaries to note: This discussion was sponsored by intelligent content management platform Box, with the four guests representing companies in AI agent development, content management, AI payments, and AI security. Emphasizing the importance of governance complexity and content quality naturally aligns with their business models. Furthermore, all recommendations are practitioner experience-based judgments; no quantified implementation costs, failure rate comparisons, or independently validated framework effectiveness data were provided. They serve as useful practical starting points, not verified standard answers.