Intuit Credit Karma Engineer Harish Gaggar Details Governed Enterprise AI Agent Infrastructure
In an AI Engineer conference talk, Harish Gaggar from Intuit Credit Karma outlines the 'paved path' for deploying governed enterprise AI agents. The session covers registration workflows, centralized MCP gateways for authentication, LangGraph policies with human approval, and observability using OpenTelemetry. It also discusses PII filtering and context optimization strategies, providing a reference architecture for moving agents from experimentation to production.
AI Engineer conference speaker Harish Gaggar, an engineer at Intuit Credit Karma, presents a detailed walkthrough of the infrastructure required to move enterprise AI agents from local development to production. The talk focuses on the 'paved path'—a standardized deployment process that ensures agents have consistent access to identity, memory, traces, evaluations, and secret handling.
Gaggar describes an architecture where a centralized Model Context Protocol (MCP) gateway sits between agents and tools, managing authentication and routing for every request. Custom MCP servers expose reviewed tools, while observability is maintained through OpenTelemetry, Tempo, Prometheus, and Grafana, allowing teams to monitor failures, latency, and access behavior.
The session details a LangGraph workflow that incorporates policy checks and human approval for sensitive actions. State management is handled through threads, runs, and checkpoints, with separate tracking tables recording model usage, tool calls, costs, and evaluation fields. Gaggar emphasizes that these records allow for inspection of agent behavior rather than establishing a validated quality score for the entire system.
Security and efficiency measures include a PII filtering layer designed to intercept sensitive inputs before they reach models or traces, though detection remains a control to be evaluated rather than a guarantee. Additionally, context optimization techniques such as pruning, reranking, deduplication, and compression are demonstrated, showing significant token reductions in specific queries, although no paired answer quality study or measured business ROI was presented.