New York Life Experts Analyze 1,368 AI Failures: Building Evaluation Systems from Real Incidents
Vishal Srivastava and Tanmay Sah from New York Life shared insights from studying 1,368 public AI incidents on the AIE Podcast. They categorize failures into technical strength, deployment controls, and business impact, noting that weak deployment controls like human review are most closely tied to harm. The episode demonstrates how to rank use cases by risk and apply a build-break-fix loop to validate AI systems, offering a concrete framework for practitioners.
As AI systems iterate rapidly, effectively testing and evaluating their reliability has become a critical industry challenge. This episode of the AIE Podcast features Vishal Srivastava and Tanmay Sah from New York Life, who discuss what real-world failures teach us about AI evaluation, based on an analysis of 1,368 incidents from a public AI event database.
The guests propose a classification framework that sorts AI failures into three main areas—technical strength, deployment controls, and business impact—and 13 sub-areas covering approximately 98% of incidents. Their research highlights that the factors most closely tied to severe harm are often not technical flaws in the model itself, but weak deployment controls, such as insufficient human review and monitoring mechanisms.
The discussion also covers practical evaluation methodologies, including how to rank use cases by materiality and why the evaluator itself needs verification. They demonstrate a 'build-break-fix' iterative loop designed to prove a system wrong rather than just right, illustrated with a real case of an airline chatbot providing incorrect refund policies. For readers focused on AI safety implementation and evaluation engineering, this episode offers a structured, evidence-based perspective.