독서리서치레이더투자 프레임워크
로그인 / 회원가입
← 읽기로 돌아가기
뉴스간단한 요점1 분 소요DisconfirmAI 편집부

Arize AI Product Lead SallyAnn DeLucia Demonstrates Hands-On AI Evaluation Workshop

SallyAnn DeLucia, Head of Product at Arize AI, presented a hands-on workshop at the AI Engineer World's Fair on moving from vibe checks to real evals. The session covers what an eval is, how it differs from unit tests, main eval types (including LLM as a judge), reading traces, and building an LLM judge.

2026-10-26 07:00 GMT+8

SallyAnn DeLucia, Head of Product at Arize AI, presented a hands-on workshop on AI evaluations at the AI Engineer World's Fair. This resource aims to help engineers understand the transition from informal 'vibe checks' to systematic evaluation processes.

According to the publisher's chapter list, DeLucia explains what an eval is and how it differs from a unit test, while also discussing why agents make evals harder. She covers various types of evals, including LLM as a judge, decision models, code evals, annotations, and agent as a judge.

The workshop details practical skills such as how to read a trace, define what 'good' means, write concrete pass/fail criteria, and build a dataset from traces. It concludes with a demonstration of building and iterating on an LLM judge in Arize's prompt playground, along with next steps like alignment and online evals.

출처

  • youtube.com↗
리서치 계속하기 →
로그인 / 회원가입
독서리서치레이더투자 프레임워크