MIT PhD Student Jacob X. Li on Machine Studying: How AI Agents Prepare Like Students for Unseen Exams

Weaviate Podcast Episode 148 invites MIT PhD student Jacob X. Li to discuss 'Machine Studying', a new concept where AI agents prepare themselves for a domain defined only by a corpus of documents. Unlike traditional agents that explore repositories from scratch at runtime, Machine Studying suggests agents should study the corpus ahead of time—like a student preparing for an unseen exam—while retaining full access to the corpus during testing.

The episode explores how this framework redefines expertise: an expert is an agent that efficiently turns inference-time search into accurate work, measured as the weighted area under the performance versus compute curve. The search budget of a ReAct-style agent loop serves as the knob, rather than just reasoning effort. Stacking this level higher yields 'studying intelligence', defined as expertise as a function of study-time budget, with retraining from scratch as the baseline.

Jacob X. Li also distinguishes Machine Studying from continual learning, which is often framed around sequences of benchmarks and collapses into long-context processing. He outlines three bets every studying method makes: self-supervised objectives or synthetic data; updating weights through LoRA, KV-cache cartridges, or trained retrievers; and amortized context management like notes and cheat sheets. For instance, cheat sheets helped on DSPy but less on the larger OpenClaw codebase, leaving open questions about the trade-offs between weight updates and context management.