CoreWeave Engineer Joël Henry: When to Use AI Agents to Build Tools vs. As Tools in Data Center Anomaly Detection

Managing a physical system like an AI data center involves processing massive streams of telemetry data—power draw, inlet temperature, coolant flow—every second. The core challenge is identifying nodes that might interrupt training runs from this continuous flow. Joël Henry, writing for the CoreWeave engineering blog, argues that the question is not whether to use AI agents, but where to use them. Pointing an agent at every part of the problem results in a slow system capable of giving different answers on different days, because language models are inherently probabilistic.

To resolve this tension, Henry describes an architecture that assigns AI agents two distinct roles: "agents to build the tool" and "agents as the tool." In the first mode, the agent works offline, exploring data, interpreting requirements, and assembling conventional machine learning pipelines. Once finished, the agent steps away, and the production system runs deterministic code and models. This approach suits repetitive, well-defined tasks requiring constant operation, such as anomaly detection itself. It ensures the same input always produces the same output and allows results to be explained by inspecting the code.

The second mode embeds the agent inside the live system to handle open-ended problems where each case is genuinely new. When the detector flags an anomaly, the agent answers questions like "Is this real?", "What caused it?", and "What should be done?" These tasks are hard to codify in advance and require judgment across GPU, network, or storage domains. In this mode, the agent incurs a real cost per call and does not guarantee determinism, but it provides human-readable reasoning to help operators understand complex situations.