Logan Lawler, Head of AI Strategy for Dell Pro Precision Workstations: Practical Guide to Selecting Models for Agents — Benchmark Shortlists, Real-World Comparisons, and Keeping the Cheapest Option
Logan Lawler, Head of AI Strategy for Dell Pro Precision Workstations at Dell Technologies, proposes a practical methodology for selecting models for AI agents: first, build a shortlist using benchmark sites such as Artificial Analysis, Arena (formerly LMArena), the Berkeley Function Calling Leaderboard, and SWE-bench; then, run comparative tests of two or three models on real-world tasks, scoring them across four dimensions—completion rate, accuracy, feedback requirements, and cost—and retain the cheapest model that performs correctly. Route routine daily steps to smaller models, reserving frontier models only for processes that truly require them. When LangChain tested NVIDIA’s NeMo Switchyard routing library on 145 agent tasks, only 7% of the steps required a frontier model.
A small model was asked to install a tool, and it replied: 'You have a brain and I don’t.' Logan Lawler, Head of AI Strategy for Dell Pro Precision Workstations at Dell Technologies, recalled this testing experience in a blog post—the toolchain had fed the model all the information needed to complete the installation, yet the model completely failed to understand the task. This anecdote leads to his core judgment: in the formula where an agent equals a model plus a tool harness, the model is the brain, and even with all necessary conditions met, a wrong brain will fail.
His method consists of three steps. First, build a shortlist from benchmark websites. He recommends four: Artificial Analysis compares quality, price, and speed using independent tests, making it suitable for filtering by budget; Arena (formerly LMArena) allows real people to anonymously compare answers from two models and vote for the better one, reflecting actual preferences in everyday writing and Q&A; the Berkeley Function Calling Leaderboard tests how accurately a model uses tools (such as web search and file access)—since agents perform work through tools, this leaderboard is the most critical for agent scenarios; and SWE-bench tests whether a model can fix real bugs in open-source software, which is ideal for agents that need to write code. Second, run comparisons on real tasks. Lawler suggests using an OpenRouter account (which provides access to models from multiple providers via a single account) or LM Studio (which loads models locally) to test one frontier model and one or two smaller models, running the same task two to three times. Third, score each run across four dimensions: completion, correctness, amount of feedback needed, and cost. Keep the cheapest model that gets the job done right.