LeituraPesquisaRadarFramework de investimento
Entrar / Cadastrar
← Voltar para a leitura
NotíciasTópico · 戴尔智能体模型选型策略Material3 min de leituraEditorial DisconfirmAI

Logan Lawler, Head of AI Strategy for Dell Pro Precision Workstations: Practical Guide to Selecting Models for Agents — Benchmark Shortlists, Real-World Comparisons, and Keeping the Cheapest Option

Logan Lawler, Head of AI Strategy for Dell Pro Precision Workstations at Dell Technologies, proposes a practical methodology for selecting models for AI agents: first, build a shortlist using benchmark sites such as Artificial Analysis, Arena (formerly LMArena), the Berkeley Function Calling Leaderboard, and SWE-bench; then, run comparative tests of two or three models on real-world tasks, scoring them across four dimensions—completion rate, accuracy, feedback requirements, and cost—and retain the cheapest model that performs correctly. Route routine daily steps to smaller models, reserving frontier models only for processes that truly require them. When LangChain tested NVIDIA’s NeMo Switchyard routing library on 145 agent tasks, only 7% of the steps required a frontier model.

2026-11-12 22:00 GMT+8

A small model was asked to install a tool, and it replied: 'You have a brain and I don’t.' Logan Lawler, Head of AI Strategy for Dell Pro Precision Workstations at Dell Technologies, recalled this testing experience in a blog post—the toolchain had fed the model all the information needed to complete the installation, yet the model completely failed to understand the task. This anecdote leads to his core judgment: in the formula where an agent equals a model plus a tool harness, the model is the brain, and even with all necessary conditions met, a wrong brain will fail.

His method consists of three steps. First, build a shortlist from benchmark websites. He recommends four: Artificial Analysis compares quality, price, and speed using independent tests, making it suitable for filtering by budget; Arena (formerly LMArena) allows real people to anonymously compare answers from two models and vote for the better one, reflecting actual preferences in everyday writing and Q&A; the Berkeley Function Calling Leaderboard tests how accurately a model uses tools (such as web search and file access)—since agents perform work through tools, this leaderboard is the most critical for agent scenarios; and SWE-bench tests whether a model can fix real bugs in open-source software, which is ideal for agents that need to write code. Second, run comparisons on real tasks. Lawler suggests using an OpenRouter account (which provides access to models from multiple providers via a single account) or LM Studio (which loads models locally) to test one frontier model and one or two smaller models, running the same task two to three times. Third, score each run across four dimensions: completion, correctness, amount of feedback needed, and cost. Keep the cheapest model that gets the job done right.

Entrar / Cadastrar
LeituraPesquisaRadarFramework de investimento

After scoring comes routing. Lawler cites specific data: when LangChain tested NVIDIA’s NeMo Switchyard routing library on 145 agent tasks, only 7% of the steps required a frontier model. This means that the vast majority of daily steps—calling tools, summarizing results—can be handled by small models. His personal production setup uses frontier models (via OpenRouter and his Claude plan) for designing agents and writing skills, because that represents 'hard thinking that requires payment'; once skills are built, they run daily on DeepSeek V4 Flash, deployed on a Dell Pro Precision GB10 workstation.

Lawler also defines boundaries. If comparison testing shows that only a frontier model can get the task done correctly, or if the task falls into the hardest categories of coding, reasoning, or research, do not force the use of open-source models. His exact words were: 'Don’t force an open-source model into a race it cannot win yet.' He also emphasized that he is constantly testing and advised readers to ask him again in a quarter.

Contextual note: This article was published on the official Dell Technologies blog. The author’s role is to lead the AI strategy for Dell Pro Precision workstations, and the NVIDIA Nemotron series models, NeMo Switchyard routing library, and Dell GB10 workstation recommended in the article are directly related to his commercial interests. The 7% figure from LangChain comes from its testing of the NVIDIA Switchyard routing library and is not an independent third-party assessment. While the methodology itself—building a shortlist, running comparisons, scoring across four dimensions, and keeping the cheapest option—is transferable, the specific tool recommendations should be viewed as suggestions carrying vendor bias.

Fontes

  • dell.comDell↗
Tópico

戴尔智能体模型选型策略

【背景】在智能体开发中,模型是“大脑”,错误的模型选择会导致即使工具完备也无法完成任务。戴尔Pro Precision工作站AI策略负责人Logan Lawler提出了一套基于“基准短名单+真实任务对比”的选型策略:利用Artificial…

4 etapas1 peçasAtualizado 2026-11-17 15:20

Como chegamos aqui

  1. 2026-06-30NVIDIA发布NeMo Switchyard v0.1.0版本,作为开源LLM路由层。dell.com ↗
  2. 2026-08-10NVIDIA发布NeMo Switchyard v0.2.0(Rust重写版)及Nemotron 3.5 Lightning模型。dell.com ↗
  3. 2026-08-01Dell IT发布Model IQ白皮书,介绍用于LLM选择和生命周期管理的程序化评估框架。dell.com ↗

O que ainda está indefinido

  • NeMo Switchyard何时能从‘评估/演示’状态过渡到稳定的生产级组件?
  • 在不同行业垂直领域(如法律、医疗),Model IQ框架如何定制Ground Truth数据集以反映特定业务意图?
  • 随着开源模型能力逼近前沿模型,未来六个月两者在编码等关键领域的性能差距是否会进一步缩小?
Ler o tópico completo →
Continuar pesquisando →