ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial3 min readDisconfirmAI Editorial

Solar Equipment Company Exowatt Self-Test: Fine-Tuning 1.7B Model Achieves Accuracy on Alarm Classification Task Surpassing Claude Sonnet 4.6

Exowatt self-tested Qwen3-1.7B after fine-tuning, achieving a macro-F1 of 0.911 on edge alarm classification, surpassing Claude Sonnet 4.6's 0.698; the key finding is that training data diversity is more important than model scale and training epochs—running for 11 epochs with the same data yielded only 0.879, while running one epoch with expanded, rewritten data of 2,522 items reached 0.911.

2026-10-28 00:50 GMT+8

Solar equipment company Exowatt provides dispatchable solar power systems for AI data centers. Its energy management system operates across an expanding fleet of sites, each continuously generating hardware alarms. The team needed to automatically classify alarms and provide next-step action recommendations on edge controllers without GPUs and often without network connectivity. Exowatt described this engineering practice in an Amazon Web Services (AWS) machine learning blog: fine-tuning the Qwen3-1.7B model for the alarm classification task achieved a self-test macro-F1 of 0.911, surpassing Claude Sonnet 4.6's 0.698, with zero inference costs (running on existing site CPUs).

Why are frontier models unfeasible in this scenario? Exowatt cites three reasons: the task is narrow and proprietary, as alarm classification relies on their specific hardware and operational logic; it must run at the edge with no GPU and often no network; and costs are unsustainable at the fleet scale, making it economically unfeasible to call cloud-based frontier models for every decision. The team initially attempted Qwen3-8B with prompting and guardrails, but macro-F1 plateaued at 0.735, missing the target. They then discovered a counterintuitive result: none of thirteen fine-tuned Qwen3-8B schemes across twelve dataset versions exceeded the prompted 8B baseline. Conversely, Qwen3-1.7B, with only one-fifth the parameters, surpassed the 8B model when paired with more diverse training data.

Sign in / Sign up
ReadingResearchRadarInvestment framework

Data strategy was the core finding. The training set consisted of only 2,522 items: 684 verified site-scenario decisions plus 1,838 rewrites generated by Qwen3-VL-235B (preserving decisions, labels, evidence citations, and all numbers, altering only phrasing). The paraphrase generation cost approximately $3.72, with 1,838 of 2,218 candidates passing deterministic checks. Key control experiments showed that using only the 684 original data points for 11 epochs (matching the step count of the expanded dataset) yielded a macro-F1 of 0.879; using the expanded, diverse 2,522-item dataset for just one epoch yielded 0.911. Exowatt concluded that for tightly bounded tasks, data quality and diversity outweigh model scale and volume; simply increasing training epochs performs worse than increasing data diversity.

In a cloud-based energy market analysis task, the same pipeline lifted Qwen3-14B from a baseline macro-F1 of 0.415 to 0.975, surpassing Claude Opus 4.7's 0.625, with a cost of $0.60 per thousand responses compared to $29.26 (a roughly 49-fold difference). Exowatt emphasized that these two models were merely the first outputs of the pipeline; the more enduring outcome was the pipeline itself: a repeatable path from raw operational data to certified training sets, fine-tuned models, and evaluated deployment artifacts, compressing iteration cycles from months to days.

Boundaries to note: All performance data are self-tests by Exowatt and AWS, without independent reproduction. Both tasks involved single-turn strict JSON output classification, where every claim returned by the model had to cite specific rows from the input data; unsupported claims were discarded. Test sets comprised 320 and 2,000 samples respectively. Cost comparisons used Amazon Bedrock list prices multiplied by measured tokens; the edge model's zero cost refers to the absence of API call fees, excluding hardware depreciation and operations/maintenance. These figures illustrate relative advantages on specific narrow tasks and cannot be extrapolated to multi-step reasoning, open-ended generation, or scenarios requiring world knowledge.

Sources

  • aws.amazon.comAmazon Web Services↗
Continue researching →