Dharma-AI Fine-Tuning Time Prediction
2 steps1 pieces4 Sources
Dharma-AI, a Brazilian AI lab specializing in Small Specialized Language Models (SSLMs), has released findings from an internal study on predicting the end-to-end duration of supervised fine-tuning (SFT) jobs. The team analyzed 272 production runs on NVIDIA H200 GPUs, totaling approximately 7,382 GPU-hours. Their results indicate that the industry-standard FLOP-based calculation formula is a poor predictor of actual wall-clock time, achieving an R² of only 0.31 and a mean absolute error (MAE) of 12.41 hours. In contrast, a CatBoost machine learning model trained exclusively on configuration data available before job dispatch achieved an R² of 0.98 and an MAE of 1.71 hours.
The core finding is that computational volume (parameters and tokens) is not the primary driver of fine-tuning duration; rather, operational overheads such as evaluation frequency, checkpoint writing, and shared cluster contention are dominant factors. The most significant predictive feature identified was num_eval_data (total samples processed during evaluations). This suggests that for teams managing shared GPU clusters, scheduling accuracy depends more on I/O and evaluation cadence settings than on raw compute estimates.
However, these results are currently issuer-reported and based on a single organization's logs using specific hardware (H200s). The full paper, titled "Predicting End-to-End Fine-Tuning Time for Generative Language Models," is forthcoming on arXiv, but no code or dataset has been released yet. The model's transferability to other GPU generations or workloads like RLHF remains unverified.
How it got here
November 18, 2026: Dharma-AI publishes a blog post previewing their upcoming paper, detailing the analysis of 272 SFT jobs on H200 GPUs.
huggingface.co ↗November 18, 2026: Secondary outlets including Aiholics, Quanta AI Labs, and DataPhoenix report on the findings, highlighting the disparity between FLOP estimates and learned predictors.