NVIDIA Technical Blog: How to Fine-Tune Speech Recognition Models for Saudi Arabic Dialects While Retaining Multilingual Capabilities

Automatic Speech Recognition (ASR) models often perform well on mainstream languages and standard pronunciations but struggle when deployed in specific regional dialects or noisy environments. On October 1, 2026, the NVIDIA Technical Blog published a detailed tutorial demonstrating how to use the NeMo framework to fine-tune the Nemotron 3.5 ASR model to accurately recognize Saudi Najdi and Hijazi dialects without sacrificing existing English and other Arabic capabilities.

The core of this workflow addresses "catastrophic forgetting," where a model loses old knowledge while learning new dialects. The authors propose a weighted replay mixing strategy: combining 90% target dialect data with 10% previously learned data (7% English FLEURS and 3% Arabic FLEURS). This explicit declaration of proportions prevents issues where sliding-window shuffling might delay the effect of appended replay data until late in training. Experiments showed that just 7% English replay data was sufficient to slightly improve English Word Error Rate (WER) from 11.04% to 10.42%, proving the method effectively retains multilingual capabilities.

Regarding data processing, the article emphasizes "minimal curation." Instead of blindly applying default quality score thresholds, the authors inspected distributions before setting cutoffs. For instance, a default UTMOS quality threshold of 3.0 would have rejected almost all samples, whereas the actual threshold used was 1.25. By removing inaudible annotations (like "غيرواضح") and severely misaligned clips, they retained 103,559 valid utterances (82.5% of the original set). Additionally, duration-based bucketing reduced padding waste, making training more efficient for streaming encoders.