InfiFPO lifts Phi-4 avg score by 3.4
New fusion method raises Phi-4's average benchmark score from 79.95 to 83.33.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
The InfiFPO method raises Phi-4's average performance across 11 benchmarks from 79.95 to 83.33.
Traditional model fusion often focuses on supervised fine-tuning and discards source model probability information. InfiFPO replaces the reference model in Direct Preference Optimization (DPO) with a fused source model that synthesizes multi-source probabilities at the sequence level.
By introducing probability clipping and max-margin fusion strategies, the method aligns with human preferences while effectively distilling knowledge. Experiments show significant improvements in mathematics, coding, and reasoning tasks.
This is an author-reported result from a paper accepted to NeurIPS 2025; independent reproduction is pending.