InfiFPO lifts Phi-4 avg score by 3.4

The InfiFPO method raises Phi-4's average performance across 11 benchmarks from 79.95 to 83.33.

Traditional model fusion often focuses on supervised fine-tuning and discards source model probability information. InfiFPO replaces the reference model in Direct Preference Optimization (DPO) with a fused source model that synthesizes multi-source probabilities at the sequence level.

By introducing probability clipping and max-margin fusion strategies, the method aligns with human preferences while effectively distilling knowledge. Experiments show significant improvements in mathematics, coding, and reasoning tasks.

This is an author-reported result from a paper accepted to NeurIPS 2025; independent reproduction is pending.