Baidu Open-Sources ERNIE 5.5 Flash Base: Claims SOTA Among Same-Size Models with New Sparse Attention
Baidu releases a 280B-parameter MoE base model featuring Latent-Gated Sparse Attention to optimize long-context efficiency, self-reporting performance leads over DeepSeek-V4-Flash.
On November 5, Baidu officially open-sourced its latest base model, ERNIE 5.5 Flash Base. The model utilizes a Mixture-of-Experts (MoE) architecture with 280 billion total parameters and 14 billion active parameters per token, trained on 33.5 trillion tokens. It supports context lengths up to 1 million tokens during fine-tuning and inference.
To address high computational costs in long-document analysis and agent tasks, the model introduces Latent-Gated Sparse Attention, combining hybrid sparse attention with low-rank gated attention to reduce gating projection weights by 75%. It also employs Multi-Router Latent MoE to improve the correlation between routing scores and expert utility.
Baidu claims the model achieves state-of-the-art (SOTA) performance among same-size models across benchmarks for knowledge, reasoning, math, code, multilingual capabilities, and long context, surpassing DeepSeek-V4-Flash-Base, Hy3-preview-Base, and MiMo-V2.5-Base. Model weights are available on Hugging Face.