Meta self-reports Bumblebee beating HSTU and DLRM on industrial c-NE
Meta's team self-reports Bumblebee at 0.7895 average c-NE on 81 billion industrial samples, below HSTU's 0.7964 and DLRM's 0.8034; not independently reproduced.
ImportanceLocalEvidenceE2 unreplicated
Bumblebee, a recommendation architecture from Meta's team, achieves an author-reported average c-NE of 0.7895 on industrial data with 155.2 million unique users per 1-hour timestep and 81 billion samples in total, below HSTU and DLRM.
Previously, industrial recommenders used HSTU (0.7964) and DLRM (0.8034) as baselines, with sequence modeling and feature crossing arranged sequentially, leaving the gains of interleaving untapped.
Author-reported: Bumblebee interleaves sequence modeling and feature crossing into stackable blocks, averaging c-NE 0.7895 versus HSTU's 0.7964 and DLRM's 0.8034; interleaving the same components improves NE by 0.20% over sequential arrangement, at the cost of roughly 7-10% slower training throughput.
The results have not been independently reproduced; the data comes from an arXiv preprint (2607.24804, updated to v3 on September 15).
Sources: