Preprint: MaLoW Internalizes Streaming Video History into LoRA Weights, Boosting OVO-Bench Backward Accuracy by 3.9-11.8%
New framework MaLoW consolidates video history into LoRA weights via a recurrent HyperNetwork, resolving context competition in streaming VLMs.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
Researchers from the University of Wisconsin–Madison and Peking University have proposed MaLoW, a new memory framework for streaming Vision-Language Models (VLMs). Instead of reintroducing historical observations into the limited input context, the framework uses a recurrent HyperNetwork to incrementally consolidate incoming video observations into fixed-size LoRA weights.
On the backward split of the OVO-Bench benchmark, MaLoW outperforms prior memory approaches by 3.9% to 11.8% across different video backbones while incurring only modest latency overhead. The method also generalizes to vision-language-action models, improving the action success rate on Simpler WidowX by 9.3%.
To support training, the team constructed the MaLoW-Instruct dataset, containing 381K question-answer pairs over 56K videos. Current results are based on author-reported benchmarks and have not yet been independently reproduced.