New framework claims intermediate-layer representations beat final layers in video models

A preprint proposes LAYERSCOPE, a framework that compares a model's representations layer by layer without task labels; its authors report that intermediate-layer representations can outperform final-layer and model-default outputs.

The team used local, global, distributional and correspondence-based geometric metrics across seven architecturally diverse models on video and multimodal classification, clustering and text-to-video retrieval tasks from MVEB/MVEB+. The paper also finds that no single geometric metric consistently predicts downstream performance.

The results are the authors' own runs; whether they generalize to more models awaits third-party testing.