After modernizing convolutional networks, their results can match attention models

Some prediction tasks involve data that is not a row of a table but an entire image — judging what is in the picture and drawing boxes around where objects are. Comparing which of these models is stronger usually comes down to their accuracy on commonly used image-recognition test sets. The ConvNeXt work showed that after retrofitting old-style convolutional networks with new training methods, their results matched the strongest attention-based models of the time.

When vendors demo new models, they often credit the advantage to architectures like the attention mechanism. This work showed that the gap mostly comes from training methods rather than the architecture itself, and later discussions of "does model architecture actually matter" still cannot get around it.

If the task is not this kind of image-recognition benchmark, or the input resolution is extremely high, don't use it to judge architectural merit; it also did not prove that convolutional networks dominate in all vision scenarios.

《A ConvNet for the 2020s》(2022) | Next review 2027-09-20