How Amazon Marketplace Evaluates Content Quality with Frozen Vision-Language Models and OCR Signals
The Amazon Marketplace team introduces a method for evaluating A+ content quality that avoids fine-tuning vision-language models (VLMs). Instead, it trains a lightweight CatBoost classifier on the VLM's structured scoring outputs and integrates optical character recognition (OCR) readability signals. On a holdout set of approximately 1,100 pages, the fused model achieved 94% accuracy, outperforming the VLM-only approach by 5 percentage points.
A+ Content is the rich media widget on Amazon product detail pages, where brands use combinations of images, text, and video to tell their product stories. The challenge is that checking each asset in isolation—such as verifying image clarity or grammar errors—misses cross-module quality issues: for example, a feature list might advertise a 64-ounce capacity while a lifestyle photo displays a single-serving model. To address this, the Amazon team renders the entire page as a single snapshot, allowing the vision-language model to evaluate the content holistically, much like a customer would.
A core architectural decision is to not fine-tune the vision-language model. The team reasoned that fine-tuning a base model with only a few thousand labeled samples requires GPU training infrastructure and carries a risk of overfitting. As an alternative, they keep the VLM and its scoring rubric fixed. The model outputs scores from 0 to 10, confidence values, and natural language rationales across four dimensions: Visual Storytelling, Feature Presentation, Layout, and Conversion Optimization. These structured outputs are then used as features to train a CatBoost gradient-boosted decision tree classifier to predict expert-labeled high-quality or low-quality tags. Because only the classifier is retrained, updates required as annotations accumulate or criteria change can be performed as a short CPU-only task.