AI summaries flip a quarter to a third of buy/sell calls
Compressing filings can reverse downstream recommendations, making the summarization step a measurable risk in agentic investing.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
When a large language model compresses a financial filing into a summary, the summary can read as fluent and factually accurate yet still change the buy/hold/sell recommendation the model would give from the original text: re-running on the full text changed about one in ten recommendations, with buy/sell direction flipping in a quarter to a third of cases.
Investing workflows previously assumed a summary was safe to use as long as it was fluent and factually accurate; the decision impact of the compression step went unmeasured.
The experiment was run by Lee et al. (per Klement on Investing, the team includes people from JP Morgan, BlackRock and State Street): ChatGPT, Gemini, Qwen and DeepSeek summarized filings for the 100 largest US stocks, then Gemini issued recommendations; these specific figures come from that column's reading, not the paper's abstract page. The authors propose Agentic Context Compression: generate multiple candidate summaries and audit their disagreements against the source, arguing compression should be judged by whether it preserves decision-relevant context, not just efficiency and factual accuracy.
The study does not cover material beyond filings for the 100 largest US stocks, and no third party has yet reproduced these figures; it was submitted by Lee et al. on June 28, revised September 17, and accepted to the EMNLP 2026 Industry Track.