読書リサーチレーダー投資フレームワーク
ログイン / 新規登録
← 読解に戻る
ニュース重大1 分で読めるDisconfirmAI 編集部

Study Finds Timing Is Key for Proactive AI Assistants

A new study finds that while frontier models can answer correctly if told when to speak, their performance collapses when they must decide the timing themselves, highlighting a major gap in proactive assistance.

2026-10-19 12:00 GMT+8

Researchers have introduced ProPerMem, a new benchmark designed to test whether AI assistants can proactively intervene at the right moment using long-term personal memory. Unlike traditional question-answering tasks where the system simply retrieves facts, proactive assistance requires the model to determine both what to say and when to interrupt the user.

The study reveals a stark performance drop when models are forced to manage timing independently. When researchers provided GPT-5.3-Chat with the correct evidence exactly within the optimal "trigger window," it achieved a 96% success rate. However, when the model had to select the timing itself, accuracy plummeted to 38%. Standard baselines, even those using retrieval-augmented generation, performed poorly, scoring at most 15%.

To address this, the team developed ProPerAgent, a framework that uses a graph-organized memory store to filter relevant information and gate interruptions. Without additional training, this approach raised accuracy to 37.5% on Qwen3.8-27B and 38% on GPT-5.3-Chat. With reinforcement learning post-training, performance reached 42.4%, demonstrating that explicit timing control mechanisms are essential for viable proactive AI systems. These results are author self-tested and have not yet been independently reproduced.

ソース

  • arxiv.org↗
リサーチを続ける →
ログイン / 新規登録
読書リサーチレーダー投資フレームワーク