Under platform conflicts of interest, shopping agents' optimal-purchase rate falls to 17.3%
The CAVEAT benchmark measures agents' optimal-purchase rate falling from 78.6% to 17.3%; deployers of delegated agents should note the conflict-of-interest risk.
ImportanceLocalEvidenceE2 unreplicatedWrite-upQuick
When a platform has interests of its own, shopping agents buy the user-optimal product only 17.3% of the time, down from 78.6% — the first measurement of this conflict-of-interest setting, from the CAVEAT benchmark.
Before this, incentive-conflict environments had no benchmark, so deployers had no way to gauge how far a delegated agent could be steered away from the user's goals by a platform.
The benchmark was submitted by Yuxuan Li and three co-authors on September 23. It spans nine simulated marketplace environments and eight steering mechanisms across five model families. The authors also propose a targeted intervention they say lifts the optimal-purchase rate by up to 80 percentage points.
All results come from the authors' own simulated environments, not real platforms, and have not yet been reproduced by third parties.