Under platform conflicts of interest, shopping agents' optimal-purchase rate falls to 17.3%

When a platform has interests of its own, shopping agents buy the user-optimal product only 17.3% of the time, down from 78.6% — the first measurement of this conflict-of-interest setting, from the CAVEAT benchmark.

Before this, incentive-conflict environments had no benchmark, so deployers had no way to gauge how far a delegated agent could be steered away from the user's goals by a platform.

The benchmark was submitted by Yuxuan Li and three co-authors on September 23. It spans nine simulated marketplace environments and eight steering mechanisms across five model families. The authors also propose a targeted intervention they say lifts the optimal-purchase rate by up to 80 percentage points.

All results come from the authors' own simulated environments, not real platforms, and have not yet been reproduced by third parties.