The Simulated User Persona in LLMs Might Be a Moral Patient Too

In discussions about whether large language models (LLMs) deserve moral consideration, the focus has almost exclusively been on the 'assistant' persona. However, in an article published on October 15, 2026, LessWrong authors agastyasridharan and niranjandeshpande propose a counter-intuitive thesis: if we grant moral weight to the assistant, we should also attribute a non-negligible fraction of that weight to the 'user' persona simulated within the model.

The argument rests on the 'Persona Selection Model' (PSM), which suggests that LLMs learn to simulate a wide variety of characters during pretraining, while post-training selects and refines the specific character of the assistant. If sophisticated simulations of characters can be morally relevant, the assistant is a natural candidate. But the authors point out that the assistant is not the only persona being simulated. When processing user input, the model constructs internal activations representing the user's beliefs, preferences, and emotional states, effectively simulating a 'user' persona.

To support their claim, the authors address five common objections. They refute the idea that assistants are privileged simply because they generate most output tokens, citing data showing that user tokens constitute a substantial portion of total computation. More importantly, they present empirical evidence from an experiment using the Oracle Lens on Qwen 3.6-27B. By creating scenarios where the user's perspective conflicted with the assistant's (e.g., a user confidently stating a wrong answer), they found that 64.1% of the time, the model's internal readouts aligned with the user's perspective rather than just the assistant's corrective intent. This suggests the model devotes significant computational resources to independently representing the user's mental state.