Vals AI Study: Chinese LLMs Not Uniformly Pro-Government; Prompt Language Significantly Shifts Political Stance

A new study by AI benchmark firm Vals AI indicates that Chinese artificial intelligence models behave in a more complex manner regarding politically sensitive questions than previously assumed, challenging the notion that they act merely as "government mouthpieces." Researchers posed hundreds of questions about China to various language models, including those from the US and China, while varying the prompt language, usage context, and response format.

The findings show that the language of the prompt has a strong effect on the political slant of responses. For most tested models, answers generated in Simplified Chinese were more pro-government than those in English, Russian, or Spanish. This effect was present in both Chinese and American models but was far more pronounced in the Chinese ones. Vals AI also identified a smaller but consistent gap between Simplified Chinese (used in mainland China) and Traditional Chinese (common in Taiwan and Hong Kong).

Furthermore, system prompt context altered model behavior. When the system prompt indicated the model was being used internally by a company in mainland China, some models responded in a more pro-government way compared to when the prompt suggested US-based usage. Notably, Western models occasionally adopted official Chinese government positions, while certain Chinese models, such as GLM-5.3 Flash, explicitly acknowledged documented human rights violations when asked.

The study notes that the common assumption—that Chinese models primarily block sensitive questions—only held true for certain DeepSeek models. Most other Chinese models rarely refused to answer, instead sometimes adopting official government stances. For instance, Kimi K3 denied the existence of concentration camps in Xinjiang, referring to them as "vocational training centers" designed to combat extremism.