LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
← Volver a la lectura
NoticiasTema · AI合成受访者Resumen rápido1 min de lecturaEditorial de DisconfirmAI

Korean study finds AI synthetic respondents cannot yet replace real surveys

Peer-reviewed testing put synthetic respondent error at 14-19 percentage points; the panels are diagnostic tools, not survey substitutes.

2026-10-02 12:00 GMT+8

A Korean research team has published a validation in IEEE Access: using LLM personas to simulate survey respondents produces errors too large to replace real questionnaires.

The study had Gemini 3.5 Flash and EXAONE each generate about 8,000 Korean personas, answer eight digital and AI service-use items, and compared the responses with weighted estimates from the Korea Media Panel Survey. Overall mean absolute error was 14-19 percentage points, and segment error across five axes was 14-18 percentage points, with the two models showing different bias patterns.

Even after calibrating on 30% of the real data, which cut error to about 5 percentage points, direct estimation from that same real subsample was more accurate (3.6 percentage points). The authors conclude that synthetic panels are diagnostic tools, not survey substitutes.

Fuentes

  • arxiv.org↗
  • ieeexplore.ieee.org↗
Tema
Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión

AI合成受访者

AI合成受访者能否替代真人调查?最新研究划定了明确边界:在多数通用大模型场景下,其误差高达14至19个百分点,仅具诊断价值;但在特定优化框架或专用平台中,准确率可显著提升。争议核心在于“像人”不等于“可替代”,关键在于区分预测试、方向性比较…

6 pasos2 piezasActualizado 2026-10-08 09:16

Cómo llegó hasta aquí

  1. 2026-07-23韩国研究提交arXiv预印本,评估基于Gemini和EXAONE的合成面板,发现平均绝对误差达14-19个百分点,结论为合成面板不能替代调查,仅具诊断价值。arxiv.org ↗
  2. 2026-07-28《When Synthetic Users Fail》论文提交,建立跨域基准,发现LLM无法超越非LLM基线,且会过度决定人口统计特征,导致细分目标错误。arxiv.org ↗
  3. 2026-09-19问卷派发布指南,提出将“像人”与“可替代”分开验证,定义三层风险用途及六步对照校验协议,强调子群方向反转是常见失败模式。wj.pro ↗

Lo que aún está pendiente

  • 专用合成框架(如Artificial Societies)的低误差结果是否能在其他独立研究中复现?
  • 对于小语种或低代表性人群,合成数据的偏差结构是否与英语/高收入国家一致?
  • 行业是否会形成统一的合成数据披露标准,以区分“假设生成”与“决策支持”?

También en este hilo

  • LLMs Fail as Synthetic Survey Users2026-10-06
Leer el tema completo →
Continuar investigando →