LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
← Volver a la lectura
NoticiasTema · 选项名称语义干扰Material1 min de lecturaEditorial de DisconfirmAI

Type Safety Does Not Ensure Logical Correctness

New preprint reveals LLMs often follow option name semantics over definitions, causing high decision flip rates.

2026-09-23 12:00 GMT+8

Even when outputs strictly adhere to type definitions, large language models may deviate from intended logic due to semantic cues in option names.

A new preprint by Yu Sun et al. finds that changing binary decision option names from "0/1" to "yes/no" increases decision-flip rates by up to 70.4 percentage points. Despite a 0% type-error rate, models tend to follow the polarity of the name rather than the bound definition text.

The study tested Jev and two open-weight models on 1200 tasks. Results show that random strings as names perform similarly to neutral controls, indicating that semantically loaded names like "yes/no" introduce instability beyond simple reassignment.

These are author-reported results without independent reproduction yet. The finding impacts prompt engineering and evaluation frameworks, highlighting risks of logical misalignment masked by structural compliance.

Fuentes

  • arxiv.org↗
Tema

选项名称语义干扰

【背景】大语言模型在结构化决策任务中常出现“类型安全但逻辑错误”的现象,其核心原因在于模型过度依赖选项名称的语义线索,而非绑定的定义。

2 pasos1 piezasActualizado 2026-10-09 13:25

Cómo llegó hasta aquí

Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión
  1. 2026-09-22Yu Sun等人提交预印本v1,提出类型安全的决策模型仍受选项名称语义干扰。arxiv.org ↗
  2. 2026-10-08预印本更新至v3,确认在1200个任务中,“yes/no”名称相比“0/1”控制组导致最高70.4个百分点的决策翻转率增加。arxiv.org ↗

Lo que aún está pendiente

  • 如何在不牺牲可读性的前提下,设计对模型无偏见的选项命名规范?
  • 结构化分解(LLM提取+规则决策)是否能完全消除名称语义带来的AUC下降?
  • 该现象在多步推理或复杂状态空间中的表现是否与二元决策一致?
Leer el tema completo →
Continuar investigando →