New Study Diagnoses 'Right Values, Wrong Places' Error in LLM-Generated JSON
Researchers identify record misplacement as a dominant error mode in LLM JSON generation and propose input-output alignment fixes that achieve exact matches on tested models.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
Large language models frequently generate valid JSON where field values are correct but bound to the wrong records or hierarchy levels. This 'right values, wrong places' error mode poses significant risks to downstream data pipelines and API integrations.
The study introduces JSONPlacement, a controlled task to diagnose these errors when transforming flat records into hierarchical JSON. Analysis across six open-weight models reveals that whole-record moves account for 98.6% of strictly traceable synthetic misplacements.
The authors demonstrate that aligning input organization with target traversal reduces assignment errors, while replacing nested output with record-level paths reduces hierarchy errors. Combined, these interventions enabled three 8-12B parameter models to achieve exact matches on all new synthetic instances and a real-data validation set.
These findings are based on first-party benchmarks (E2) and await independent reproduction. The research advocates for explicit evaluation of record placement and joint design of input/output representations to preserve value-path bindings.