Redwood Research Shows Small LLMs Can Compose Synthetic Facts If Bridge Entity Is Real

Researcher GabrielKS from Redwood Research has published experimental results showing that small large language models, specifically Qwen3.6-35B-A3B and Qwen3-8B, can perform two-hop reasoning without chain-of-thought (no-CoT) on propositions injected via fine-tuning. Previous studies suggested that models fail to connect two synthetic propositions in latent space.

The new findings indicate that this limitation only applies when the 'bridge' entity (the middle term connecting the two hops) is also fabricated or extremely obscure. If the bridge entity is real—such as a well-known country or city—the models successfully compose the information and reach correct conclusions, even if the first and last entities and the specific propositional content are entirely synthetic. Furthermore, when the bridge entity is real and famous, the source of the other entities (real vs. synthetic) has negligible impact on reasoning accuracy.

These results are significant for understanding model monitorability and designing alignment evaluations based on belief instillation. They suggest that beliefs implanted through synthetic document fine-tuning (SDF) or supervised fine-tuning (SFT) can participate in latent reasoning similarly to pretraining knowledge, provided they are anchored to real-world entities. Code is open-source, but datasets and model weights are available upon request.