t-mlr-12bcf4e8·1 events·first seen Aliases: T²MLR
Researchers introduce T²MLR, a latent reasoning architecture that injects a cached middle-layer representation from the previous token into an earlier layer of the current token, allowing intermediate reasoning states to persist across autoregressive decoding steps. The approach consistently outperforms parameter-matched Transformer baselines on natural-language pretraining and multi-hop reasoning, with recurrence applied to only ~20% of the network often beating full-layer recurrence. Notably, T²MLR can be retrofitted into an existing pretrained 1.7B Transformer with brief finetuning, substantially improving math reasoning without pretraining from scratch. The work challenges prior assumptions that effective latent recurrence requires looping over all layers.