LongBench-Pro

Self-Guided Test-Time Training improves long-context LLM utilization by up to 15%

Researchers propose Self-Guided Test-Time Training (S-TTT), a method that addresses the degradation in accuracy LLMs exhibit on long inputs by having the model first identify relevant evidence spans before applying test-time training only to those spans. A preliminary study on LongBench-v2 shows that TTT on randomly sampled spans hurts performance while TTT on oracle spans substantially helps, motivating the self-guided selection approach. Evaluated on LongBench-v2 and LongBench-Pro with Qwen3-4B-Thinking-2507 and Llama-3.1-8B-Instruct, S-TTT achieves up to 15% relative accuracy improvement. The method offers a practical path to better long-context utilization without the prohibitive cost of full-context adaptation.