test-time-training-f5ac4944·3 events·first seen Aliases: test-time training
A new arXiv preprint proposes a neural memory architecture for real-time novel view synthesis from multi-view streaming video of dynamic scenes. The method decouples memory update frequency from memory application frequency, using periodic gradient-based updates with per-frame cross-view attention to handle deformations. Two mechanisms — an auxiliary Memory Loss and a Memory Caching strategy — prevent catastrophic forgetting over long contexts. The approach achieves state-of-the-art performance on dynamic human motion scenes with minute-scale online memorization at real-time speeds.
NVIDIA researchers introduce RoboTTT, a robot foundation model training recipe that extends visuomotor context to 8,000 timesteps — three orders of magnitude beyond current state-of-the-art — without increasing inference latency. The approach integrates Test-Time Training into Vision-Language-Action policies, using fast weights (parameters updated by gradient descent during inference) to compress long histories into weight space. On real-robot manipulation tasks, RoboTTT achieves 87% performance improvement over single-step baselines and is the first system to fully complete a five-minute, ten-stage assembly task. The work identifies context length as a new scaling axis for robot foundation models, with 8K-context pretraining outperforming 1K-context by 62%.
SIA proposes a self-improving loop in which a Feedback-Agent simultaneously updates both the scaffold (harness) and model weights of a task-specific agent, unifying two previously disjoint research lines: meta-agent scaffold rewriting and test-time training. The system is evaluated on three diverse benchmarks—Chinese legal charge classification, GPU kernel optimization, and single-cell RNA denoising—achieving gains of 56.6%, 91.9% runtime reduction, and 502% respectively over baselines. The paper argues that harness updates shape agentic behavior while weight updates instill domain intuition that prompting alone cannot provide, and that combining both levers consistently outperforms either alone.