jepa-6f0c0610·2 events·first seen Aliases: JEPA
A new arXiv preprint reframes the LLM-vs-world-model debate by arguing that LLMs are a degenerate special case of world models rather than a fundamentally different paradigm, with the state space being token sequences and the only action being token appending. The paper maps a continuous spectrum from next-token prediction through multi-token prediction, future-summary prediction, and next-latent prediction up to JEPA-style architectures. It identifies two open research challenges in moving along this spectrum: the data cliff from self-supervised text to action-labeled environments, and whether transformers generalize to continuous-state prediction or require a new architectural primitive. The work directly engages with Yann LeCun's 2022 argument that general intelligence requires abandoning autoregressive prediction.
This paper critiques the standard practice of regularizing Joint-Embedding Predictive Architecture (JEPA) encoders toward isotropic Gaussian marginals, showing that this Euclidean symmetry assumption incurs a quantifiable 'price of isotropy' and that no geometry-independent fixed marginal target is universally canonical. The authors prove that oracle one-view marginals do not identify the view-to-view predictive coupling, arguing structural bias should enter the cross-view coupling instead. They introduce HamJEPA, which encodes views as phase-space states and uses a learned Hamiltonian leapfrog map for view-to-view prediction, with symplectic coupling identified as the key driver of gains. HamJEPA outperforms SIGReg on CIFAR-100 by up to +6.45 kNN@20 and +10.64 linear-probe points at 80 epochs, with similar improvements on ImageNet-100.