TrajTok: Adaptive Spatial Tokenization for Trajectory Representation Learning
TrajTok is a trajectory encoder that learns transferable GPS trace representations via multi-resolution hexagonal spatial tokenization and masked-token pretraining. It uses a factorized transformer with per-modality self-attention, cross-attention fusion, and spatiotemporal rotary position embeddings (ST-RoPE) to jointly encode geometry and kinematics. A single frozen TrajTok encoder with lightweight adapters outperforms task-specific methods on trajectory similarity search, classification, ETA, and travel-time regression on the Porto dataset. The work positions learned spatial tokenization plus masked pretraining as a viable path toward general-purpose trajectory foundation models.
Related guides (2)
Related events (8)
Good Token Hunting: Token Selection Framework for Visual Geometry Transformers
This paper introduces a two-stage token selection framework to address the quadratic computational scaling of global attention in visual geometry transformers used for multi-view 3D reconstruction. The approach combines diversity-based inter-frame selection (frame-level) with entropy-guided intra-frame sparsification (token-level within frames). Experiments demonstrate over 85% acceleration for 500-image scenes while maintaining or improving baseline reconstruction quality, offering a favorable speed-accuracy trade-off.
STORM: Internalized Spatial-Temporal Reasoning for Video-Language Models via Latent Trajectories
STORMS is a two-stage training framework that teaches large vision-language models to perform spatial-temporal video reasoning through bounded continuous latent trajectories rather than explicit textual chain-of-thought, keyframe selection, or external tool use. In Stage I, latent tokens are aligned with thought-video representations derived from generated videos; in Stage II, answer-only supervision internalizes the reasoning process. At inference time, no video regeneration or frame reinsertion is required, reducing latency and engineering complexity. Evaluations on VideoMME, MVBench, TempCompass, and MMVU show improved accuracy with substantially lower inference overhead versus tool-based pipelines.
ConvexTok: Tokeniser Construction via Linear Programming and Convex Optimisation
This paper proposes ConvexTok, a new tokenisation algorithm that formulates vocabulary construction as a linear program solved via convex optimisation, replacing the greedy approaches used by BPE and Unigram. ConvexTok consistently improves intrinsic tokenisation metrics and bits-per-byte (BpB) for language models, with less consistent gains on downstream tasks. A key feature is the ability to certify proximity to optimality via a lower bound, with empirical results showing the algorithm is within 1% of optimal at common vocabulary sizes.
Apple's AToken: A Unified Multimodal Tokenizer and Encoder for Images, Videos, and 3D Objects
Apple researchers introduced AToken, a transformer model with a single 4D tokenizer and encoder-decoder architecture that handles images, videos, and 3D objects in a shared token space. The model is trained to both reconstruct and classify all three media types, using a pretrained SigLIP2 vision encoder extended to four dimensions with 4D Rotary Position Embedding. AToken approaches or matches specialized models on image classification (82.2% ImageNet), image generation (0.21 rFID), and 3D reconstruction (28.28 PSNR), while remaining competitive on video tasks. The work addresses a longstanding tension between generation-focused and classification-focused encoders by forcing embeddings to retain both fine visual detail and semantic content.
Systematic framework for selecting trajectories in data augmentation evaluated across five strategies
A thesis-derived arXiv preprint proposes a framework for evaluating five trajectory selection strategies—Outlierness, Diversity, Representativeness, Uncertainty, and Random—for data augmentation in spatio-temporal ML tasks. The study tests these strategies across four datasets spanning animal behavior, maritime, and urban traffic domains using linear and non-linear models with Optuna-based hyperparameter optimization. Key findings show systematic strategies (especially Outlierness and Uncertainty) outperform random selection in sparse datasets but can degrade performance in dense, high-quality datasets, with UMAP visualization confirming topological effects.
FedTSV: Fairness-Aware Federated Learning via Trajectory Shapley Value
This paper introduces the Trajectory Shapley Value (TSV), a contribution metric that evaluates each federated learning client's influence on the global model's optimization trajectory using validation-based, temporally consistent utility. Building on TSV, the authors propose FedTSV, an adaptive aggregation method that converts per-round evaluations into dynamic client weights to handle heterogeneous and adversarial participation. Experiments on benchmark datasets demonstrate improved convergence speed, robustness, and equitable contribution assessment compared to fixed-weight aggregation baselines.
ATLAS: Unified Agentic and Latent Visual Reasoning via Functional Tokens
ATLAS proposes a framework where a single discrete 'functional token' serves dual roles as both an agentic operation trigger and a latent visual reasoning unit in multimodal models. This design avoids the computational cost of generating intermediate images while sidestepping the context-switching latency of external tool calls and the generalization limitations of pure latent methods. The framework is compatible with standard SFT and RL training pipelines without architectural changes, and introduces Latent-Anchored GRPO (LA-GRPO) to stabilize reinforcement learning when functional tokens are sparse. Experiments show strong performance on visual reasoning benchmarks with maintained interpretability.
Test-Time Training End-to-End (TTT-E2E) Retrains Model Weights to Handle Long Inputs
Researchers from Astera Institute, Nvidia, Stanford, UC Berkeley, and UC San Diego introduced TTT-E2E, a method that compresses long context into transformer weights by training the model during inference via meta-learning. The approach uses sliding-window attention restricted to 8,000 tokens and updates only the fully connected layers of the last quarter of the network on each 1,000-token chunk at inference time, keeping per-token generation latency roughly constant as context scales to 128,000 tokens. TTT-E2E slightly outperforms vanilla transformers on next-token prediction loss across long contexts and matches efficient architectures like Mamba 2 and Gated DeltaNet on inference speed, but fails dramatically on Needle-in-a-Haystack retrieval beyond 8,000 tokens and incurs substantially higher training latency. The work reframes long-context handling as a training-inference trade-off rather than an architectural design problem.