A doctoral thesis proposes a unified framework for deep learning on manifold-valued representations, addressing limitations of existing approaches tied to specific manifolds or relying on Euclidean approximations. The work generalizes batch normalization to Lie groups and gyrogroups, extends multinomial logistic regression to SPD and general Riemannian manifolds, and introduces new architectures for hyperbolic space and correlation matrices. It also contributes adaptive, computationally efficient Riemannian metrics including learnable Log-Euclidean and Cholesky-based geometries, validated across vision, signal processing, graph learning, and genomics tasks.
A new arXiv preprint constructs and analyzes 1-Lipschitz neural networks on Hadamard manifolds, including hyperbolic spaces and the manifold of symmetric positive definite (SPD) matrices. The architecture uses Busemann functions and gradient-descent-type layers that are geometry-preserving and quasi-α-firmly nonexpansive. Experiments demonstrate robust classification on the Poincaré disk and improved covariance reconstruction on the SPD manifold compared to Log-Euclidean and data-only baselines.
EntroPath is a new manifold learning method that uses maximum entropy random walks (MERW) to build free-energy dissimilarities that converge to squared geodesic distances, addressing concentration bias in locally-normalized random walks and sensitivity to shortcut edges in shortest-path methods. The method provides a diffusion depth parameter that interpolates between local and global geometry, admits an exact Gram factorization connecting it to kernel methods, and includes scalable landmark projection extensions. Evaluations on synthetic manifolds and single-cell genomics benchmarks show EntroPath matches or outperforms diffusion- and shortest-path-based methods, with strongest gains on non-uniformly sampled manifolds and branching trajectories.
This paper analyzes neural network architectures designed to classify symmetric positive-definite (SPD) matrices, focusing on congruence-like layers as used in SPDNet. The authors prove that imposing semi-orthogonality constraints on weight matrices limits expressivity, causing deep architectures to collapse to single-hidden-layer equivalents due to spectral diversity loss—a consequence of Poincaré's separation theorem. The work also compares Riemannian classifiers for compatibility with congruence-based feature maps.
A new arXiv preprint introduces a theoretical framework for understanding how ML models trained on small inputs generalize to larger, unseen input sizes — covering sequences, graphs, point clouds, and tensors. The approach uses random sampling maps (generalizing sampling with replacement, random binning, and species sampling) to compare inputs of different sizes and derive explicit generalization and sketching rates. The framework applies to transformers, graph neural networks, and moment polynomials, among other architectures. This is a foundational theoretical contribution addressing out-of-distribution generalization across input dimensionality.
A new arXiv preprint introduces Deep Gaussian Processes (DGPs) defined over directed acyclic graphs (DAGs), providing a principled probabilistic framework for compositional function estimation in causal models, multi-fidelity simulations, and gene-regulatory networks. The authors derive theoretical results on prior-collapse behavior and information preservation as a function of graph topology, and propose a structured variational approximation that respects graph dependencies and collider explaining-away effects. Empirical validation covers a protein signalling network and a multi-fidelity heavy-ion collision emulation task, claiming state-of-the-art performance. The work is a methodological contribution at the intersection of probabilistic ML and structured graphical models.
Researchers introduce VLM-IE3D, a framework that enhances VLMs with 3D spatial understanding by injecting both Implicit Geometry Tokens (IGTs) and Explicit Geometry Tokens (EGTs) derived from RGB video inputs alone. A 3D-aware adapter fuses these geometric representations with 2D visual features, requiring no depth sensors or additional 3D inputs. The system is evaluated across 3D video detection, visual grounding, dense captioning, and spatial reasoning tasks, reporting consistent performance gains. Code and models are publicly released.
This paper proposes the 'matching principle': a unified geometric framework arguing that robustness methods (CORAL, IRM, adversarial training, augmentation, metric learning, Jacobian penalties, alignment constraints) are all estimators of the same object—the covariance of label-preserving deployment nuisance—and that regularizing the encoder Jacobian along this covariance's range is the core statistical problem. The authors prove closed-form optimality results in a linear-Gaussian model, introduce the Trajectory Deviation Index (TDI) as a label-free embedding sensitivity probe, and validate predictions across 13 pre-registered experimental blocks including Qwen2.5-7B. At 7B scale, matched style-PMH improves selective honesty while standard DPO degrades Style TDI, connecting the theory to alignment safety.
A new arXiv preprint presents a theoretical framework explaining how Transformers develop inductive reasoning abilities by proving that training dynamics of attention models are confined to a low-dimensional invariant manifold. The framework unifies several synthetic inductive tasks (in-context n-grams, multi-hop reasoning) and characterizes how data statistics govern competition between in-context and in-weights learning. The authors show that random initializations determine which circuit 'wins' when multiple solutions exist, and that the manifold's coordinate frame can automatically detect learned circuits in trained models. The work advances mechanistic interpretability by casting circuit formation as a tractable low-dimensional dynamical phenomenon.