Hartley Neural Operator: real-valued spectral basis for PDE operators with Green's function alignment theory
Researchers introduce the Hartley Neural Operator (HNO), a real-valued alternative to Fourier Neural Operators (FNO) that replaces the complex FFT with the Discrete Hartley Transform, yielding iso-parametric models with no complex arithmetic. The central finding is that the optimal spectral basis depends on the symmetry of the underlying PDE's Green's function: HNO outperforms FNO on self-adjoint elliptic operators (Poisson, biharmonic) whose Green's functions are real and symmetric, while FNO is favored for time-dependent operators with phase content (wave, advection, Navier-Stokes). Benchmarks across PDE classes confirm a monotone elliptic-versus-time-dependent split that matches the developed theory, yielding a predictive rule for basis selection rather than a universal winner.
Related events (8)
Topological Neural Operators: operator learning on cell complexes via Discrete Exterior Calculus
Researchers introduce Topological Neural Operators (TNOs), a framework that extends neural operators from point/edge functions to general topological domains (cell complexes) using Discrete Exterior Calculus. The design decouples fixed topological information flow from learned transformations, enabling models that respect geometric structure and conservation laws. A hierarchical variant (HTNOs) adds learned coarse complexes for long-range propagation. TNOs subsume existing neural operators as a special case and show accuracy improvements on PDE benchmarks including irregular-geometry flow problems.
Functional Attention: Reinterpreting Attention as Functional Correspondences for Operator Learning
This paper introduces Functional Attention, a novel attention mechanism for operator learning that replaces standard softmax token-wise affinities with structured linear operators inspired by geometric functional maps. The approach treats attention as a correspondence between adaptive bases rather than discrete tokens, yielding a resolution-invariant, globally-aware representation. Experiments show competitive or state-of-the-art performance on PDE solving, 3D segmentation, and regression tasks, with robustness to varying discretizations.
Expressivity Limits of Congruence-Based Architectures for Neural Networks on Positive-Definite Matrices
This paper analyzes neural network architectures designed to classify symmetric positive-definite (SPD) matrices, focusing on congruence-like layers as used in SPDNet. The authors prove that imposing semi-orthogonality constraints on weight matrices limits expressivity, causing deep architectures to collapse to single-hidden-layer equivalents due to spectral diversity loss—a consequence of Poincaré's separation theorem. The work also compares Riemannian classifiers for compatibility with congruence-based feature maps.
Hamiltonian Probability Gradient Flow Analysis of the Muon Optimizer
This paper develops a rigorous theoretical framework for the Muon optimizer by interpreting its regularized orthogonalization map as the gradient of a Fenchel-dual smoothing of the nuclear norm, identifying Muon updates as mirror/prox steps with momentum as dual coordinates. The authors lift this structure to probability measures over matrix-valued parameters, deriving a mean-field phase-space equation that constitutes a damped Hamiltonian probability dynamics with monotonically decreasing Hamiltonian energy. Exponential convergence rates are established under gradient-dominance and curvature assumptions, and propagation-of-chaos guarantees are provided for the interacting particle system. The framework extends to transformer mixture-of-experts architectures via blockwise Muon probability flows.
P-K-GCN: Physics-augmented Koopman-enhanced Graph Convolutional Network for spatiotemporal super-resolution
Researchers propose P-K-GCN, a framework combining graph convolutional networks, Koopman operator theory, and physics-informed loss functions for spatiotemporal super-resolution on irregular geometries. The method linearizes nonlinear dynamics in a latent space and enforces physical constraints to improve reconstruction fidelity. Theoretical analysis claims guaranteed error reduction via Rademacher complexity bounds. The framework is evaluated on reconstructing high-resolution cardiac electrodynamics from sparse 3D heart geometry measurements.
COGENT: Continuous graph emulator with Neural ODEs for long-term physical forecasting on irregular meshes
COGENT is a new architecture combining graph neural networks with Neural Ordinary Differential Equations for continuous-time physical forecasting on irregular geospatial meshes. The model encodes historical system states and forcings into latent dynamics that can be queried at arbitrary future times, avoiding the error accumulation of autoregressive rollout. Evaluated on ice-sheet simulations from the Ice-sheet and Sea-level System Model, COGENT shows improved long-range stability over autoregressive graph baselines. The work introduces training stabilization strategies including rollout-horizon sampling and progressive scheduling.
The N Implementation Details of RLHF with PPO
This Hugging Face blog post catalogs the numerous low-level implementation details that matter when applying Reinforcement Learning from Human Feedback (RLHF) using Proximal Policy Optimization (PPO) for language model fine-tuning. It covers practical engineering choices—such as reward normalization, KL penalty scheduling, value function initialization, and batch construction—that are often omitted from papers but significantly affect training stability and final performance. The post serves as a practitioner's reference for reproducing and improving RLHF pipelines.
Theoretical analysis of truncated positional encodings for graph neural networks
A new arXiv paper initiates a formal study of truncated positional encodings (PEs) for graph neural networks, showing that truncation breaks the theoretical equivalence between spectral and walk-based PE families. Key findings include that truncated spectral PEs lose their advantage over the 1-WL expressivity test, and that k-harmonic distances differ meaningfully from other closely related truncated spectral PEs. Experiments on real-world datasets suggest mixing truncated PE families outperforms any single family.