Entity · other

permutation-invariant transformers

otheractiveprovisionalpermutation-invariant-transformers-88c608b6·1 events·first seen 2d ago

Aliases: permutation-invariant transformers

Co-occurring entities

Any-Dimensional Learning by Sampling Graph Neural Networks

More like this (12)

transformer architecture Signed-Permutation Coordinate Transport for RMSNorm Transformers Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers Dynamic Short Convolutions Improve Transformers Training-Free Looped Transformers Variable-Width Transformers First-Token Broadcasters: Mechanistic Origins of Language Identity and Distributed Robustness in Transformers Sentence Transformers Sparse Transformer transformer-based neural renderer Transformers (library)autoregressive transformer

Recent events (1)

5arXiv · cs.LG·2d ago·source ↗

Unified framework for generalization and sketching across variable-size inputs via random sampling maps

A new arXiv preprint introduces a theoretical framework for understanding how ML models trained on small inputs generalize to larger, unseen input sizes — covering sequences, graphs, point clouds, and tensors. The approach uses random sampling maps (generalizing sampling with replacement, random binning, and species sampling) to compare inputs of different sizes and derive explicit generalization and sketching rates. The framework applies to transformers, graph neural networks, and moment polynomials, among other architectures. This is a foundational theoretical contribution addressing out-of-distribution generalization across input dimensionality.

Evaluation and Benchmarking permutation-invariant transformers Any-Dimensional Learning by Sampling Graph Neural Networks