technique

congruence-based layers

techniqueactiveprovisionalcongruence-based-layers-86ae7910·1 events·first seen 15d ago

Aliases: congruence-based layers

Co-occurring entities

SPDNet Poincaré separation theorem symmetric positive-definite matrices Riemannian classifiers

More like this (12)

selective-verification layer Tying the Loop -- Tied Expert Layers in Mixture-of-Experts Language Models Cross-Layer Sparse Attention layer pruning Dual Layer Aggregation (DLA)Reversible Residual Layers multilayer perceptron (MLP)Mondrian Conformal Prediction Layer-Adaptive Expert Pruning Conformal Prediction superposition Confluent

Recent events (1)

4arXiv · cs.LG·15d ago·source ↗

Expressivity Limits of Congruence-Based Architectures for Neural Networks on Positive-Definite Matrices

This paper analyzes neural network architectures designed to classify symmetric positive-definite (SPD) matrices, focusing on congruence-like layers as used in SPDNet. The authors prove that imposing semi-orthogonality constraints on weight matrices limits expressivity, causing deep architectures to collapse to single-hidden-layer equivalents due to spectral diversity loss—a consequence of Poincaré's separation theorem. The work also compares Riemannian classifiers for compatibility with congruence-based feature maps.

Evaluation and Benchmarking congruence-based layers SPDNet Poincaré separation theorem +2 more