Entity · technique

Manifold Power Iteration

techniqueactivemanifold-power-iteration-e2d33f6a·1 events·first seen Jun 11, 2026

Aliases: Manifold Power Iteration

Co-occurring entities

Redesign Mixture-of-Experts Routers with Manifold Power Iteration

More like this (12)

Manifold Redesign Mixture-of-Experts Routers with Manifold Power Iteration Manifold Bio Iterated Amplification G*Power power-law scaling Complexity Bounds and Approaches to Learning Projected Gradient Descent Solver Iterates Expanding Generative Flows Iterative Skill-Aware Decomposition Maximal Update Parameterization (μP)Sequentially-Controlled Interactive Multi-Particle Flow-Maps Layer Looping

Recent events (1)

5arXiv · cs.CL·Jun 11, 2026·source ↗

Manifold Power Iteration redesigns MoE routers by aligning rows with expert singular directions

A new arXiv preprint proposes Manifold Power Iteration (MPI), a principled redesign of Mixture-of-Experts router matrices that aligns each router row with the principal singular direction of its associated expert. The method uses a 'Power-then-Retract' paradigm to enforce norm constraints while driving convergence toward these singular directions. Empirical validation spans MoE pretraining at scales from 1B to 11B parameters, showing improved model effectiveness.

Training Infrastructure Frontier Model Releases Redesign Mixture-of-Experts Routers with Manifold Power Iteration Manifold Power Iteration