4arXiv cs.LG (Machine Learning)·
TemporalSinkhorn: Parallel-in-Time Certified Sinkhorn for Dynamic Entropic Optimal Transport
A new arXiv preprint introduces TemporalSinkhorn, a parallel-in-time algorithm for solving dynamic entropic optimal transport problems with certified correctness guarantees. The method batches future Sinkhorn candidates and uses a deterministic safe-prefix certificate to ensure no inaccurate outputs are authorized, achieving 1.42x–3.55x speedups over sequential baselines on synthetic streams and 3.05x–3.63x speedups on Flow Matching minibatch streams on 4 A100 GPUs. The work is directly relevant to optimal-transport Flow Matching, a technique used in generative model training pipelines.
Related guides (2)
Related events (8)
Minimax framework establishes fundamental statistical limits for transport map estimation in generative models
A new arXiv preprint formalizes the task of estimating valid transport maps (used in diffusion models, normalizing flows, and flow matching) within a minimax statistical framework. The key result is that under standard stability assumptions from optimal transport theory, estimating any valid transport map is as statistically hard as estimating the optimal transport map itself. The authors also show that when those stability assumptions fail, alternative transport maps can be learned substantially more accurately than the OT map, clarifying when targeting sub-optimal maps yields real statistical advantages. The work provides theoretical sample complexity lower bounds applicable to a broad class of modern generative modeling methods.
Optimal Mixture Transport (OMT): Biconvex Formulation for Scalable, Stable Optimal Transport
This paper introduces Optimal Mixture Transport (OMT), a framework that reformulates optimal transport between probability distributions as a strictly biconvex optimization problem with a provably unique global minimizer. By operating at the level of mixture components (modeled as exponential-family distributions) rather than individual samples, OMT decouples computational complexity from sample size. The authors provide theoretical stability guarantees showing bounded perturbations yield bounded changes in transport plans, and validate the approach on image data and large-scale single-cell RNA sequencing datasets.
Asynchronous pipeline parallelism for LLM pretraining made viable with Muon optimizer and error feedback correction
A new arXiv paper challenges the assumption that gradient staleness in asynchronous pipeline parallelism (specifically PipeDream-2BW) is fundamentally unstable, showing the degradation is optimizer-dependent rather than intrinsic. The authors demonstrate that the Muon optimizer is robust under one-step gradient delay where AdamW fails, and introduce an optimizer-agnostic Error Feedback correction to further close the gap with synchronous training. Experiments on models up to 10B parameters confirm the approach matches synchronous training performance, potentially unlocking higher GPU utilization by eliminating pipeline bubbles.
IMPFM: Multi-particle flow-map framework for online feedback-driven generative search
A new arXiv preprint introduces Sequentially-Controlled Interactive Multi-Particle Flow-Maps (IMPFM), a framework for sample-efficient online search under sequential human feedback. The method transports an ensemble of particles toward a target distribution using flow maps and a posterior sample-sharing mechanism, aiming to balance global exploration with reward over-optimization avoidance. The authors prove convergence to a KL-tilted target via a Feynman-Kac corrector and demonstrate improvements over baselines on alignment and search tasks. The work addresses a gap in training-free reward alignment methods that struggle with unknown or sequentially revealed preferences.
Express: Efficient causal attention approximation with formal guarantees and FlashAttention 2 speedups
A new tool called Express converts non-causal attention approximations into causal ones with matching theoretical guarantees, achieving log^(3/2)(n)/s approximation error with O(s) memory. Combined with the Thinformer approximation and an I/O-aware Triton implementation, it demonstrates substantial speedups over FlashAttention 2. The work targets four practical bottlenecks: long-context prefill, KV cache compression, and both memory- and compute-constrained long-form decoding.
Signed-Permutation Gauge Theory for RMSNorm Transformers Improves Coordinate Transport
A new arXiv preprint formalizes the residual-stream gauge symmetry of transformer architectures, showing that RMSNorm models have a signed-permutation gauge group B_d = S_d ⋉ {±1}^d rather than the permutation-only S_d of LayerNorm models. The authors introduce sign-marginalized Hungarian matching and demonstrate that coordinate-preserving transport along fine-tuning trajectories recovers 91.1% of cross-run coordinates versus 60.3% for endpoint matching. Practical consequences include dramatically improved sparse autoencoder reconstruction (NMSE 0.004 vs 1.08), preserved steering vector effects, and correct AdamW optimizer state transfer — with implications for mechanistic interpretability, model merging, and activation engineering.
FlowPipe: LLM-conditioned Generative Flow Networks for automated data preparation pipeline construction
FlowPipe is a new framework that frames ML data preparation pipeline synthesis as conditional probabilistic flow generation over a directed acyclic graph, using Conditional Generative Flow Networks (C-GFlowNets) with a Trajectory Balance objective. LLM-derived semantic priors are injected into the policy via Feature-wise Linear Modulation (FiLM), and a failure-aware flow objective steers search away from invalid states. Evaluated on 74 real-world datasets across two benchmark suites, FlowPipe improves accuracy by 11.96% on average over SOTA baselines and achieves 12.5x faster training convergence. The work addresses long-standing limitations in automated data pipeline construction including weak credit assignment and inefficient exploration.
LatentFlow: Training-free framework for conditioning stochastic processes via latent-space inference
LatentFlow is a new general framework for conditioning stochastic processes without learned neural approximations or training. The approach maps a stochastic process to a tractable latent innovation space, reduces conditioning to latent-space inference via a guided probability flow, and pushes samples forward — yielding provably exact conditioning at the level of the target law. The method is claimed to work across a broad range of model classes (spatial priors, stochastic PDEs, mechanistic models, point processes) and runs in seconds on a single CPU.