A new arXiv preprint proposes OT-ICA, an algorithm for linear Independent Component Analysis that replaces classical proxy contrast functions (cumulants, parametric likelihoods) with the squared Wasserstein-2 distance to a standard Gaussian as a non-Gaussianity measure. The authors prove that maximizing this distance over linear projections recovers independent components, then implement gradient-based optimization. Empirical results on simulated data show OT-ICA outperforms proxy-based methods, with applied validation on EEG artifact removal and econometric price discovery.
This paper introduces Optimal Mixture Transport (OMT), a framework that reformulates optimal transport between probability distributions as a strictly biconvex optimization problem with a provably unique global minimizer. By operating at the level of mixture components (modeled as exponential-family distributions) rather than individual samples, OMT decouples computational complexity from sample size. The authors provide theoretical stability guarantees showing bounded perturbations yield bounded changes in transport plans, and validate the approach on image data and large-scale single-cell RNA sequencing datasets.
KLIP proposes an out-of-distribution detection metric for computational imaging that computes KL-divergence between a diffusion model prior and the posterior distribution. Unlike prior approaches, it requires no calibration data or knowledge of the shifted distribution, and can both flag whole images and localize OOD patches within images. The method is validated on medical imaging tasks such as detecting liver tumors in CT scans and generalizes across diffusion model architectures, datasets, and inverse problem types.
Researchers from Berkeley present a framework for evaluating and optimizing imaging systems based on mutual information content rather than traditional metrics like resolution or SNR, published at NeurIPS 2025. The method estimates mutual information directly from noisy measurements using known noise physics and learned probabilistic models (including transformers and PixelCNN), avoiding the need for task-specific decoders. Validated across four domains—color photography, radio astronomy, lensless imaging, and microscopy—the information metric predicts downstream decoder performance and enables hardware optimization with less compute and memory than end-to-end neural approaches.
A new arXiv preprint introduces a class of estimators for the Sliced-Wasserstein (SW) distance that use cumulative distribution functions (CDFs) of projected measures rather than sorting-based quantile functions. The estimators avoid sorting, scale via massive dataset parallelism, and are naturally compatible with federated learning since local CDFs can be aggregated without sharing raw samples. The work also shows advantages for mixture-of-Gaussians settings where CDFs are more tractable than quantile functions.
This paper introduces Orthogonal Residual Projection (ORP), an algorithm-hardware co-design framework for ultra-low-bit quantization of LLMs and Vision Transformers targeting edge deployment. ORP addresses the structural limitations of Power-of-Two (PoT) quantization by formulating quantization as a dual-basis geometric projection that synthesizes higher-resolution residual lattices using only shift-and-add operations, eliminating multipliers. At 3-bit (W3/A16), ORP achieves 6.10 perplexity on LLaMA-2-7B, competitive with MAC-intensive baselines like AWQ, while reducing full-model calibration time to ~15 minutes. RTL synthesis at 28nm confirms hardware efficiency by mitigating timing bottlenecks from dense multiplier trees.
This paper proposes the 'matching principle': a unified geometric framework arguing that robustness methods (CORAL, IRM, adversarial training, augmentation, metric learning, Jacobian penalties, alignment constraints) are all estimators of the same object—the covariance of label-preserving deployment nuisance—and that regularizing the encoder Jacobian along this covariance's range is the core statistical problem. The authors prove closed-form optimality results in a linear-Gaussian model, introduce the Trajectory Deviation Index (TDI) as a label-free embedding sensitivity probe, and validate predictions across 13 pre-registered experimental blocks including Qwen2.5-7B. At 7B scale, matched style-PMH improves selective honesty while standard DPO degrades Style TDI, connecting the theory to alignment safety.
A new arXiv preprint provides a rigorous theoretical framework for understanding what discrete diffusion models learn, proving the 'Oracle Distance' theorem: the negative ELBO exactly equals data entropy plus the path KL from the oracle reverse process to the learned one. The work shows that denoiser, score ratio, and bridge plug-in parameterizations are the same object in different coordinates, with closed-form conversions among them. It unifies several existing discrete diffusion losses (MDM, UDM, SEDD, GIDD) as special cases and identifies practical consequences, such as why denoiser parameterization causes the uniform ELBO to diverge at initialization. All identities are verified numerically on an exactly solvable model.
A new arXiv preprint introduces Logit-Contribution Scoring (LOCOS), a method for identifying attention heads responsible for non-literal retrieval in long-context LLMs — cases where models synthesize answers from meaning rather than copying tokens verbatim. Existing detectors fail at this task because they rely on a literal-copy criterion that misses the output-value (OV) circuit mechanism. Evaluated across Qwen3, Gemma-3, and OLMo-3.1, LOCOS outperforms prior attention-based detectors on the NoLiMa benchmark, with ablation of 50 heads on Qwen3-8B collapsing ROUGE-L from 0.401 to 0.000 while the best baseline retains 0.292. The identified heads are retrieval-specific, leaving parametric recall and arithmetic reasoning unaffected.