technique

Variance-Calibrated Modulation

techniqueactiveprovisionalvariance-calibrated-modulation-36e1f6a9·1 events·first seen 40h ago

Aliases: Variance-Calibrated Modulation

Co-occurring entities

Adaptive Self-Debiasing Contextual Searchlight via PMI

More like this (12)

Feature-wise Linear Modulation multicalibration Uncertainty Calibration Expected Calibration Error Hadamard modulation model calibration per-pixel affine modulation Calibrated Surprise Variance Reduction Toward Calibrated Mixture-of-Experts Under Distribution Shift Channel-wise Vector Quantization Multimodal Gain

Recent events (1)

4arXiv · cs.CL·40h ago·source ↗

Variance-Calibrated Modulation (VCM): training-free decoding intervention to address LLM likelihood trap

Researchers propose Variance-Calibrated Modulation (VCM), a training-free pre-decoding method that reshapes LLM probability distributions before truncation to combat repetitive degeneration and vocabulary dullness. VCM combines two mechanisms: Contextual Searchlight via PMI (suppressing stopwords, elevating context-relevant tokens) and Adaptive Self-Debiasing (scale-invariant penalization using real-time logit standard deviation). Evaluated across open-ended generation, factual QA, and mathematical reasoning, VCM improves diversity, coherence, and reasoning accuracy at higher temperatures with negligible overhead. The method is compatible with existing decoding strategies like Top-p and Min-p.

Evaluation and Benchmarking Inference Economics Adaptive Self-Debiasing Variance-Calibrated Modulation Contextual Searchlight via PMI