paper
Optimization Dynamics Imprint Semantic Specificity in Contrastive Embedding Norms
paperactiveprovisional
optimization-dynamics-imprint-semantic-specificity-in-contrastive-embedding-norms-d2270cf5·1 events·first seen 15h agoAliases: Optimization Dynamics Imprint Semantic Specificity in Contrastive Embedding Norms
More like this (12)
contrastive semantic alignmentDecomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustnessdistributionally robust optimizationContrastive-Difference CKA Reveals Concept-Specific Structural Alignment Across Language Model ArchitecturesSemantic Edit Regularizationinstruction embedding modelsSupervised Semantic DifferentialGeometry of Semantic Space: Comparative Study of Discrete and Continuous ModelsInstructional Segment EmbeddingsDo LLM Embedding Spaces Recover Expert Structure?Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detectionembedding models
Recent events (1)
Theoretical framework explains why contrastive embedding norms encode semantic specificity
A new arXiv preprint provides a formal theoretical explanation for why embedding magnitudes in contrastive models trained with scale-invariant losses correlate with semantic properties like concept specificity, token frequency, and human uncertainty — despite norms being ignored by cosine similarity metrics. The authors derive an analytic formula showing that embedding length encodes this information as a byproduct of optimization dynamics. The work suggests these norms can serve as 'free' calibration signals in retrieval tasks, grounding a previously heuristic observation.