Entity · paper

Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings

paperactiveyour-unembedding-matrix-is-secretly-a-feature-lens-for-text-embeddings-1ccbf43d·1 events·first seen Jun 8, 2026

Aliases: Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings

Co-occurring entities

EmbedFilter

More like this (12)

Latent Embedding Alignment Massive Text Embedding Benchmark Sparse Embedding Models Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects Instructional Segment Embeddings text-embedding-3-large embedding models Do LLM Embedding Spaces Recover Expert Structure?sentence embeddings Feature Auto-Encoder OpenAI Embeddings API Unstable Features, Reproducible Subspaces: Understanding Seed Dependence in Sparse Autoencoders

Recent events (1)

5arXiv · cs.CL·Jun 8, 2026·source ↗

EmbedFilter: Using the unembedding matrix to suppress high-frequency token noise in LLM text embeddings

Researchers identify that LLM text embeddings over-express high-frequency but semantically uninformative tokens when projected onto vocabulary space, degrading embedding quality. They introduce EmbedFilter, a simple linear transformation that filters out the subspace of the unembedding matrix responsible for writing these tokens into embedding space. The method improves zero-shot performance on text embedding benchmarks across multiple LLM backbones and yields a byproduct of dimensionality reduction without quality loss. Code is publicly released.

Evaluation and Benchmarking Inference Economics Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings EmbedFilter