technique
Positional Encoding
techniqueactive
positional-encoding-a2de4c6d·1 events·first seen 28d agoAliases: Positional Encoding
Co-occurring entities
More like this (12)
Perception EncoderRotary Position Embedding (RoPE)Surface-Anchored Position Embeddingstatic embeddingssentence embeddingsUnderstanding Truncated Positional Encodings for Graph Neural Networksvideo encodingRandom Codingembedding modelsPerception Encoder AudiovisualStratified Out-of-Fold Teacher LabelingPositive Alignment
Recent events (1)
You Could Have Designed State of the Art Positional Encoding
A Hugging Face blog post walks through the design space of positional encoding for transformer models, building intuition for why modern schemes like RoPE emerged. The post takes a pedagogical approach, showing how one could derive state-of-the-art positional encoding from first principles. It covers the evolution from absolute to relative positional encodings and the properties that make certain schemes preferable for long-context generalization.