Entity · paper

Linear representations of grammaticality in neural language models

paperactiveprovisionallinear-representations-of-grammaticality-in-neural-language-models-d7e763f8·1 events·first seen 4d ago

Aliases: Linear representations of grammaticality in neural language models

Co-occurring entities

mass-mean probing

More like this (12)

Scaling Laws for Neural Language Models Transformer Language Models Verbalizable Representations Form a Global Workspace in Language Models Reasoning Language Models encoder-only language models Language Modeling Loss Neural Collapse Is Forbidden: Information Floors in Language Models Understanding Truncated Positional Encodings for Graph Neural Networks multi-turn language models Random Language Model Recursive Language Models (RLMs)linear representation hypothesis

Recent events (1)

4arXiv · cs.CL·4d ago·source ↗

Linear representations of grammaticality found in neural language model internals

A new arXiv preprint investigates whether grammaticality is encoded in the internal representations of neural language models, moving beyond probability-based evaluations. Using mass-mean probing, the authors find that grammatical and ungrammatical sentences are robustly separated in representational space across a wide range of pretrained models, grammatical phenomena, and languages. The encoding is not fully explained by confounds like lexical frequency or plausibility, suggesting grammaticality constitutes a coherent representational dimension. The findings contribute to debates about syntactic knowledge in LLMs and offer a probability-independent framework for evaluating grammatical competence.

Evaluation and Benchmarking Linear representations of grammaticality in neural language models mass-mean probing