model

1B-scale language models

modelactive1b-scale-language-models-0bafaf60·1 events·first seen 1mo ago

Aliases: 1B-scale language models

Co-occurring entities

aphasia lesioning technique Text Aphasia Battery (TAB)

More like this (12)

8B autoregressive language model 7B language model large language models large language model agents 32B language model Transformer Language Models Scaling Laws for Neural Language Models Character-level Language Model Large Language Models (frontier)protein language models Diffusion Language Models AnyLanguageModel

Recent events (1)

5arXiv · cs.LG·1mo ago·source ↗

Artificial Aphasias in Lesioned Language Models

Researchers apply an aphasia-inspired 'lesioning' technique to five 1B-scale language models by zeroing out model parameters and measuring resulting language impairments against a Text Aphasia Battery (TAB). Across 112,426 outputs, the full range of aphasia symptoms emerges but in distributions distinct from human aphasia profiles. The study finds systematic differences between attention components (query, key, value, output) and feed-forward components, as well as depth-dependent effects where early-layer lesions cause syntactic/semantic symptoms and late-middle layers yield phonological and fluency deficits. The qualitative divergence between LM and human aphasia patterns suggests aphasia syndromes are shaped by learning and processing details rather than being universal consequences of disrupted language processing.

Evaluation and Benchmarking aphasia 1B-scale language models lesioning technique +1 more