5arXiv cs.CL (Computation and Language)·
Lesioned multimodal LLM reproduces individual aphasic naming error profiles at high fidelity
Researchers applied controlled perturbations (layer-targeted noise) to LLaVA 1.6 and found that the resulting error distributions in picture-naming tasks closely match those of 278 persons with post-stroke aphasia on the Philadelphia Naming Test. Six of seven clinical error categories emerged at clinically comparable proportions, and perturbation configurations reproducing individual patient error profiles were found for 97.8% of participants across six categories and 79.5% across all seven. Monte Carlo baselines confirm the matching reflects genuine joint inter-category structure rather than chance. The work proposes language models as 'digital twins' for aphasia simulation and opens a quantitative framework for clinical cognitive modeling.
Related guides (2)
Related events (8)
Artificial Aphasias in Lesioned Language Models
Researchers apply an aphasia-inspired 'lesioning' technique to five 1B-scale language models by zeroing out model parameters and measuring resulting language impairments against a Text Aphasia Battery (TAB). Across 112,426 outputs, the full range of aphasia symptoms emerges but in distributions distinct from human aphasia profiles. The study finds systematic differences between attention components (query, key, value, output) and feed-forward components, as well as depth-dependent effects where early-layer lesions cause syntactic/semantic symptoms and late-middle layers yield phonological and fluency deficits. The qualitative divergence between LM and human aphasia patterns suggests aphasia syndromes are shaped by learning and processing details rather than being universal consequences of disrupted language processing.
Language-based digital twins using LLMs proposed for elderly cognitive health monitoring
Researchers propose a framework that uses large language models to construct digital twins of elderly individuals by mimicking their conversational patterns and stylometric cues, enabling continuous, non-invasive monitoring for Mild Cognitive Impairment. A multi-head conditional variational autoencoder (cVAE) is introduced to evaluate fidelity and predict cognitive scores (MoCA). Experiments on the I-CONECT dataset show the approach preserves individual identity characteristics and outperforms baseline GPT-generated responses on reconstruction and cognitive score prediction. The work positions language-based digital twins as a scalable alternative to clinical cognitive assessment.
VLMs May Not Globally Enhance Human Alignment over LLMs During Natural Reading
This paper compares matched LLM and VLM pairs in a text-only setting to isolate the effect of multimodal training history on human-like language processing. Using whole-cortex fMRI and eye-tracking data from natural reading, the authors find that multimodal pretraining does not confer a uniform global advantage in human alignment. However, VLMs show selective advantages when sentences contain stronger visual semantic content, with converging evidence from both neural and behavioral measures. The findings suggest language-internal representations remain the primary driver of human text processing alignment.
LoMo: Local Modality Substitution for Deeper Vision-Language Fusion
This paper identifies a 'carrier sensitivity' problem in Vision-Language Models (VLMs), where replacing textual queries with rendered-image equivalents causes significant performance degradation due to asymmetric roles of text and images in training data. The authors propose Local Modality Substitution (LoMo), a data curation paradigm that reformulates single-modality prompts into interleaved multimodal sequences by dynamically rendering text spans as images, enforcing cross-modal representational invariance. Evaluated across 13 multimodal benchmarks, LoMo improves over standard supervised fine-tuning by 2.67 points on LLaVA-OneVision-1.5-8B and 2.82 points on Qwen3.5-9B. The approach is architecture-agnostic and lightweight, requiring no changes to model architecture.
Test-time training corrects modality order sensitivity in vision-language models
Researchers find that vision-language models consistently perform better when images are presented before questions rather than after, a semantically irrelevant prompt-order artifact that persists across three models and three benchmarks. The authors propose an order-consistent test-time training method that closes this gap and also improves the stronger image-first baseline. Activation patching localizes the failure to a narrow mid-network region where representations diverge between orderings, framing modality-order sensitivity as a circuit-level failure. The work contributes both a diagnostic finding about VLM robustness and a lightweight mitigation technique.
MyMentorLLM: Multimodal LLM simulation environment for CBT psychotherapy training
Researchers present MyMentorLLM, a multimodal (voice and text) simulation environment for deliberate practice in Cognitive Behavioural Therapy training, generating 2,100 complete CBT sessions linking simulated DSM-5-TR-grounded patients, trainee therapists, and expert supervisors. Simulated patients expressed disorder-congruent emotional profiles and trainees mirrored these as in real counselling, while supervisor quality varied across LLMs with most overestimating trainee competence. Native speech-to-speech models scored closest to human supervisors, and symptom identification accuracy scaled with model size. The work demonstrates feasibility of LLM-based deliberate practice simulation for clinical training while flagging calibration and harmful feedback as open risks.
Systematic study reveals effectiveness-fluency trade-offs in LLM conditioning methods
A new arXiv paper systematically evaluates a range of LLM conditioning methods across both concept injection and removal scenarios, finding that efficient steering methods often degrade fluency significantly. A key finding is that activation steering is substantially less effective on instruction-tuned models than on base models, a previously overlooked interaction. Simple prompting and supervised fine-tuning work for concept injection but not removal, and cheap textual metrics are found to correlate well with expensive LLM-as-judge evaluations.
Probing VLM activations reveals counting knowledge is encoded but misaligned with outputs
Researchers find that vision-language models often internally encode correct object counts even when they verbalize wrong answers, diagnosing the failure as a misalignment between internal representations and output directions rather than missing knowledge. Using nonlinear probes and SVCCA analysis across four VLMs and five counting datasets, they identify a partially shared but misaligned activation subspace between ground-truth and model-output probes. A causal steering intervention confirms the diagnosis, and a detector-guided self-correction method that re-prompts only on predicted failures improves counting accuracy by up to 15.6 absolute percentage points at inference time with no parameter updates.