paper
Energy-Based Transformers as Predictors of Reading Difficulty
paperactiveprovisional
energy-based-transformers-as-predictors-of-reading-difficulty-f1378b17·1 events·first seen 34h agoAliases: Energy-Based Transformers as Predictors of Reading Difficulty
Co-occurring entities
More like this (12)
The Energy Consumption of Transformer Fine-Tuning: A Roofline-Inspired Scaling ModelEnergy-Based ModelsTransformer Language ModelsVariable-Width TransformersGraph TransformerDecision TransformerListening with Attention: Entropy-Guided Explainability for Transformer-Based Audio ModelsFirst-Token Broadcasters: Mechanistic Origins of Language Identity and Distributed Robustness in TransformersLearning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School TutoringSparse TransformerDynamic Short Convolutions Improve TransformersApparent Psychological Profiles of Large Language Models are Largely a Measurement Artifact
Recent events (1)
Energy-based transformers as unified predictors of reading difficulty in computational psycholinguistics
A new arXiv preprint introduces energy-based transformer measures as predictors of human reading difficulty, evaluated across three reading-time corpora (Natural Stories, UCL eye-tracking, UCL self-paced reading). The energy measure outperforms surprisal alone and appears to subsume both surprisal and attention entropy effects, suggesting it could serve as a single unified predictor. The work connects transformer language models to Hopfield networks and dense associative memory literature, marking the first application of energy-based transformer measures in computational psycholinguistics.