Amazon Reviews Dataset

Unsupervised Sentiment Neuron

OpenAI researchers trained a character-level language model on Amazon reviews to predict the next character and discovered it spontaneously learned a single neuron encoding sentiment with high accuracy. The system achieved state-of-the-art sentiment classification with minimal labeled data, demonstrating that unsupervised language modeling can yield interpretable, task-relevant representations. This was an early result connecting unsupervised pretraining to downstream NLP tasks.