technique
Unsupervised Pre-training
techniqueactive
unsupervised-pre-training-7ef48c25·1 events·first seen 28d agoAliases: Unsupervised Pre-training
Co-occurring entities
More like this (12)
Self-Supervised Pretrainingunsupervised learningContrastive Pre-trainingunsupervised language modelingSelf-Supervised Learninginstruction-based multitask pretrainingVideo PreTraining (VPT)Supervised Memory TrainingPretraining Recurrent Networks without Recurrencecontrastive vision-language pretrainingzero-shot learningImitation Learning
Recent events (1)
Improving Language Understanding with Unsupervised Learning (GPT-1)
OpenAI published the GPT-1 paper in June 2018, demonstrating state-of-the-art results across diverse language tasks by combining transformer architectures with unsupervised pre-training followed by supervised fine-tuning. The approach is task-agnostic and scalable, showing that pre-training on large unlabeled text corpora and then fine-tuning on specific tasks yields strong generalization. This work established the foundational paradigm that would evolve into GPT-2, GPT-3, and subsequent large language models.