Improving Language Understanding with Unsupervised Learning (GPT-1)
OpenAI published the GPT-1 paper in June 2018, demonstrating state-of-the-art results across diverse language tasks by combining transformer architectures with unsupervised pre-training followed by supervised fine-tuning. The approach is task-agnostic and scalable, showing that pre-training on large unlabeled text corpora and then fine-tuning on specific tasks yields strong generalization. This work established the foundational paradigm that would evolve into GPT-2, GPT-3, and subsequent large language models.
Related guides (4)
Related events (8)
Better language models and their implications
OpenAI announced GPT-2, a large-scale unsupervised language model capable of generating coherent multi-paragraph text and achieving state-of-the-art performance on language modeling benchmarks. The model demonstrated zero-shot capability across reading comprehension, machine translation, question answering, and summarization without task-specific fine-tuning. OpenAI notably withheld the full model release citing misuse concerns, marking an early high-profile instance of staged/responsible release policy.
Language models are few-shot learners
OpenAI published the GPT-3 paper introducing a 175-billion-parameter autoregressive language model demonstrating strong few-shot learning capabilities across a wide range of NLP tasks. The work showed that scaling language models dramatically improves task-agnostic, few-shot performance, often matching or exceeding fine-tuned models without any gradient updates. This paper became a foundational milestone in the development of large language models and the modern AI landscape.
GPT-4 Release
OpenAI released GPT-4, a large multimodal model accepting image and text inputs and producing text outputs. The model demonstrates human-level performance on various professional and academic benchmarks. It represents OpenAI's latest milestone in scaling deep learning.
WebGPT: Improving the factual accuracy of language models through web browsing
OpenAI fine-tuned GPT-3 to answer open-ended questions more accurately by giving it access to a text-based web browser. The system, called WebGPT, uses reinforcement learning from human feedback to learn to search the web, read pages, and cite sources. This work represents an early demonstration of retrieval-augmented generation and tool-use in large language models.
Aligning language models to follow instructions
OpenAI published a blog post describing their work on aligning language models to follow human instructions, corresponding to the InstructGPT research. This work introduced reinforcement learning from human feedback (RLHF) as a core technique for training models to be more helpful, honest, and aligned with user intent. The approach demonstrated that smaller instruction-tuned models could outperform larger base models on human preference evaluations, marking a foundational shift in how language models are trained and deployed.
Introducing GPT-5.2
OpenAI has released GPT-5.2, described as their most advanced frontier model for professional use, featuring state-of-the-art reasoning, long-context understanding, coding, and vision capabilities. The model is available through ChatGPT and the OpenAI API. It is positioned to support faster and more reliable agentic workflows.
GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
OpenAI published research examining the potential labor market impacts of large language models, analyzing which occupations and tasks are most exposed to automation or augmentation by GPT-class models. The study introduces a framework for assessing LLM 'exposure' across job categories, finding that a significant share of U.S. workers could see at least 50% of their tasks affected. The paper represents an early systematic attempt to quantify economic disruption potential from frontier AI systems.
Customizing GPT-3 for your application
OpenAI announced fine-tuning capabilities for GPT-3, enabling developers to customize the model for specific applications via a single command. This feature allows users to adapt GPT-3's behavior to their use case by training on domain-specific data. The announcement marks an early milestone in making large language model customization accessible through an API.