paper

Fine-tuning GPT-2 from Human Preferences

paperactivefine-tuning-gpt-2-from-human-preferences-bd26ab92·1 events·first seen 28d ago

Aliases: Fine-tuning GPT-2 from Human Preferences

Co-occurring entities

Reinforcement Learning from Human Feedback GPT-2 OpenAI

More like this (12)

GPT-4o vision fine-tuning Parameter-Efficient Fine-Tuning behavioral fine-tuning GPT-2 OpenAI Fine-Tuning Chain-of-Thought Fine-Tuning Language Model Finetuning Semantic Generative Tuning (SGT)GPT-2 355M reinforcement fine-tuning supervised fine-tuning Humanoid-GPT

Recent events (1)

6Openai Blog·28d ago·source ↗

Fine-tuning GPT-2 from Human Preferences

OpenAI fine-tuned the 774M parameter GPT-2 model using human feedback across summarization and style-continuation tasks, requiring 60k and 5k human labels respectively. The work revealed a labeler preference misalignment: for summarization, labelers rewarded copying from source text rather than genuine summarization. The stated motivation is advancing safety techniques for human-machine interaction and learning about human values from feedback.

Frontier Model Releases Evaluation and Benchmarking Reinforcement Learning from Human Feedback GPT-2 Fine-tuning GPT-2 from Human Preferences +2 more