Minecraft

Learning to play Minecraft with Video PreTraining (VPT)

OpenAI trained a neural network to play Minecraft using Video PreTraining (VPT) on a large unlabeled video dataset of human gameplay, supplemented by a small amount of labeled contractor data. The model operates via native human interface inputs (keypresses and mouse movements) rather than game APIs. After fine-tuning, it can craft diamond tools—a task requiring over 20 minutes and ~24,000 actions for skilled humans. The work is framed as a step toward general computer-using agents.