Minecraft isn’t a game most would think AI could master, but OpenAI has other plans. They managed to train a neural network to play Minecraft utilizing Video PreTraining (VPT) on a large unlabeled video dataset of human Minecraft play, while using only a small amount of labeled contractor data.
We trained a neural network to competently play Minecraft by pre-training on a large unlabeled video dataset of human Minecraft play and a small amount of labeled contractor data. https://t.co/a2pyBqvLvg pic.twitter.com/XbqtwQSTwU
— OpenAI (@OpenAI) June 23, 2022
With some more tuning, OpenAI says its model can learn to craft diamond tools, a task that usually takes human players over 20 minutes (24,000 actions). How? Their model uses the native human interface of key presses and mouse movements, which means it’s quite general, and thus represents a step towards general computer-using agents. Minecraft and AI is already sort of a thing with Facebook teaming up with MIT to create an artificial intelligence agent for the game.
No products found.
Perhaps the most important hypothesis of our work is that it is far more effective to use labeled contractor data to train an IDM (as part of the VPT pipeline) than it is to directly train a BC foundation model from that same small contractor dataset. To validate this hypothesis we train foundation models on increasing amounts of data from 1 to 70,000 hours,” said the researchers.
