Most of the artificial intelligence used for translation, tagging photos on Facebook, and optimizing the best route for navigation relies on humans feeding the AI some information to start. We show the algorithms which sentences are equivalent in other languages, what a person looks like in different photos, and how to plot the ideal course for a car.
But some AI researchers are exploring how to give algorithms a sense of curiosity, so they can learn without any human guidance. New research from OpenAI, the non-profit AI lab founded by Elon Musk, Sam Altman, and other Silicon Valley bigwigs, in collaboration with researchers from UC Berkeley and the University of Edinburgh, has found that when an AI algorithm is given a simple definition of curiosity, it can, without any human-provided information, explore more than 50 video games—and even beat some of them.
But curiosity comes with a cost. The researchers also found that since the AI agent was rewarded for seeing new things, sometimes it would die on purpose just to see the Game Over screen, or become enrapt with a fake TV and its remote, flipping through channel after channel to find something new.
What is artificial curiosity?
The definition that OpenAI team used for artificial curiosity was relatively simple: The algorithm would try to predict what its environment would look like one frame into the future. When that next frame happened, the algorithm would be rewarded by how wrong it was. The idea is that if the algorithm could predict what would happen in the environment, it had seen it before.
That’s why the AI agents were so good at games like Super Mario—the game is based in exploring ahead and getting to the next level.
What’s so special about TV?
OpenAI researcher Harri Edwards tells Quartz that the idea for letting the AI agent flip through channels came from a thought experiment called the noisy-TV problem. The static on a TV is immensely random, so a curious AI agent could never truly predict what would happen next, and get drawn into watching the TV forever. In the real world, you could think of it as something completely random, like the way light shimmers off a waterfall.
The researchers tested their theory by putting a digital TV inside a 3D environment, and allowing the agent to press a button to change the channel. When the agent found the TV and started flipping through the channels, the stream of new images made the TV irresistible.
Edwards said there were instances when the AI could pry itself away from the TV, but only when the AI’s surroundings somehow seemed more interesting than the next thing on TV.
Beyond games
The point of this research isn’t just to beat video games with AI, but also to understand how algorithms might better interpret the world around them. Since these algorithms proved to be efficient at exploring all the nooks and crannies of video games, the researchers say they could also be adapted to make debugging code easier or to play through a video game to make sure there aren’t any glitches.