We have several ways to test the progress of artificial intelligence. One of the more fun ways to do so is to have them play games and chart their victories over humans. You take champions of games like checkers, chess, or Go and pit them against the AI. If the robot wins, that's one point for our future digital overlords.
In August, a team of five bots competed in a more sophisticated gaming scenario. OpenAI, a research institute co-founded by Tesla CEO Elon Musk, had created them all. Their game of choice? Dota 2.
Dota 2 is significantly more difficult than checkers. When first released, human players complained about its steep learning curve and complexity. In the game, players (human or robot) compete in two teams of 5. They choose the type of characters they want. Then assemble and army as they occupy and protect a base.
In addition to being fun, looking at the way the robots lost helps researchers. It highlights AI weaknesses. This is in part because bots learn through trial and error. They react and act on a moment to moment basis. They missed several opportunities the humans capitalized on. Susan Zhang, a software engineer on the project, blamed the bots planning abilities. Through this exercise they learned the bots can only "think" and "plan" about 14 minutes ahead. That's not a lot of time in a complex strategy game.
What does this mean for us? Algorithms have a lot of room for improvement - good news for those of us who've seen Blade Runner.