AlphaGo, the AI-based computer program developed by Google’s artificial intelligence (AI) business DeepMind, has been in the news for defeating the Go world champion, Lee Sedol, four games to one in their five match series. Go is one of the most complex board games of all the “perfect information games”, a category that includes chess, checkers, and tic-tac-toe. Go has 10360 potential moves, more than the number of atoms in the universe. There have been multiple attempts to utilize computers and algorithms to defeat human Go players, and it was believed that this AI research milestone was at least 10 years away. It is now clear that we have crossed that intelligence chasm, and the question is where we go from here.
Instead of trying to construct a search tree of all possible positions, which is almost impossible to do with today’s computing power, DeepMind has used a 12 layer deep neural network (DNN) with advanced tree search, which takes a description of the Go board as an input and processes it through the 12 layers, learning throughout the process. One neural network, called the policy network, selects the next move of play, and another called the value network predicts the winner of the game. AlphaGo was trained on 30 million moves from games played by experts until it could predict a human expert’s move. After extensive training, it started to get better than humans by using a technique called reinforcement learning by having the neural networks play against themselves. Over time, with training and by beating both humans and machines, AlphaGo was ready to play with Lee Sedol, and now has shown that it is good enough to beat the reigning world champion in Go.
(Source: Google DeepMind)
AlphaGo is the best current example of a machine learning system that has been trained using supervised learning, and over a short period of time has gotten better, eventually outperforming the best that humans have to offer. The speed at which this has happened has astonished everyone, which in fact is full credit to the skill and ingenuity of the DeepMind researchers, their algorithms, and Google’s vast hardware resources upon which these algorithms were trained. The interesting bit starts from here, as everyone tries to figure out what’s next for AlphaGo and DeepMind.
On one hand, Demis Hassabis, the co-founder of DeepMind, has said that they might try to use unsupervised learning to develop another version of AlphaGo where there is absolutely no human intervention of what is wrong or right, and just based on observing the board, the machine is able to learn the game, and ultimately outperform a human. That would be an impressive achievement to top the current one. Both Hassabis and Jeff Dean, who leads Google Brain, have said that they might now look at tackling other types of games that are strategy-based, but multiplayer, where a computer is essentially competing against multiple humans. StarCraft is one of the most popular online multiplayer games in the world, with a huge following in Korea, just like Go. In fact, StarCraft games are broadcast on television in Korea, with players being media celebrities. Tackling StarCraft seems interesting, although I am still unclear if there is a scientific research goal that is worth going after. Also, it could be a distraction for DeepMind and Google, although Google’s parent company Alphabet is known to take bold risks even if there are no immediate payoffs. For now, DeepMind sits within Google, rather than as a separate entity, and so has an immediate impact on its biggest business.
Also, it’s no secret that Google is focusing more on AI, with Sundar Pichai having said that AI and machine learning are at the center of everything they do, and is forcing them to rethink their whole product strategy. Google Brain, which is the other AI arm aside from DeepMind, has a more direct impact on Google products and near-term strategy, with offerings like Google Now or Google Photos search. However, DeepMind is not known to have a particular remit, other than to advance AI algorithms and push the state of the art. However, now that it has reached the Go milestone, DeepMind should start to see parts of the AlphaGo solution be used within the Google strategy. Hassabis has already mentioned in a recent Verge interview that their first stop will be smartphone assistants. While DeepMind did launch its Health platform, that is a kind of a side project where it is helping the National Health Service (NHS) modernize and transform, with no real immediate implications of using AI or machine learning.
So, it’s clear that Sundar Pichai will try and use some of the DeepMind expertise to drive a major advancement in Google Now and its future avatars. I expect to see a lot of focus on Now on Tap at the Google I/O 2016 event, although none of that will be related to DeepMind. However, in the immediate term the implications of AlphaGo are minimal. My hunch is that DeepMind is still going to continue to focus on the bigger scientific research questions and applications using AI, like helping CERN find the next particle, or helping to understand the Big Bang, or find the grand universal theory of everything. At the same time, DeepMind will also focus on healthcare, finding cures for diseases or making healthcare ecosystems much more effective and sustainable.
The other area where DeepMind is going to be very useful is self-driving cars, which would make it more applicable to Alphabet and its other interests. I won’t be surprised if DeepMind is already working closely with the self-driving car team, helping to develop algorithms that are better at predicting different scenarios during driving, make predictions just like humans, and develop a level of intuition, something that human drivers develop naturally. For me, DeepMind has a lot to offer in the area of autonomous driving, with hundreds or thousands or even millions of corner cases that these algorithms need to anticipate in the real world. If there is an AI system that has the potential to speed it up development of self-driving cars today, it is DeepMind, and AlphaGo is DeepMind just getting warmed up. Also, DeepMind might eventually be pulled out of Google and be allowed to exist as a separate company within Alphabet, along with Google Brain.