(These are excerpts from my book "Intelligence is not Artificial")
The Curse of the Large Dataset
The most damning evidence that A.I. has posted very little conceptual progress towards human-level intelligence comes from an analysis of what truly contributed to A.I.'s most advertised successes: the algorithm or the training database? The algorithm is designed to learn an intelligent task, but has to be trained by human-provided examples of that intelligent task.
Neural networks learn patterns. There is a pattern about neural networks that has become the norm after the 1990s: an old technique stages spectacular performance thanks to a large training dataset, besides more powerful processors.
In 1997 Deep Blue used the NegaScout algorithm of 1983. The key to its success, besides the massively-parallel 30 high-performance processors, was a dataset of 700,000 chess games played by masters, a dataset created in 1991 by IBM for the second of Feng-hsiung Hsu's chess-playing programs, Deep Thoughts 2.
In 2011 Watson utilized (quote) "90 clustered IBM Power 750 servers with 32 Power7 cores running at 3.55 GHz with four threads per core" and a dataset of 8.6 million documents culled from the Web in 2010, but its "intelligence" was Robert Jacobs' 20-year-old "mixture-of-experts" technique.
All the successes of convolutional neural networks after 2012 were based on Fukushima's 30-year-old technique but trained on the ImageNet dataset of one million labeled images created in 2009 by Feifei Li.
In 2015 DeepMind's videogame-playing program used Chris Watkins' Q-learning algorithm of 1989 but trained on the Arcade Learning Environment dataset of Atari games developed in 2013 by Michael Bowling `s team at the University of Alberta.
In 2016 AlphaGo used the dataset of millions of go positions stored and ranked at the KGS Go Server (Kiseido Go Server).
It is easy to predict that the next breakthrough in Deep Learning will not come from a new conceptual discovery but from a new large dataset in some other domain of expertise. Progress in Deep Learning depends to a large extent on many human beings (typically PhD students) who manually accumulate a large body of facts. It is not terribly important what kind of neural network gets trained to use those data, as long as there are really a lot of data. The pattern looks like this: at first the dataset becomes very popular among hacker; then some of these hackers utilize an old-fashioned A.I. technique to train an artificial intelligence until it exhibits master-like skills in that domain.
Back to the Table of Contents
Purchase "Intelligence is not Artificial"