The Age of Big Data and the Death of Theory
- Thousands of years ago the human race invented something called "science"
that basically consists in learning a general rule out of many events such
that one can later predict events of the same kind.
- Science creates "theories" that constitute "understanding" of how the world
works, and that one can then apply while living in the world.
"Understanding" the world and "surviving" in the world became two sides of
the same coin.
- The World-wide Web is one tool that has changed that relationship. We can
search for and find instructions to do something without ever learning
any theory about it.
- Of course there have always been "experts" who knew
the theory and would help people who didn't know the theory; but now the
vast amount of data on the Web is making the expert irrelevant.
We don't need to "understand" the world in order to find a solution to our
problem: we just need to know how to search for the solution on the Web.
The "navigator" is another example: we don't need to understand the
territory to travel from one point to another point because the navigator
has all the data required to plan the route.
- It is not just that these tools are replacing the human mind. That would
be the case if they contained the theory, if they worked the way the human
mind used to work; but they don't use a theory. The theory itself becomes
irrelevant when the tool can access enough data.
- For example, we learn the
theory called "arithmetic" that allows us to computer the addition of two
numbers. Once you know the theory, you can compute the addition of any numbers.
However, you wouldn't need the theory anymore if you had a virtually infinite
list of all possible additions and their results: 1+1=2, 1+2=3, 1+3=4, etc.
Then you would simply look up the addition that you need and find the result.
We "discovered" the laws of gravitation to be able to predict where Mars will be on a particular day; but if we can just store and access Mars' orbit minute by minute (and the orbits and motions of all planets and stars), why bother
figuring out that law?
What is happening as more and more data become available is that
we don't need to "teach" a tool the theory: the tool has access to
enough data that can provide all the answers.
- The very concept of knowledge is changing. Knowledge used to be about theories. Even when encyclopedias were invented, there were still meant for minds equipped with theories. The Web and many of the digital tools that can access a virtually infinite amount of "cases" are relieving minds of the need for theories. You need to know only "what you want", not how the world implements that thing.
- Knowledge is no longer about "understanding the world". You don't need to know how your navigator works, a fact that has to do with latitude, longitude, the fact that the Earth is round, down to the details of your country, region, town and neighborhood. You just need to enter the address where you want to go. In fact, you can simply enter the name of the person or business, and not even know the address (not even have a theory of what an address is).
- Knowledge is not a set of theories, but instead just data, and lots of data, and the machine is a better place to store them than the human mind.
- If "understanding" the world is becoming less and less important, then the way the human mind works is changing dramatically. It is likely that the ability to create "theories" is precisely the main difference between the human species and other species. The mind of other species deals with a much more basic
"understanding" of the world (e.g., bananas are food). The human mind must have been doing something very different all those thousands of years when it needed to continuously invent, store, use and teach theories.
- The human mind might be undergoing a transformation that is comparable (but exactly the opposite in direction) to the transformation that took place when it first started constructing general, abstract truths from specific cases.
- If theory dies, then perhaps the human mind as we know it will die too.