Book review of Andy Clark

Andy Clark:
ASSOCIATIVE ENGINES (MIT Press, 1993)

Home | The whole bibliography | My book on Consciousness

The British philosopher Andy Clark provides a philosophical introduction to the field of neural networks. In his preface, he correctly points out that the rebirth of this field signals a shift of perspective in the way we view the mind, away from a "static" view of mental representations and towards a fluid view of the cognitive activity of the mind, towards the process, not just the structure. Concepts are continuously in flux, and neural networks are a computational device that seems able to capture that flux.
Clark shows this new perspective by attacking Fodor's representational theory of mind. Fodor's theory was meant to provide an explanation of how thoughts become "causes". Fodor assumes that propositional attitudes (I believe that, I hope that, I fear that, etc) are computations on mental representations (eg, a concept such as "my name is piero"), which, in turn, can be objects of computation because they are symbolic expressions (as in "logic"). Each kind of propositional attitude (eg, "belief") expresses a different kind of role and therefore a different kind of computation. Thus, "I believe that my name is piero" is different from "I hope that my name is piero" because the computation performed on the mental representation is different. The human brain knows how to represent and compute because it comes equipped with a "language of thought" that works just like the language of mathematical logic.
Clark does not believe such a language exists in the mind and does not believe that Fodor's vision of the mind can account for the "process" of thinking. In Fodor's model, learning is a secondary phenomenon and is largely independent of the environment. Clark, instead, advocates a model in which learning is a fundamental feature of the mind and learning is largely dependent on the environment. That is precisely the difference between knowledge-based models and connectionist models. Moreover, Fodor clearly distinguishes between the computation and the representation, whereas Clark believes that process and representation are one and the same.
Neural networks provide a more plausible model for cognition. Clark highlights three key features of connectionism: superposition, context sensitivity and representational charge. Superposition is the ability to represent two things with the same structure: a same neural network can be trained (by changing the weights of its connections) to recognize multiple items. Context sensitivity follows from the fact that those weights encode multiple items and therefore the "representation" of something is automatically context-sensitive. Fodor's symbols are always the same regarding of the context in which they are located (the context is expressed via relationships between symbols), whereas neural networks embody the context of what they represent (the context is expressed internally). Representational change is not only the ability to create new representations (Fodor's models can do that by combining symbolic expressions to create new symbolic expressions) but also the acquisition of new representational capacities. The difference is that the former learns by combining pre-existing, internal expressions, whereas a connectionist model learns by being trained by an external environment.
Clark observes features that are unique to neural networks. One cannot breakdown or troubleshoot how a network does what it does because it depends on the "process" of learning: just looking at the result of learning is not enough to understand how the network performs the task that it has learned to perform. It is like watching a man ride a bike without having watched how he learned to ride the bike: the process of learning is what explains how he is now capable of riding the bike. If we try to analyze his action of riding the bike, we basically try to reduce the task of riding the bike to a set of symbols, which is a contradiction in terms because, again, riding a bike is not obtained by computing symbols, it is obtained by learning how to ride a bike.
Clark points out that, ideally, neural networks should also be able to undergo what developmental psychologist Karmiloff-Smith calls "redescriptions", or complete reorganizations of knowledge that open up new cognitive abilities and lead to a new developmental stage (e.g., during child development).
Since they are trained by a set of data that comes from the environment, connectionist systems depend on luck: they can only learn if the set of data includes enough statistical information (technically: associative learning is heavily dependent on the statistical distribution of input data). Our brain somehow learns even in a hostile environment that does not provide enough data about this or that concept, but neural networks fail badly to learn anything unless the set of input data is favorable to the correspondent training (their success depends on the continued availability of a friendly training environment). Clark's suggestion is that the human mind is a neural network that has evolved over thousands of years and therefore has absorbed huge amounts of innate knowledge. In other words, connectionism is not all there is to human cognition: evolution is another big piece of the story.
While Fodor views concepts as the building blocks of thoughts and as represented by fixed structures and as causing action through their relationships, Clark views a concept as a set of skills that a network learns, and views the effect of those skills as the "behavior" of that network. Folk psychology creates the belief that there are such things as concepts when in reality there are only sets of learned skills. To ascribe a concept to a person is to ascribe a set of skills to that person. The set of skills defines the potential behavior of that person or network. Thus "concepts" are basically an illusion created by the language of folk psychology.
The rest of the book provides an up-to-date survey of the status of the field, what neural networks already do and what they can't yet do.
Clark is not a gifted writers and his ideas are often difficult to grasp (not because they are difficult but because his language is convoluted). One also senses that his arguments are weak at best and that most claims are simply based on different terminology. Most of the differences Clark lists between knowledge-based and connectionist systems are really different views of the world, not necessarily incompatible views of the world. Despite all this, Clark provides a powerful analysis of connectionism and its philosophical implications.

Permission is granted to download/print out/redistribute this file provided it is unaltered, including credits.