Representing, Similarity, and the Storage of Information

 

            Representing is an activity, a process through which a subject cognizes the world. Most theories of representation take one element or component of this activity and identify it as the representation by attributing to that element, in isolation, properties it has only in the context of the act of representing. The purpose of this paper is to argue that such views are false and that representation can only play its normal roles in cognition if it is seen as an activity or process rather than a static element in the cognitive process.

            I will look at three basic views of representation and argue that they fail because they abstract out one component of the representing process and try to explain how this component can play its role in cognition in virtue of its intrinsic properties. First, I will consider views of iconic representation: views that see representations as objects that represent in virtue of their similarity to the object they represent. I will argue that similarity is neither sufficient nor necessary for representation, although it can play an important role in representation. Second, I will look at views of symbolic representation: views that see objects as representing in virtue of some isomorphism effected by a function that maps elements of the representation onto elements of its object. I will take language and information storage as the paradigmatic examples of this type of representation and will argue that neither the objects that have the isomorphism nor the functions that do the connecting (nor the physical embodiments of these functions) can be seen as representations. Third, the views of representation involved in Parallel Distributed Processing models and in Paul Churchland's analysis of neural networks as phase space sandwiches will be considered. These views identify representation as a pattern of activation distributed over a network or a phase space that is connected to other representations through a network of weighted connections. I will argue that, although these views properly analyze the structure of the representing process, they mistakenly identify a single part of this process as the representation. Finally, I will try to suggest what a view of representation that avoids these mistakes might look like and how it might solve some of the puzzles about the role of representation in cognition that these mistakes tend to generate. I will argue that representing is a process of connecting two different modes of interaction with the world and that its content, and its role in cognition, lies in the connections made within this process, not in the two things connected, nor in the function that connects them.

 

I. Iconic Representation

 

            The simplest, and perhaps the most fundamental, view of representation is the iconic, which views a representation as an object that represents in virtue of its similarity to its object. The epistemological difficulties with such a view are commonplace. (How is the similarity between the representation and the object ascertained? Which of the many similarities are important and how are they picked out?) I aim to point out a more fundamental problem, which I shall argue gives rise to the epistemological problems: Similarity is not essentially connected to representation. It is neither sufficient nor necessary for representation.

            Similarity is not sufficient for representation. This point seems obvious. (Although the reason why it is true is, perhaps, not so obvious.) Not all things that bear a similarity to one another are representations of one another. A leaf and the flame emitted from burning copper wire are similar in color, yet no one would take one as representing the other. Even in cases that are more plausible, such as the resemblance of a photograph to its subject, if similarity is sufficient for representation then one could equally claim that the subject is a representation of the photograph. As Putnam has pointed out (Putnam 1981, pp. 1-5) a physical object cannot represent an object simply in virtue of its similarity to it. If an ant traced out a perfect caricature of Winston Churchill in its wanderings, we would not say that the ant has produced a representation of Churchill. Why not, and why would anyone think it plausible to say that an iconic representation represents in virtue of its similarity to the object?

            We do not recognize the trail of the ant or the green leaf as representations because we realize that these things require interpretation by an observer with certain abilities and knowledge in order to serve as representations. They don't do it themselves, in virtue of their similarity to certain objects. In order to represent an object, the icon must call up other properties of the object  besides those to which it is similar. The lines in the sand must call to mind the three dimensional rotund shape or the other features we happen to associate with Churchill. The icon cannot do this itself. It requires an interpreter with certain abilities, knowledge, and associations to connect the properties of the icon  that are similar to the other properties of the object.  (Witness the dog who passes by the life-like statue in the museum without ever seeing it as representing a man.)

            The argument can be put in a slightly more rigorous form. A representation must call up more properties than those to which it is similar. It cannot simply represent the properties that it shares with its object. This is true for two reasons: (1) It does not represent these properties; it instantiates them.^[1] A yellow disk as an icon for the sun does not represent just the yellowness and  roundness of the sun, it is yellow and it is round. It can only be said to represent the sun if it calls up other properties besides the particular yellowness and roundness that it possesses, and these it cannot call up on its own, simply in virtue of its similarity to the sun. This requires an interpreter who knows these other properties and associates them with the properties instantiated in the yellow disk. (2) If icons represent in virtue of their similarity then they can never represent individuals, but only universal sets of properties. No icon is similar in all respects to the object it represents. (Even if it were, it would not represent its object; it would be it.) Hence, it cannot represent only that one object, but also every other object and possible object to which it is similar. If the yellow disk represents merely in virtue of similarity, then it cannot represent the sun; it represents all yellow, round objects. ^[2] 

            Hence, similarity alone cannot be sufficient for representation. Icons represent only in virtue of calling up other properties of the object besides those to which they are similar, and they cannot do this apart from the interpreter with certain abilities, knowledge, and dispositions. Why did anyone think they could? The answer to this question is revealing with respect to the epistemological problems that arise from these views of representation. The problem arises, I believe, from taking certain types of physical icons as a paradigm of representation, incompletely analyzing how they work, and then applying this analysis to the paradigmatic mental representations, visual images. Hence, I call this view the Physical-Visual Model of Representation.

            Imagine looking at an object and a physical representation of that object. Take, for example, a statue and the woman it was modelled after. In this case both the object and the representation are contained in the same perception. We perceive their similarities, and we can perceive the correspondences between the statue and the women. This allows us to see, for example, how the elbow of ivory maps onto the elbow of flesh. Thus we can take the ivory as representing the flesh; we project the properties of the woman onto the statue guided by the perceived similarities. The statue represents the woman only in virtue of the complete interpretation of the situation by the observer and the access that he has to both the statue and the woman.  This is a paradigmatic case of physical representation, but it is one that is bound to be incompletely analyzed. It is natural to leave out the part that the observer plays in this situation and attribute the representative qualities of the statue to its similarity to the woman. After all, it is the similarities that guide the projection of the properties of the woman onto the statue. The fact that this projection is an act of the observer, dependent upon his ability to interact perceptually with both the statue and the woman, is easily overlooked. It is this simplified analysis of physical representation that is taken as the paradigm and applied to mental representation.

            The mental image (or the piece of language in modern theories) is seen as a representation or picture. It is thought to function autonomously, apart from the action of an interpreting subject and its interaction with an external object, just as a physical representation seems to. The mental representation is seen as a mental object that represents in virtue of its own properties. Instead of seeing representation as a process of interaction between subject and object, the representation is reified into an object and the visual perspective on the external world turned into a view of this mental object, not a view of the external world.  Through a perspective we see only the reified representation, not the world; thus, mental representation becomes a veil of ideas separating the subject from any independent access to the external object. If the  mental image is seen as representing merely in virtue of its similarity and as the only medium though which we see reality, it is obvious that we can never determine whether our representation is really similar to external reality. This is where the epistemological problems that attend this view of representation arise. More importantly, it is also clear that it is also impossible for such mental images to serve as representations of external reality. Since an icon can only represent by calling up other properties of the object besides those to which it is similar, and since this can be done only by an interpreter who has access to the object and its other properties apart from the representation (as is the case of physical icons), the physical-visual model of representation makes mental representation impossible.^[3]

 

II. Symbolic Representation and the Storage of Information

 

            If similarity is not sufficient for representation, neither is it necessary. It seems that many forms of symbolic representation involve no similarity (in any obvious sense) between the symbol and what it represents. When I use X's and O's to represent football players, it is not similarity, in any normal sense, that allows these symbols to represent. Rather, there are a set of conventions or rules that allow one to move from the symbol to its meaning.

            Obviously, language is a paradigmatic example of symbolic representation. This type of representation is also often attributed to various types of physical systems that encode information. This involves taking information and placing it in a new medium in which it takes another form and from which it can be retrieved. Examples of this are the storage of a visual image on a photographic negative, the storage of information on computer disks,  the reproduction of music on tapes and on compact disks, and the encoding of genetic information on DNA strands. All of these are popular models for representation. It seems in all of these as if the information is stored by representing it in a different form and then using that representation to guide the reproduction of the original information. It is attractive to claim that the song is on the compact disk, that the words are on the computer disk, represented in a different way to allow for convenient storage. It is attractive to claim that information storage involves the representation of the information.

            It is not clear why. These types of systems are primarily involved in reproduction of various images, sounds, and organisms, rather than any representation of the type involved in cognition. Perhaps a type of personification is involved. We know that we could not reproduce an image or a sound unless we had a good representation of it that guided our activity. Hence, it is plausible to see the pattern of marks on a compact disk through which the player reproduces the original sound as analogous to the mental representation or blueprint that guides the builder's reproduction of a house. It is not at all clear why one should assume that the reproduction of certain physical forms in such machines must be mediated by a representation. Nor is it any more clear that the various physical processes through which human beings engage in similar types of reproduction (e.g., memory) involve the mediation of physical objects or states that are stored representations of the reproduced form.^[4]  I will argue that information storage and retrieval does not involve representation. Computer disks, tapes, compact disks, and configurations of neurons in the brain do not contain representations. They are components (modular and, sometimes, portable and interchangeable components) of physical systems that reproduce certain sounds, objects, or patterns of activity, but reproduction is not representation, nor does it require representation.^[5]           

            A proponent of the iconic view of representation might argue that the compact disk and its pattern of marks does represent or contain the song in virtue of an isomorphism, or highly abstract similarity of form between the pattern and the song. Of course this is just a more sophisticated version of the iconic view that things represent simply in virtue of similarity and is prey to the same arguments given above. But it is instructive to see in some detail how those arguments can be applied in this case.

            Note that the position stated above actually involves two separate possibilities: (1) that the disk actually contains the song itself; or (2) that the disk contains a representation of the song. In the first case it might be claimed that the song is actually an abstract set of functional relations that are also present in the patterns of marks on the disk. The song, which is the abstract pattern is both in the sounds and on the disk. Note that this is a case of complete similarity: the pattern of marks on the disk does not represent the song it is the song. Again, an icon cannot represent the traits that it shares with the object; it simply instantiates those traits.

            Even this claim of identity is implausible in the case of the compact disk for two reasons: (1) Even if the song were merely an abstract functional pattern, it is not clear that the disk shares this pattern in any clear sense. The song exists as a pattern of sounds in time. The pattern on the disk cannot instantiate these temporal relations. More likely, the reproduction of the sound, or even the pattern of electrical impulses that cause the reproduction in the amplifier or speaker,  instantiate this pattern. (Hence, we believe the reproduction is the same song as the original, not a representation.)  (2) More importantly, the isomorphism that the pattern on the disk bears to the song is completely relative to the program or set of functions in the disk player that can reproduce the song from the patterns on the disk. To say that there is an isomorphism between the two is to say nothing more than that there exits or could exist an algorithm that could map the elements of one onto the elements of the other in some determinate way. Of course this is true of just about any two objects, given an intricate enough algorithm.^[6] Once one sees that there is nothing unique or special about the pattern on the disk (apart from the fact that it might be especially compact or economical) that sets it apart from the infinite number of other patterns that, given the right algorithm or type of player, could also reproduce the song, it is much less plausible to take the pattern as being the song, or even as representing it.

            Of course, the above discussion of the possibility of the pattern on the disk being the song is irrelevant to the question of whether such information storage systems involve representation: If it is the song, it doesn't represent it. Some of the same arguments, however, will apply to the thesis that the pattern on the disk represents the song. They are really just versions of the arguments presented above against the thesis that icons represent in virtue of their similarity. To represent, the pattern on the disk must be able to call up the other properties of the song, besides those it possesses; it must be able to reproduce the song. This it cannot do by itself in virtue of its similarity to the song.  It can do this only as part of a causal system, the player, which embodies a function that can reproduce the other properties of the song from the pattern on the disk. Again, an analogue to an interpreter with specific abilities and dispositions is required. The song itself, or a representation of it, is not present and intact in the new medium, to which anyone who has had a computer disk without the appropriate software to read it can attest. The very same physical object with all the same properties would not be an encoding of information if the causal system, in this case compact disk players did not exist. The very same pattern that, on a compact disk, encodes MC Hammer's "You can't Touch This" might have existed on the etchings of a meteor in the orbit of Mars before the existence of human beings. Yet, it stretches the imagination, to say the least, to claim that this would have been a representation of the croonings, or rappings, of MC Hammer.

            Further, the very same pattern might allow a reproduction of many other songs given a different algorithm or type of player. The isomorphism that the pattern on the disk possesses is hopelessly indeterminate. It could be isomorphic to almost any song given an intricate enough algorithm that would allow the mapping of the elements of the song onto the elements of the pattern on the disk. If it is a representation at all in virtue of its intrinsic properties, it is a hopelessly indeterminate or universal one.

            A causal system that reproduces information does not represent. It may be useful to an interpreter who can interpret the reproduced information and use it to represent, but it does not do the job itself. A system that reproduces objects does not represent them, nor does it require representations of them. A chair factory does not represent chairs; it makes them. Thus, a system for encoding information does not itself represent, although it may be useful for a creature that can represent.

 

III. PDP and Phase Space Sandwich Models

            Parallel Distributed Processing and Paul Churchland's phase space sandwich interpretation of neural networks provide analyses in which representation is presented as firmly embedded in a physical system that is capable of calling up or connecting the representation with the other properties of the object. But, perhaps taken in by the allure of the iconic model of representation, they identify only a part of this process as the representation. I will argue that they correctly identify the essential features of the representing process, but that they mis-identify the parts of that process that are the representations.

            PDP models (Rumelhart 1987) identify distributed patterns of activation over a matrix of units as representations. Each such set of units is connected to others by sets of weighted connections by which a representation, or pattern of activation, on one set of units is transformed into another pattern, a different representation, on the other set of units. For example, a set of units connected to smell receptors might be connected by a set of weighted connections to a set of units in the visual part of the brain. Input from the nose might stimulate a certain pattern of activation in the olfactory units; this might be identified with the representation or olfactory image of the smell of a rose. The connections to the visual units pass on this pattern transforming it into another pattern along the way, which might be identified with the representation, or visual image, of the rose. Thought might be conceived as the transformation of one representation into another effected by these weighted connections.

            The representation is identified as a particular pattern or configuration of activation over certain units. Why? The pattern does bear a certain similarity or isomorphism to the object it arises from and to the pattern it will give rise to in the other set of units. It is exactly analogous to the pattern on the disk that is situated in an information reproduction system: Memory is a system through which images are reproduced according to the information "represented" in the input patterns according to the algorithms embodied in the weighted connections. But we have already seen what is wrong with those two models of representation.

            In Churchland's phase space sandwich model (Churchland 1986, 1987), we have large layers of interconnected neurons instead of the smaller sets of units in PDP models. In between these layers are sandwiched sets of interconnections between the two layers. For example, the input from the eyes might produce a certain pattern of activation on a layer of neurons in the visual cortex. This might represent some object in visual phase space. The pattern in the phase space might even be similar to the object; it might be a topological transformation of it. The visual cortex might be connected to the motor cortex by a set of connections that embody a tensor function that transforms the pattern in the visual space into one in the motor space. This pattern represents a position or a movement in motor phase space that directs the motions of some limb. Representations are positions or patterns in the phase space, ones that are actually visually similar topological transformations of the shape of the objects they represent. (Churchland 1987, pp. 279-280) Computation is the coordinate transformation of the representation on one phase space into another by means of a tensor function embodied in a set of connections sandwiched between the two layers.

            Philosophers, since the time of Berkeley, have been poking fun at the idea that there are little pictures or images of objects in the head, and now it turns out that there actually are little topologically transformed icons in our brains. The motivation for identifying these as representations is clear; hopefully, as clear as the problems with doing so. It is not these patterns that represent merely in virtue of their similarity to some objects. We have already seen how icons and isomorphic patterns involved in information reproduction systems cannot be representations

            Even if one recognizes that the patterns of activation in sets of units or in layers of neurons cannot serve as representations simply in virtue of their similarity or isomorphism to the objects they represent, one might still be tempted to identify some other part of the process as the representation. In particular, since it seems that it is the connections between the units or layers that actually do all the work, it might be plausible to see them as representations. It is these in which our knowledge is stored, these that allow us to remember, these that allow us to coordinate our motor activities with our visual perceptions. These might be seen as concepts, embodied dispositions to connect certain images, actions, or modes of interaction with the world. Kant's view of concepts as rules by which we connect or synthesize our experience seems similar to this view of representation. On this view,  representations are not the icons or images that we form, but the embodied dispositions or sets of connections that allow us to form these images in our mind. The representation is not the statue the artist makes, but the concept through which the artist creates it, or the concept through which the audience interprets it.

            This view, however, cannot be correct. Concepts, or sets of connections that effect the transformation of one pattern to another, cannot be representations. Representations serve two cognitive functions: (1) They present us with information, or images of the world.
Re-presentations are still presentations. A representation must be present to us in a way that re-presents the world to us. (2) Representations are the bearers of knowledge. Representations are that through which we know; they must be capable of being true or false.  Concepts, so interpreted, can do neither.

            The neural connections that associate the smell of a rose with a visual image of a rose are not present to us. We don't see them in any sense, and they don't re-present to us any information. We see the effects of these connections, but we don't see them. If they are not present to us, they cannot re-present anything to us. (You can see why it was more plausible to identify the patterns of activation as the representations: they, at least, are present to us through the mental images that are associated (or identical) to them.) Note that the problem here is not directly connected to the relationship of consciousness and physical states. Even if we assume that neural states can be present to us (as they might seem to be in the case of the distributed representation of the smell of a rose), there is nothing in the neural connections between units that presents us with anything. The problem is that the connections themselves are not important; it is what they do that is important. This is why we describe them dispositionally, as abilities to perform certain transformations. The physical features that embody these abilities, though necessary for the representing process, are not the representation. The hardware that connects a set of sounds to a pattern of marks on a compact disc is not the reproduction of the sound; we don't see and aren't interested the hardware. We are interested in what it does, in the connection it establishes between the pattern on the disc and the music we hear. In the same way, the connections could be said to do the representing, to do the connecting, but they themselves in isolation from the process are not representations.^[7]

 

IV. Conclusion

            Representation is not iconic resemblance; it is not symbolic isomorphism effected by an algorithm or function; it is not the algorithm, function, or concept that does the connection.  These views all mistake a part of the representing process for the whole, and then run into problems when they try to find within the intrinsic properties of the part, in isolation, properties that it only has in relation to the whole.

            The suggestion I wish to make about what representation is is fairly simple, but has some important consequences. Representation is the process of connecting one mode of interaction with the world to another. It is not the pattern of activation on one set of units, nor the connections through which the transformation is made, that is the representation. It is the process of connecting one pattern to another, the transformation itself, that is the re-presentation of the one pattern by the other. Representation is always a connection between one mode of interaction with the world and another, a projection of one set of properties onto another. It is the re-presentation of one thing by another.

            Note that this implies that the phenomenological interpretation we gave of the PDP model was false. It was assumed that we first feel an olfactory image, when the olfactory units are activated, then we see a visual image after the pattern of activation has been transformed into a pattern on the visual units. I am suggesting that we never perceive simple properties like olfactory images or visual images.

            Talk of perception of simple images is an abstraction; perception is always of connections or projections of one set of properties onto another. We identify separate properties only by abstracting them from the connection in which we find them. We interact with the rose through our sense of smell causing a pattern of activation to arise in a set of olfactory units that immediately causes a set of patterns to arise in the visual units. What we perceive is this process of connection, not any one component of it. We perceive the visual image of the rose as projected onto the smell of the rose; we perceive the smell as re-presenting the visual image. They are perceived together, inseparably. We do not first smell, then see. We see only through the smell; we smell only as re-presenting the sight. The separate images are abstractions separated only after the fact. This is more clear in the case of perceptual representation than in the case of memory. A two dimensional image is projected upon my retinas and this is connected with my knowledge of three dimensional structures, the names they are called, and the other properties associated with them. I see a rose. I see a certain type of object, a rose. I see that it has a certain shape. I see that it is soft. I see that it has a certain fragrance, as the visual image calls this to mind. All of these are part of the perception of the rose. My perception is the projection of these other properties onto my visual interaction with the rose. My perception is the visual re-presentation of the other properties of the rose. I don't first see a two-dimensional shape and then associate it with other images. My perception is the process of connections and cannot be identified with any of the components abstracted from it.

            This intrinsic connection of one part of our conscious experience to other parts is sometimes referred to as Intentionality. The properties of a representation always point beyond themselves to the properties of that which they represent. It is often thought that this property of our conscious experience presents insuperable problems for most models of mental representation. One of the distinct advantages of the model I suggest is that it sees Intentionality as an intrinsic, but not inexplicable, property of acts of representing. Since an act of representing is, and must be, an act of connecting one mode of interaction with the world to another, it is not surprising that our consciousness of these acts involves properties that point beyond themselves to other properties. Representing is a re-presenting of one set of properties by another. Views that see representations as self-contained icons or bits of information will continue to find the intrinsic Intentionality of mental representation a mystery. The problem of Intentionality cannot be solved by building more information into the system. It will be only be solved by recognizing that we must model representation by an activity of connecting bits of information, rather than by the self-contained bits of information themselves. Intentionality is not a mysterious property that the components of our models must somehow acquire; it is something they must do through their activity.

            Of course this cannot be argued for rigorously in the space allotted. Let me close by suggesting some of the epistemological implications such a view might have. The problem with the iconic view was that the icons were both the only way of contacting the world and the bearers of our knowledge. There was no way of telling if they were really similar to reality or not. The images became veils separating us from reality.

            In particular, it was seen that any of our particular images could not reflect or be similar to reality since it was a result of our peculiar constitution. Redness is not in things it is the effect they have on us. There was no way to verify our representations, and there was good reason to think they could not be similar to external objects.

            On the view I am suggesting, particular images or effects the object has on us are not representations. A representation is a process of connecting two such effects, and its content lies in this connection, not in the particular features of the images connected. The content of my perception of the rose as red and as fragrant is not the "feel" of these particular images, but the connections that are established between them by my act of representing. While the particular features of the images reflect only the effect the object has on me, the connections within my representation can reflect the connections between the real dispositions in the object that give rise to these effects.  While there is no determinate isomorphism between individual icons and the world, there can be an isomorphism between the connections made between modes of interaction within an act of representing and the properties in the object that participated in the  modes of interaction that are connected. Connections can be objective while images cannot.

            Further, if representation is seen as a process of connecting different modes of interaction with the world, the veil of ideas disappears. A representation is not an internal icon that is presented to the mind's eye. A representation is an act in which various aspects of our interaction with the world are connected. It is an organized means of interacting with the world. On the view I suggest, representations can serve their cognitive function. They can become windows rather than veils, and the mind a vista onto the world rather than a windowless room.