A Primer on Information Theory 7 



a scale. The slide rule, thus, is an early and modest example of an information- 

 handling device. At present, machines exist which accept and store considerable 

 symbolic information concerning data and operations, and which will execute 

 a wide range of manipulation with this information. There is good reason to 

 believe that machines will actually be built which can compute any number 

 that can be computed, and which, even more generally, can arrive at the results 

 of any thinking which can be described by explicitly-defined operations. 



In the situations so far mentioned the relation between the original and the 

 translated (coded) intelligence was based on a mutual agreement. Translation 

 of information is not restricted to such a situation; it may be based on a 

 one-sided choice of code, one not explained to the receiver. This occurs in the 

 case of conditioned reflexes: each time a dog is fed, a bell rings; after some 

 time, the information 'the bell is ringing' comes to represent the infonnation 

 that 'food is about to be served', and leads to preparations for eating. The 

 code is established by the experimenter; the dog is not consulted. In fact, the 

 representation of information does not have to be at the level of conscious 

 awareness in any way. For instance : the system which regulates one's breathing 

 and thereby his oxygen intake has come to depend for its regulation not on the 

 oxygen content of the blood itself, but on the concentration of carbon dioxide. 

 Ordinarily, the COg level in the blood is a reliable representation of the O2 

 level; under certain conditions the representation ceases to be correct, and 

 then difficulties can occur. In all of these cases, just as in those with arbitrarily- 

 fixed codes, information theory is concerned with the general laws which 

 govern the possibility of translating one kind of information into another; it 

 will be obvious by now to the reader that the term 'information' in the technical 

 sense covers a good deal more than in everyday language. 



^ReaV and 'Symbolic' Events 



We now turn to a more formal and general discussion of the principles of 

 representation of information. We will deal here only with information that 

 occurs in discrete units; however, the transition to treatment of the continuous- 

 function type of information would not be difficult. 



In our discussions, the terms real and symbolic will replace the cumbersome 

 expressions 'something to be represented' and 'something representing'. It will 

 be remembered that 'real' and 'symbolic' refer not to properties of things, but 

 to their functions in a given situation, and that the term 'symbolic' may but 

 does not necessarily imply that a conscious act of symbolization has occurred. 

 The 'symbolization' of the need for oxygen by the carbon dioxide level illustrates 

 that 'symbolic' is here used in a wider sense than is customary. 



'Infonnation' is not a disembodied something; it is always related to some 

 actual carrier — a thing or an event. We will use whichever word is appropriate 

 in a given situation; the word event is most frequently used as a generic term. 

 However, it must always be remembered that other terms could be substituted ; 

 information theory applies equally to all kinds of carriers of information. In 

 formal language, one refers to the information carriers as elements of discourse, 

 or points in sample space, or configurations of properties. 



A concrete event, in all its richness of detail, is not amenable to complete 

 representation. The only complete representation of a particular man, at a 



