What is Information Theory? 25 



with transmission of information from one point to another. 

 Suppose that now we are not transmitting information, but are 

 organizing a set of elements in a particular pattern or configura- 

 tion. What is the amount of information obtained by the system 

 in the transition from one state to the other? 



Saltzberg: Whether we talk about the entropy of thermo- 

 dynamics or the information in a message, we are in principle 

 talking about organization, or, more precisely, about the prob- 

 abilities of the various arrangements of the component parts of 

 the system. The second law of thermodynamics states that entropy 

 must increase or, at best, remain constant, which is another way 

 of saying that the system is becoming more disorganized or ran- 

 dom. In communication systems, however, upon the receipt of 

 information, the disorganized or uncertain state of our knowledge 

 becomes more certain or better organized; therefore, we may 

 consider received information as negative entropy since it increases 

 the organization of the receiver. 



Gregory Bateson (Palo Alto, California) : You separated rather 

 clearly the notion of measuring cjuantity of information from the 

 notion of ""meaning'' of the information measured, but it appears 

 to me that this becomes difficult when we have a secjuence of 

 items comprising a total message and so related that some of these 

 items reflect upon the significance of other items in the sequence. 

 In this case, the meaning of these meta signals is a very important 

 part of the whole economics of communication. 



Saltzberg: I think your question refers to the very strong con- 

 straints which may exist between the elements of a signal. These 

 constraints are essentially the transition probabilities and the 

 intersymbol influences which are referred to in information theory. 

 We do not look upon a knowledge of these transition probabilities 

 as implying meaning in the sense that we talk about semantic 

 meaning. In other words, the constraints between a sequence of 

 elements comprising a signal are accounted for in measuring 

 information, but the importance of the signal, i.e., its semantic 

 value, is not. 



Bernard S. Patrick (Memphis, Tennessee): Can you speak 

 for just a moment about redundancy or the use of redundancy in 

 communication systems? 



