536 BELL SYSTEM TECHNICAL JOURNAL 



The Measurement of Information 



When we speak of the capacity of a system to transmit information 

 we imply some sort of quantitative measure of information. As 

 commonly used, information is a very elastic term, and it will first be 

 necessary to set up for it a more specific meaning as applied to the 

 present discussion. As a starting place for this let us consider what 

 factors are involved in communication; whether conducted by wire, 

 direct speech, writing, or any other method. In the first place, there 

 must be a group of physical symbols, such as words, dots and dashes or 

 the like, which by general agreement convey certain meanings to the 

 parties communicating. In any given communication the sender 

 mentally selects a particular symbol and by some bodily motion, as 

 of his vocal mechanism, causes the attention of the receiver to be 

 directed to that particular symbol. By successive selections a 

 sequence of symbols is brought to the listener's attention. At each 

 selection there are eliminated all of the other symbols which might 

 have been chosen. As the selections proceed more and more possible 

 symbol sequences are eliminated, and we say that the information 

 becomes more precise. For example, in the sentence, "Apples are 

 red," the first word eliminates other kinds of fruit and all other objects 

 in general. The second directs attention to some property or condition 

 of apples, and the third eliminates other possible colors. It does 

 not, however, eliminate possibilities regarding the size of apples, and 

 this further information may be conveyed by subsequent selections. 



Inasmuch as the precision of the information depends upon what 

 other symbol sequences might have been chosen it would seem reason- 

 able to hope to find in the number of these sequences the desired 

 quantitative measure of information. The number of symbols 

 available at any one selection obviously varies widely with the type 

 of symbols used, with the particular communicators and with the 

 degree of previous understanding existing between them. For two 

 persons who speak different languages the number of symbols available 

 is negligible as compared with that for persons who speak the same 

 language. It is desirable therefore to eliminate the psychological 

 factors involved and to establish a measure of information in terms 

 of purely physical quantities. 



Elimination of Psychological Factors 



To illustrate how this may be done consider a hand-operated 

 submarine telegraph cable system in which an oscillographic recorder 

 traces the received message on a photosensitive tape. Suppose the 



