32 Henry Quastler 



The last column, Hj{x), is the uncertainty concerning the height of the daughter 

 if the height of the father is known ; it is not too surprising to find this uncertainty 

 smallest in the extreme cases, and always smaller than the unconditional 

 uncertainty of 1.92 bits. 



The father's height 'communicates' some information about the daughter's 

 height; the amount communicated is 0.22 bits. It is not more than that for a 

 number of reasons. Some of the deficit in information about the daughter's 

 height is undoubtedly due to ignorance, and could be reduced by taking proper 

 account of various concomitant factors. Some of the uncertainty may be 

 irreducible, due to a truly random process — possibly the selection of the particu- 

 lar chromosomes which go into determining the daughter's height. In the strict 

 sense, the term 'noise' is reserved for the effects of random disturbances, and 

 not to the eff"ects of ignorance. However, the problem of the final distinction 

 between uncertainty due to randomness and uncertainty due to ignorance is 

 an extremely delicate one; the practical information analyst will usually be 

 satisfied to treat any uncertainty as due to noise, which results in the greatest 

 reduction of certainty. This interpretation will be subject to revision in the 

 light of additional knowledge. 



The two-part system 'father's height-daughter's height' is not a communica- 

 tion system, and this is one reason why so little information is transmitted. 

 Suppose the numbers which define the 'father's heights' categories were not 

 observed in a given population but could be chosen arbitrarily; for instance, 

 they might be input voltages applied to a system. Accordingly, the 'daughters' 

 heights' might be output voltages, and the table of conditional probabilities 

 becomes a statement of the transfer function of the system. It is obvious that 

 this system can be made to transmit more than 0.22 bits per symbol. For instance, 

 using onlyy = 59.5 andy = 74.5, with equal frequencies, one would transmit 

 about .90 bits per signal. In general: for each channel, Piij), there exists a set 

 of input probabilities, p(i), which maximizes the transmission rate. The rate so 

 obtained is called the channel capacity. 



Even with best utihzation of the possibilities of a channel, it can do no more 

 than transmit all the input information, and in general it will not transmit quite 

 all of it. This leads to an important generalization : Manipulation of information 

 cannot increase its amount; it can at best preserve it, and it is likely to reduce it. 



This important statement will be clarified by the discussion of an apparent 

 exception. Suppose A wishes to send a message to B over the channel C; 

 conditions being very good, B picks up not only almost perfectly the message 

 sent by A but acquires, in the course of doing so, considerable amount of 

 information about conditions in the channel. His total information received 

 might be more than that contained in A's message; still, he has lost some of the 

 information contained in the message. In general: as a result of manipulating 

 information, there can be more output information than there was input 

 information — but the contribution of the input information to the total cannot 

 be more than the amount of input information. 



Error Detection and Correction 



A codebook states which output should be associated with any given input. 

 A noise-free channel fulfills these requirements perfectly. In a noisy channel 



