INCESSANT TRANSMISSION 9/21 



measure for the amount of information lost, but this interpretation 

 leads to nonsense. Thus if, in the same transmission, the line were 

 actually cut and the recipient simply tossed a coin to get a "message" 

 he would get about a half of the symbols right, yet no information 

 whatever would have been transmitted. Shannon has shown 

 conclusively that the natural measure is the equivocation, which is 

 calculated as follows. 



First find the entropy over all possible classes: 



-0-495 log 0-495 -0-005 log 0-005 

 -0-005 log 0-005 -0-495 log 0-495 



Call this Hi; it is 1-081 bits per symbol. Next collect together the 

 received signals, and their probabilities; this gives the table 



Symbol received 1 



Probabihty 0-5 0-5 



Find its entropy: 



-0-5 log 0-5 -0-5 log 0-5 



Call this H2. It is 1 000 bits per symbol. Then the equivocation 

 is Hi — H2: 0-081 bits per symbol. 



The actual rate at which information is being transmitted, allow- 

 ance being made for the effect of noise, is the entropy of the source, 

 less the equivocation. The source here has entropy 1 -000 bits per 

 symbol, as follows from: 



Symbol sent 1 



Probabihty 0-5 0-5 



So the original amount supphed is 1-000 bits per symbol. Of this 

 0-919 gets through and 0-081 is destroyed by noise. 



Ex. I : What is the equivocation of the transmission of S.9/19, if all nine combina- 

 tions of letters occur, in the long run, with equal frequency ? 



Ex. 2: (Continued.) What happens to the equivocation if the first input uses 

 only the symbols B and C, so that the combinations BE, BF, BG, CE, 

 CF, CG occur with equal frequencies? Is the answer reasonable? 

 *Ex. 3: Prove the following rules, which are useful when we want to find the 

 value of the expression —p \ogaP, and p is either very small or very near 

 to 1: 



(i) If /J = xy, - plogaP = - xyiloga x + log,, y); 



mifp = io-',-piogaP = ^^ ^° ' 



logio a 



P = 



189 



1 o2 



(iii) Ifjj is very close to l,put \ — p — q,and — p logaP = (q — -r...). 



