MATHEMATICAL THEORY OF COMMUNICATION 



409 



If we identify .v as the output of the source, y as the received signal and z 

 as the signal sent over the correction channel, then the right-hand side is the 

 equivocation less the rate of transmission over the correction channel. If 

 the capacity of this channel is less than the equivocation the right-hand side 

 will be greater than zero and Hyz{x) > 0. But this is the uncertainty of 

 what was sent, knowing both the received signal and the correction signal. 

 If this is greater than zero the frequency of errors cannot be arbitrarily 

 small. 

 Example: 



Suppose the errors occur at random in a sequence of binary digits : proba- 

 bility p that a digit is wrong and q = 1 — p that it is right. These errors 

 can be corrected if their position is known. Thus the correction channel 

 need only send information as to these positions. This amounts to trans- 



SOURCE TRANSMITTER RECEIVER 



Fig. 8 — Schematic diagram of a correction system. 



CORRECTING 

 DEVICE 



mitting from a source which produces binary digits with probability p for 

 1 (correct) and q for (incorrect). This requires a channel of capacity 



-Iplogp + q log q] 



which is the equivocation of the original system. 



The rate of transmission R can be written in two other forms due to the 

 identities noted above. We have 



R = H{x) - Hy{x) 



= H{y) - H.{y) 



= H{x) + H{y) - H{x, y). 



The first defining expression has already been interpreted as the amount of 

 information sent less the uncertainty of what was sent. The second meas- 



