418 BELL SYSTEM TECHNICAL JOURNAL 



Suppose the symbols fall into several groups such that the noise never 

 causes a symbol in one group to be mistaken for a symbol in another group. 

 Let the capacity for the wth group be Cn when we use only the symbols 

 in this group. Then it is easily shown that, for best use of the entire set, 

 the total probability Pn of all symbols in the wth group should be 



P = 



Within a group the probability is distributed just as it would be if these 

 were the only symbols being used. The channel capacity is 



C = log22''\ 

 17. An Example of Efficient Coding 



The following example, although somewhat unrealistic, is a case in which 

 exact matching to a noisy channel is possible. There are two channel 

 symbols, and 1, and the noise affects them in blocks of seven symbols. A 

 block of seven is either transmitted without error, or exactly one symbol of 

 the seven is incorrect. These eight possibilities are equally likely. We have 



C = Max [H(y) - H.{y)] 



= i[7 + flog|] 



= y bits/symbol . 



An efficient code, allowing complete correction of errors and transmitting at 

 the rate C, is the following (found by a method due to R. Hamming) : 



Let a block of seven symbols be Xi, X^, . . . X'l. Of these X's, X^, Xq and 

 X7 are message symbols and chosen arbitrarily by the source. The other 

 three are redundant and calculated as follows : 



X4 is chosen to make a = X4 -\- X^ -\- Xq -\- X7 even 



X2 " " " " ^ = X2 + X3 + Xg + X, - 



Xi " " " '' y = Xi + X, + X5 + X: " 



When a block of seven is received a, ^ and 7 are calculated and if even called 

 zero, if odd called one. The binary number a ^ y then gives the subscript 

 of the Xi that is incorrect (if there was no error). 



APPENDIX 1 



The Growth of the Number of Blocks of Symbols With A 

 Finite State Condition 



Let Ni(L) be the number of blocks of symbols of length L ending in state 

 /. Then we have 



NAP = Z NU - b\^) 



