746 THE BELL SYSTEM TECHNICAL JOURNAL, JULY 1952 



In terms of these sums, we have 



Hi = \og^ -Kj2Si\ogSi. 



Since, with the assumed distribution, the Si are all nearly equal very 

 little reduction in channel capacity is achieved by this step. 



With linear prediction, the modes of the distributions {i = j) could 

 be centered at zero merely by sending the difference between the present 

 and previous sample (previous value prediction). This would give a 

 reduced signal whose distribution may be found by adding the array 

 along the diagonals. The required channel capacity is then given by: 



H, = -Kf log Ki - 2i: ^:ii:^ log ^^^^ 





The distribution of the signal from a digrammer is found by rearrang- 

 ing each row of the table in order of decreasing probability and then 

 adding the resulting columns. Call these sums Sd . The digrammer out- 

 put will thus require a channel capacity: 



Ho = -T.KSa log KSd 



1 ' 



= log j^ - K^Sd log Sd 



K d=i 



Lastly, the true rate of the source is given by 

 H = -Yl Pii) S ViU) log piij) 



= log i, - //i + 2/v i: ^^ k log a 



Values for the above quantities were computed for a = 2 and n = 

 2, 3, 4, 6, 8, 16, 32, 00 . For the case of a = 2, we find that 



K = [3f - 4(1 - 2"')]"' 



and that as ^ -^ 00^ 



Hl,Hd,H -^i ^ log2 3 = 2.918 bits. 



The results are shown in the Table I and also are plotted in Fig. 9. 



While Ho and Hi increase without limit as ( is increased, H^ , Hd, 

 and H quickly approach a definite limit. This limit exists because we 

 assumed that the decrease in joint probability as a function of number 



