420 



BELL SYSTEM TECHNICAL JOURNAL 



Thus, taking logarithms and dividing by n log 5, 



m ^ log t ^ m \ 



n ~ log s ~ n n 



or 



logj 

 log 5 



< € 



where € is arbitrarily small. 



Now from the monotonia property of A (n) 



mAis) < nA{t) <{m^ \) A{s) 

 Hence, dividing by nA{s), 



^ A{t) 



^< 



A{t) ^m^\ 

 A{s) ~ n n 



or 



< € 



A{t) 



A{s) 



logj 



log 5 



< 26 



^W = -iiTlog; 



where K must be positive to satisfy (2) . 



Now suppose we have a choice from n possibilities with commeasurable prob- 



abihties pi = — -^ where the Ui are integers. We can break down a choice 



Zifli 



from Swt possibilities into a choice from n possibilities with probabilities 

 pi. . . Pn and then, if the ith was chosen, a choice from tii with equal prob- 

 abilities. Using condition 3 again, we equate the total choice from 2wi 

 as computed by two methods 



K log Swi = H(pi , . . . , Pn) + K^ pi log m 



Hence 



H == K\X pilogXm- i: pi log n^] 



= -K^PiXog^ = -i^S^log^-. 



If the pi are incommeasurable, they may be approximated by rationals and 

 the same expression must hold by our continuity assumption. Thus the 

 expression holds in general. The choice of coefficient iiT is a matter of con- 

 venience and amounts to the choice of a unit of measure. 



APPENDIX 3 



Theorems on Ergodic Sources 



If it is possible to go from any state with i^ > to any other along a path 

 of probability /> > 0, the system is ergodic and the strong law of large num- 

 bers can be applied. Thus the number of times a given path pa in the net- 



