MATHEMATICAL THEORY OF COMMUNICATION m 



where bij , ba , . . . 6Ty are the length of the symbols which may be chosen 

 in state / and lead to state j. These are linear difference equations and the 

 behavior as Z -^ x must be of the type 



Substituting in the difference equation 



i,S 



or 



Aj = ZA,W->''<i' 



is 



E (Z Ty-"'*' - ««)^.- = 0. 



i S 



For this to be possible the determinant 



s 



must vanish and this determines W, which is, of course, the largest real root 

 oiD = 0. 



The quantity C is then given by 



C = Lim ^ ' = log \] 



L-*oc L 



and we also note that the same growth properties result if we require that all 

 blocks start in the same (arbitrarily chosen) state. 



APPENDIX 2 



Derivation oy H = —Z pi log pi 



Let //(-,-,•••,-]= A{ii). From condition (3) we can decompose 



a choice from s'" equally likely possibilities into a series of m choices each 

 from 5 equally likely possibilities and obtain 



A (s^) = m A{s) 



Similarly 



A{0 = nA{t) 



We can choose n arbitrarily large and find an m to satisfy 



5" < /" < /"-^'^ 



