422 BELL SYSTEM TECHNICAL JOURNAL 



Let (["j be the lengths of the various symbols that can occur in passing from 

 state i to state j. What distribution of probabilities Pi for the different 

 states and p^ij for choosing symbol s in state i and going to state ^ maximizes 

 the rate of generating information under these constraints? The constraints 

 define a discrete channel and the maximum rate must be less than or equal 

 to the capacity C of this channel, since if all blocks of large length were 

 equally likely, this rate would result, and if possible this would be best. We 

 will show that this rate can be achieved by proper choice of the Pi and p% . 

 The rate in question is 



:^P,i)P^t^ M' 



Let (ij = zl ^i]- • Evidently for a maximum p\]- = k exp (^i]. The con- 



s 



straints on maximization are 2Pi = \,zli Pa = ^, ^ Piipij — 8 a) = 0. 

 Hence we maximize 



2/X t Pij iij i 



a^. - ~ Mi + X + M.- + ViPi - 0. 



Solving for pij 



Since 



Pij = AiBjD ^'\ 

 BjD-Ui 



pij 



Zb.d-^^' 



The correct value of D is the capacity C and the Bj are solutions of 



Bi = X BjC~^'^ 



for then 



Bj (,. 

 ^'' Bi 



2Pi^C-^'' = P, 

 or 



