416 BELL SYSTEM TECHNICAL JOURNAL 



Note how this checks the obvious values in the cases p = \ and p = \ . 

 In the first, jS = 1 and C = log 3, which is correct since the channel is then 

 noiseless with three possible symbols, li p = ^, ^ = 2 and C = log 2. 

 Here the second and third symbols cannot be distinguished at all and act 

 together like one symbol. The first symbol is used with probability P = 

 J and the second and third together with probabihty J . This may be 

 distributed in any desired way and still achieve the maximum capacity. 



For intermediate values of p the channel capacity will lie between log 

 2 and log 3. The distinction between the second and third symbols conveys 

 some information but not as much as in the noiseless case. The first symbol 

 is used somewhat more frequently than the other two because of its freedom 

 from noise. 



16. The Channel Capacity in Certain Special Cases 



If the noise affects successive channel symbols independently it can be 

 described by a set of transition probabilities pij . This is the probability, 

 if symbol i is sent, that j will be received. The maximum channel rate is 

 then given by the maximum of 



2] Pi pij log 2] Pipij - J^ Pi pij log pij 



i,]' i i,j 



where we vary the Pi subject to XPi = 1. This leads by the method of 

 Lagrange to the equations, 



2 Psj log ^ p. = M 5 = 1, 2, • • • . 



i 



Multiplying by P« and summing on s shows that /i = — C. Let the inverse 

 of pgj (if it exists) be hst so that zl ^stpsj = 8tj . Then: 



Hence; 



or, 



2 ^st psj log psj — log Yl Pipit = —cY^ht. 



s,j i « 



2 Pipit = exp [C Yhst + Y hst psj log psj] 



i 8 8,j 



Pi = 12 hit exp [C Yjhsf\-Yj hst psj log psjY 



This is the system of equations for determining the maximizing values of 

 Pi , with C to be determined so that ^ Pi = 1. When this is done C will be 

 the channel capacity, and the P< the proper probabilities for the channel 

 symbols to achieve this capacity. 



