MATHEMATICAL THEORY OF COMMUNICATION 415 



15. Example of a Discrete Channel and Its Capacity 



A simple example of a discrete channel is indicated in Fig. 11. There 

 are three possible symbols. The first is never affected by noise. The second 

 and third each have probability p of coming through undisturbed, and q 

 of being changed into the other of the pair. We have (letting a = — [p log 



TRANSMITTED 

 SYMBOLS 



P 



Fig. 11 — Example of a discrete channel. 



p -\- q log q\ and P and Q be the probabiUties of using the first or second 

 symbols) 



Hix) = -PlogP - 2(3 log G 



Hy{x) = 2Qa 



We wish to choose P and Q in such a way as to maximize H{x) — Hy(x), 

 subject to the constraint P -\- 2Q = 1. Hence we consider 



U = -P log P - 2(3 log Q - 2Qa + X(P + 2Q) 



dV 



Eliminating \ 



^^ = -1 -logP + X = 

 ^_£= -2 -21og(3-2a+2X = 



log P = \ogQ + a 

 P = Qe' = Q^ 



P = 



i8 + 2 ^ /3 + 2* 

 The channel capacity is then 



C=log-^. 



