MATHEMATICAL THEORY OF COMMUNICATION 655 



This should equal 



|[floglMo|-i2^.v^..,] 



which requires A ij = -^ C,y . 



TJ 



In this case A ij = -— B^j and both equations reduce to identities. 

 H2 



APPENDIX 7 



The following will indicate a more general and more rigorous approach to 

 the central definitions of communication theory. Consider a probability 

 measure space whose elements are ordered pairs (x, y). The variables x, y 

 are to be identified as the possible transmitted and received signals of some 

 long duration T. Let us call the set of all points whose x belongs to a subset 

 Si of X points the strip over Si , and similarly the set whose y belongs to S2 

 the strip over S2 . We divide x and y into a collection of non-overlapping 

 measurable subsets Xi and Yi approximate to the rate of transmission R by 



^■ = ^i:P(A-..,F..)log^g^^ 



where 



P(Xi) is the probability measure of the strip over Xi 

 P{ Y^) is the probability measure of the strip over Yi 

 P{Xi, Yi) is the probability measure of the intersection of the strips. 



A further subdivision can never decrease Ri . For let Xi be divided into 

 Xi= X[ + Xi and let 



P{Yi) = a P{Xi) = b + c 



P(X[) = b P(X[, Yi) = d 



P{Xi) = c P{Xi, Yi) = e 



P(Xi, Yi) = d-\-e 



Then in the sum we have replaced (for the Xi, Yi intersection) 



{d + e) log .,.\ by dlog- -\- e log - . 

 a{o -f- c) ao ac 



It is easily shown that with the limitation we have on h, c, d, e, 



d'e' 



m 



- b^c 



