MATHEMATICAL THEORY OF COMMUNICATION 635 



where the fi are equally spaced through the band W. This becomes in 

 the Hmit 



expl£log|F(/)fd/. 



Since J is constant its average value is this same quantity and applying the 

 theorem on the change of entropy with a change of coordinates, the result 

 follovvs. We may also phrase it in terms of the entropy power. Thus if 

 the entropy power of the first ensemble is Ni that of the second is 



iV,expi/^log|F(/)frf/. 



The final entropy power is the initial entropy power multiplied by the geo- 

 metric mean gain of the filter. If the gain is measured in db, then the 

 output entropy power will be increased by the arithmetic mean db gain 

 over W. 



In Table I the entropy power loss has been calculated (and also expressed 

 in db) for a number of ideal gain characteristics. The impulsive responses 

 of these filters are also given for W = It, with phase assumed to be 0. 



The entropy loss for many other cases can be obtained from these results. 



For example the entropy power factor — for the first case also applies to any 



gain characteristic obtained from 1 — w by a measure preserving transforma- 

 tion of the CO axis. In particular a linearly increasing gain G(aj) = co, or a 

 ''saw tooth" characteristic between and 1 have the same entropy loss. 



The reciprocal gain has the reciprocal factor. Thus - has the factor e^. 



Raising the gain to any power raises the factor to this power. 



22. Entropy or the Sum of Two Ensembles 



If we have two ensembles of functions /«(/) and g^{t) we can form a new 

 ensemble by "addition." Suppose the first ensemble has the probability 

 density function p(xi , - • ■ , Xn) and the second q{xi , • • • , Xn). Then the 

 density function for the sum is given by the convolution: 



r{xi , . . . , a;„) = j • • • j p{yi , • • • , y„) 



• q{xi — Ji, •■ ■ ,Xn — Jn) dyi,dy2, " ■ ,dyn. 



Physically this corresponds to adding the noises or signals represented by 

 the original ensembles of functions. 



