628 BELL SYSTEM TECHNICAL JOURNAL 



than or equal to E correspond to points in a 2TW dimensional sphere with 

 radius r = \/2WE. 



An ensemble of functions of limited duration and band will be represented 

 by a probability distribution p{xi • • • Xn) in the corresponding n dimensional 

 space. If the ensemble is not limited in time we can consider the 2TW co- 

 ordinates in a given interval T to represent substantially the part of the 

 function in the interval T and the probability distribution p(xi , • ■ • , Xn) 

 to give the statistical structure of the ensemble for intervals of that duration. 



20. Entropy of a Continuous Distribution 



The entropy of a discrete set of probabilities pi, • ■ ■ pn has been defined as : 



^ = -Z^log^-. 



In an analogous manner we define the entropy of a continuous distribution 

 •with the density distribution function p{x) by: 



p(x) log p(x) dx 



00 



With an n dimensional distribution p{xi , • • • , Xn) we have 



H = — j • • • / p(xi - ' ■ Xn) log p{xi , " • ,Xn) dX] 



dXr 



If we have two arguments x and y (which may themselves be multi-dimen- 

 sional) the joint and conditional entropies of p(x, y) are given by 



H{x, y) = -jj p{x, y) log p{x, y) dx dy 



and 



where 



H.(y) = -fj p{x, y) log ^-^ dx dy 

 Hyix) = -fj p{x, y) log ^^ dx dy 



p(x) = j p(x, y) dy 



P(y) = j P(x, y) dx. 



The entropy of continuous distributions have most (but not all) of the 

 properties of the discrete case. In particular we have the following: 



