626 BELL SYSTEM TECHNICAL JOURNAL 



particular function in the set.^ Roughly speaking, each function can be ex- 

 pected, as time progresses, to go through, with the proper frequency, all the 

 convolutions of any of the functions in the set. 



Just as we may perform various operations on numbers or functions to 

 obtain new numbers or functions, we can perform operations on ensembles 

 to obtain new ensembles. Suppose, for example, we have an ensemble of 

 functions /a(0 and an operator T which gives for each function /«(/) a result 

 g.(0: 



gc{l) = TUt) 



Probability measure is defined for the set ga{l) by means of that for the set 

 Ja{t)' The probability of a certain subset of the ga{t) functions is equal 

 to that of the subset of the /«(/) functions which produce members of the 

 given subset of g functions under the operation T. Physically this corre- 

 sponds to passing the ensemble through some device, for example, a filter, 

 a rectifier or a modulator. The output functions of the device form the 

 ensemble ga{t). 



A device or operator T will be called invariant if shifting the input merely 

 shifts the output, i.e., if 



gM) = TUt) 

 implies 



gait -F h) = TUt + /l) 



for all/a(0 and all h . It is easily shown (see appendix 1) that if T is in- 

 variant and the input ensemble is stationary then the output ensemble is 

 stationary. Likewise if the input is ergodic the output will also be ergodic. 



A filter or a rectifier is invariant under all time translations. The opera- 

 tion of modulation is not since the carrier phase gives a certain time struc- 

 ture. However, modulation is invariant under all translations which are 

 multiples of the period of the carrier. 



Wiener has pointed out the intimate relation between the invariance of 

 physical devices under time translations and Fourier theory. He has 



3 This is the famous ergodic theorem or rather one aspect of this theorem which was 

 proved is somewhat different formulations by Birkhoff, von Neumann, and Koopman, and 

 subsequently generalized by Wiener, Hopf, Hurewicz and others. The literature on ergodic 

 theory is quite extensive and the reader is referred to the papers of these writers for pre- 

 cise and general formulations; e.g., E. Hopf "Ergodentheorie" Ergebnisse der Mathematic 

 und ihrer Grenzgebiete, Vol. 5, "On Causality Statistics and Probability" Journal of 

 Mathematics and Physics, Vol. XIII, No. 1, 1934; N. Weiner "The Ergodic Theorem" 

 Duke Mathematical Journal, Vol. 5, 1939. 



* Communication theory is heavily indebted to Wiener for much of its basic philosophy 

 and theory. His classic NDRC report "The Interpolation, Extrapolation, and Smoothing 

 of Stationary Time Series," to appear soon in book form, contains the first clear-cut 

 formulation of communication theory as a statistical problem, the study of operations 



