What is Information Theory? 21 



another aspect of information theory which I have not mentioned 

 before. Information theory is concerned with the properties of 

 ensembles of messages or objects. One of the properties which is 

 quite important in scientific analysis is known as ergodicity. 

 In analyzing" many problems, the assumption of ergodicity is one 

 that is a practical necessity rather than a statement of fact relative 

 to the nature of things. However, this simplifies analysis in that 

 it allows one to examine a long time sample of one of the mem- 

 bers of an ensemble and to conclude from this that he knows 

 something about the statistics of the ensemble. This is not always 

 true since, for example, it would imply that the statistics associated 

 with the EEG record of a single individual apply equally well to 

 another subject. If this were the case, then an ensemble of messages 

 composed of the EEG's of many subjects would be an ergodic 

 ensemble. In testing engineering components, one ordinarily takes 

 a single component and tests it for a long period of time and then 

 draws implications about the behavior of all similar components. 

 This is an aspect of statistical analysis and information theory 

 which, when applied to the life sciences, creates a great many 

 problems since one may not be aware that this assumption may 

 underly the mathematical formulation of certain problems. 



Herman Blustein (Chicago, Illinois): How do you determine 

 the validity of the samples when you analyze the EEG's in this 

 manner and make a generalization from them? 



Saltzberg: The validity of the sample is not the question here. 

 For example, an EEG record may be sufficiently long to give you 

 a good estimate of its properties for a particular individual. How- 

 ever, unless EEG's of different individuals are statistically similar, 

 or ergodic, this does not allow you to draw any conclusions about 

 the properties of another individual's EEG record. 



Harold W. Shipton (Iowa City, Iowa) : The way the discus- 

 sion is going means, I think, that we have to say a little more 

 about the properties of noise, because when we deal with formal 

 information theory we use "noise" in exactly the way that we 

 used to use the phrase "Brownian movement." This is quite 

 acceptable. However, when we perform an experiment, we are 

 dealing with band limited noise, and we are also probably dealing 

 with nonrandom perturbations in the system. I would like to hear 



