40 Henry Quastler 



who wishes to use this knowledge in his field is bound to run into some diffi- 

 culties. A typical difficulty is that a natural situation does not present itself 

 neatly classified with a complete set of categories and probability measures. 

 It often takes considerable ingenuity to supplement the missing components 

 of the picture. Wherever ingenuity must be used, the result will not be unequi- 

 vocal. Hence it becomes important to estimate not individual information 

 measures but rather whole ranges compatible with reasonable assumptions. 



The Relativity of Information Measures 



'Information content' is a measurable quantity, just as length; and, just 

 as length, it is a function and not a property of a particular set of events. The 

 theory of relativity asserts that the measured length of an object depends on 

 certain relations between the object and the measuring system. However, 

 under everyday conditions these relations will not produce any significant 

 effect and, most of the time, lengths behave as if they were properties of objects. 

 The infomiation content of an event depends on the manner in which this 

 event is related to the frame of reference of the evaluating system. Unlike 

 with length, these relations are not fixed under everyday conditions. Therefore, 

 information content behaves only rarely as if it were a property of an event. 



The amount of information, H{x), associated with an event, x, is defined 

 as the expectation of the logarithm of the probability that x will fall into some 

 category, /. Thus, the measure of information depends on three decisions: 



(1) the choice of a unit event, 



(2) the establishment of categories, 



(3) the selection of a set of probabihty measures. 



In general, each of these decisions involves a degree of arbitrariness. Accor- 

 dingly, a considerable range of information measures will be compatible with 

 a given real situation. 



The question of an appropriate selection of a unit event cannot be solved 

 by mechanical application of hard and fast rules. There is a lower limit to 

 the size of elements, imposed by limits of observability. In general, selection 

 of these lower limits will force one to take cognizance of a tremendous amount 

 of detail, most of which is bound to be irrelevant. Thus, one will try to select 

 a unit event broad enough that all irrelevant details are submerged in its internal 

 structure, yet narrow enough so that no relevant relations get lost within the 

 unit event. In practice, one has to make a guess, subject to revision by later 

 experience. This difficulty occurs with all kinds of analyses, and is not specific 

 to informational analysis. 



The situation is quite similar with respect to categories. There, too, exists 

 a bound, imposed by the capabiHties of discrimination. In general a large 

 number of discriminations can be made which are irrelevant to the problem 

 at hand. For instance, if one deals with the semantic content of a printed 

 message, it will be quite irrelevant to categorize by shapes of letters, quality 

 of paper, type of printing ink, etc. The decision is not always so easy. For 

 instance, in categorizing the atoms found in living matter it will, by and large, 

 not be necessary to distinguish between isotopes; in the overwhelming majority 

 of occasions, differences between isotopes will have no effect. Occasionally, of 



