A Primer on Information Theory 39 



there occurs a change from a rigorous and complete exposition to a heuristic 

 and incomplete one. 



The situation is similar in the study of organization by information theory. 

 Each time a new part is added to a system, a qualitatively new information 

 function appears. As long as one deals with a single variable, the problem 

 is one of efficient use of existing variations. A two-part system introduces 

 relations between parts; a three-part system, relations between relations; a 

 four-part system, relations between a part and a complex of relations. 



Unitization — It is an empirical fact that when a system is complex enough 

 to require very many components, the phenomenon of unitization occurs. 

 That is, some components get organized in such a way that they interact strongly 

 among each other, and act as a unit with respect to the remainder of the system 

 and the external world. Unitization seems to be a necessary evil; it might be 

 an important key for the study of complex organization and complex mental 

 activities. The phenomenon has never been really explained; it is possible 

 that a quantitative treatment will be made possible through the use of infor- 

 mation theory (18). 



Unitization is always coupled with the phenomenon of limited span. Any 

 real part has a limited information content. In any single act of communication, 

 the capacity for non-redundant transmission of a part is limited by its own 

 infomiation content. This amount must somehow be partitioned into inter- 

 action with the external world, and interaction with the other members of the 

 unit. If each of these interactions is to be of significant size, then only a limited 

 number is possible. The interaction of a unit with the outside may be only 

 a fraction of the information traffic within the unit. Hence, several units can 

 be organized into a secondary structure of greater versatility, and this process 

 can be repeated on successive levels of organization. 



There appears, thus, a possibility that information theory can be helpful 

 in formulating both the causes and the effects of unitization, and in establishing 

 rational interpretations of the size of the units. This would be a very important 

 contribution to any theory of organization. 



Conclusion — We have proceeded from simple processes of representation 

 to discussions of communication and, finally, organization. It was attempted 

 to treat in a heuristic and perspicuous manner the basic principles of Information 

 Theory: there exists a generalized concept of 'information' which includes 

 communication and organization and is so general that every real event or 

 structure has its informational aspects; this general concept is related to a 

 measurable quantity; the operation of taking a measurement of this quantity 

 is done by means of symbolization in a standard language. The functions 

 as defined obey two fundamental theorems: the Representation Theorem, 

 and the Theorem of the Noisy Channel. Both theorems impose a limit on 

 the amount of information which can be effectively processed in a given 

 situation; both also state that it is possible to reach this limit. 



APPENDIX I 

 THE EVALUATION OF INFORMATION CONTENT 



The examples and exercises should have familiarized the reader with the 

 techniques of taking information measurements. However, the investigator 



