6 Information Storage and Neural Control 



been removed by the receipt of a message. For example, if you 

 are told that the baby Dr. Jones delivered today is a boy, then 

 you have been given one bit of information* (by definition one 

 bit is the amount of information necessary to resolve two equally 

 likely alternatives). If the uncertainty is greater, the amount of 

 information necessary to remove it is greater. Therefore, a message 

 which identifies one of 32 equally likely alternatives contains more 

 information (five bits as we shall explain later) than a message 

 which resolves 16 equally likely alternatives (four bits). 



Some elementary examples to convey the basic notions of 

 information measure may be helpful in developing some qualitative 

 insights. Consider a simple game in which you are asked to guess 

 a number with possible values from one to eight. With no a priori 

 knowledge of which of these numbers is the correct choice, the 

 probability of guessing the correct number is 1/8. In the language 

 of information theory, this situation might be described as follows: 

 A system is in one of eight equally probable states, and the state 

 of the system is completely unknown to the receiver. It is appro- 

 priate to ask how much information is conveyed to the receiver 

 by completely resolving the uncertainty of the receiver's knowledge. 

 Let us designate the eight equally probable states of the system 

 by the numbers from one to eight. Assume that you are to ask 

 only binary questions, i.e., questions which admit only of a yes 

 or no answer, in an attempt to determine the state of this system. 

 It is a simple matter to discover that the minimum number of 

 such questions certain to establish the state of the system is three. 

 In this simple illustration we have introduced the basic concepts 

 from which the quantitative definition of information can be 

 formulated: namely, the number of equally probable states of 



nects more directly with certain ideas generated about thirty years ago by H. Nyquist 

 and R. V. L. Hartley, both of Bell Telephone Laboratories. Professor Norbert Weiner's 

 work in the study of Cybernetics, which deals mainly with the use of information to 

 effect certain control actions, has been a major impetus in applying information 

 theory to biological and central nervous system phenomena. 



*Information theory does not deal with the importance of the information in a 

 message. For example, the information in the message, "the baby is a boy," is one bit 

 independent of whether you are the father. This comment is made in order to em- 

 phasize the fact that information theory docs not deal with the subjective value of 

 information, which falls more properly into the domain of semantics, but rather with 

 objective measures of information. 



