8 Information Storage and Neural Control 



ask questions in accordance with his a priori knowledge of these 

 probability states. Information theory as well as intuition tells us 

 that a good strategy would be to inquire about the most likely 

 probability states first. This point might be more clearly illustrated 

 by considering the information storage problem, which is equiva- 

 lent in principle. 



INFORMATION STORAGE 



Mathematically, there is no important difference between the 

 application of information theory to communications systems 

 through which information flows continuously and to static sys- 

 tems used for storing information. The problem of storing infor- 

 mation is essentially one of making a representation. The repre- 

 sentation can take any form as long as the original or something 

 equivalent to it can be reconstructed at will. It is clear for example, 

 that even though information exists as sound, there is no need 

 to store it acoustically. There is no objection to the use of a re- 

 versible code since information is invariant under such a trans- 

 formation and, therefore, can be stored equally well electrically 

 or magnetically; as for example on a recording tape. We simply 

 have to insure that every possible event to be recorded can be 

 represented in the store. This implies that an empty store must 

 merely be capable of being put into different states and that the 

 precise nature of these states is quite immaterial to the question 

 of how much information can be stored. Thus, the capacity of 

 an empty information store depends only on the total number 

 of distinguishable states of which it admits. Hence, the larger the 

 number of states, the larger the capacity. 



If a storage unit such as a knob with click positions has n possible 

 states, then two such units provide altogether n'' states. From this 

 it is clear that duplication of the basic units is a powerful way to 

 increase storage capacity. Since physically, it is generally easier 

 to make two ^/-state devices than one single device with n'' states, 

 practical storage systems will generally be found to consist of a 

 multiplicity of smaller units. Thus, 1000 two-state devices can 

 provide a total of 2'""'' possible states. 



The exponential dependence of the number of states on the 

 number of units immediately suggests a logarithmic measure of 



