380 BELL SYSTEM TECHNICAL JOURNAL 



such as time, bandwidth, number of relays, etc., tend to vary Hnearly with 

 the logarithm of the number of possibiUties. For example, adding one relay 

 to a group doubles the number of possible states of the relays. It adds 1 

 to the base 2 logarithm of this number. Doubling the time roughly squares 

 the number of possible messages, or doubles the logarithm, etc. 



2. It is nearer to our intuitive feeling as to the proper measure. This is 

 closely related to (1) since we intuitively measure entities by linear com- 

 parison with common standards. One feels, for example, that two punched 

 cards should have twice the capacity of one for information storage, and two 

 identical channels twice the capacity of one for transmitting information. 



3. It is mathematically more suitable. Many of the limiting operations 

 are simple in terms of the logarithm but would require clumsy restatement in 

 terms of the number of possibilities. 



The choice of a logarithmic base corresponds to the choice of a unit for 

 measuring information. If the base 2 is used the resulting units may be 

 called binary digits, or more briefly hits, a word suggested by J. W. Tukey. 

 A device with two stable positions, such as a relay or a flip-flop circuit, can 

 store one bit of information. N such devices can store N bits, since the 

 total number of possible states is 2^ and log22^ = N. If the base 10 is 

 used the units may be called decimal digits. Since 



log2 M = logio M/logio2 



= 3.32 logio M, 



a decimal digit is about 2>\ bits. A digit wheel on a desk computing machine 

 has ten stable positions and therefore has a storage capacity of one decimal 

 digit. In analytical work where integration and differentiation are involved 

 the base e is sometimes useful. The resulting units of information will be 

 called natural units. Change from the base a to base h merely requires 

 multiplication by logb a. 



By a communication system we will mean a system of the type indicated 

 schematically in Fig. 1. It consists of essentially five parts: 



1. An information source which produces a message or sequence of mes- 

 sages to be communicated to the receiving terminal. The message may be 

 of various types : e.g. (a) A sequence of letters as in a telegraph or teletype 

 system; (b) A single function of time /(/) as in radio or telephony; (c) A 

 function of time and other variables as in black and white television — here 

 the message may be thought of as a function f{x, y, /) of two space coordi- 

 nates and time, the light intensity at point {x, y) and time / on a pickup tube 

 plate; (d) Two or more functions of time, say /(/), g(/), h{t) — this is the 

 case in ''three dimensional" sound transmission or if the system is intended 

 to service several individual channels in muhiplex; (e) Several functions of 



