A Primer on Information Theory 9 



form. Any standard form would be acceptable; it has become customary to 

 employ the simplest of all possible alphabets, the binary alphabet. The two 

 symbols commonly used are T and '0'; it must be emphasized that '0' does 

 not necessarily imply the absence of some physical action; e.g., T and '0' might 

 stand for right-left, positive-negative, dash-dot, etc. The standard symbolization 

 of any event will be a binary number, such as 1001 101 ... , where the symbolic 

 meaning of each digit and combination of digits is fixed by some law of associa- 

 tion. 



It must be pointed out that the Morse code is not a strictly binary representa- 

 tion if one thinks of whole messages. The Morse code is really a quaternary 

 code. This is so because, in addition to the 'blacknesses', the dots and dashes, 

 it uses two 'whitenesses' of different length, namely, an inter-letter space and 

 an inter-word space. Both of these are integral parts of the code system, because 

 otherwise we could not know whether this: 



means 'hen', 'sue', 'sin' or 'site'. 



How many events can be represented by words made up of a certain number 

 of binary symbols? There are two different 'words' (T and '0') consisting of a 

 single symbol; they can represent a partition of a set of real events into two 

 classes. There are four different two-symbol 'words' (11, 10, 01, 00), and, in 

 general, 2" code groups consisting of /z binary symbols. Accordingly: 



A sequence of « binary tests will discriminate between 2" possibilities; 



A sequence of « binary choices will select any one of 2" alternatives; 



A sequence of n binary statements will identify any one of a set of 2" items, 



etc. 



Conversely, if a code book with r distinct representations is to be made up 

 in standard binary code, then each word will have to be a binary number with 

 about log2 r digits*. For instance, eight categories of events can be represented 

 by code groups consisting of three binary symbols (3 = log2 8): 



Category A 1 1 1 



Category B 1 1 



Category C 10 1 



Category D 1 00 



Category E 1 1 



Category F 10 



Category G 1 



Category H 



Observe that the meaning of each binary symbol depends on its position and 

 on the nature of the other symbols in the word. For example, a 1 in the second 

 position means 'A or B or E or F' ; if preceded by a 0, then it can mean only 

 'E or F'; if also followed by a 1, then it designates 'E', unequivocally. This 



* Logarithms to the base 2 can be found in pubHshed tables, or read on a slide rule with a 

 log-log scale, or obtained by multiplying the base- 10 logarithms by 3.322. 



