9/7 AN INTRODUCTION TO CYBERNETICS 



*Ex. 8: If Z) is the column vector of the populations in the various states, D' 

 the vector one step later, and M the matrix of transition probabilities, 

 show that, in ordinary matrix algebra, 



D' = MD, D" = M2Z), and /)<"> = M^D. 



(This simple and natural relation is lost if the matrix is written in trans- 

 posed form. Compare Ex. 2/16/3 and 12/8/4.) 



9/7. Dependence on earlier values. The definition of a Markov 

 chain, given in S.9/4, omitted an important qualification : the proba- 

 bilities of transition must not depend on states earlier than the operand. 

 Thus if the insect behaves as a Markov chain it will be found that 

 when on the bank it will go to the water in 75% of the cases, whether 

 before being on the bank it was at bank, water, or pebbles. One 

 would test the fact experimentally by collecting the three corres- 

 ponding percentages and then seeing if they were all equal at 75%. 

 Here is a protocol in which the independence does not hold: 



AABBABBAABBABBABBABBAABBABBABABA 

 The transitions, on a direct count, are 



In particular we notice that B is followed by A and B about equally. 

 If we now re-classify these 18 transitions from B according to what 

 letter preceded the B we get : 



. . . AB was followed by 



. . . BB „ „ ,, 



So what state follows B depends markedly on what state came before 

 the B. Thus this sequence is not a Markov chain. Sometimes the 

 fact can be described in metaphor by saying that the system's 

 "memory" extends back for more than one state (compare S.6/21). 

 This dependence of the probability on what came earlier is a 

 marked characteristic of the sequences of letters given by a language 

 such as English. Thus: what is the probability that an s will be 

 followed by a /? It depends much on what preceded the s; thus 

 es followed by / is common, but ds followed by / is rare. Were the 

 letters a Markov chain, then .v would be followed by / with the same 

 frequency in the two cases. 



170 



