MENDELSSOHN: USING MARKOV DECISION MODELS 



Van Hee (1977a) defined a set of policies that he 

 terms Bayes equivalent policies. For problems 

 such as the salmon models under discussion, a 

 Bayes equivalent policy would be found as follows: 



1) At the start of the period, the prior probability 

 distribution is q^( B). 



2) The expected transition function (expectation 

 with respect to O) is calculated, i.e., 



p(d,q) = jp(d\e)q(de) 



(4.1) 



where p( • | • ) describes the dependence of the ran- 

 dom variable d on O. 



3 )p(d,q) is used to solve a non-Bayesian Markov 

 decision process, with p{d, q) as the transition 

 function. 



4)The optimal policy from step 3 above is used 

 for one period. 



5) q-l O) is updated using Bayes theorem and the 

 observations from the last period, and the updated 

 (?( • ) is used in step 1 at the next time period. 



It is worth noting that a Bayes eqivalent policy 



8 1.2 1.6 2.0 2.4 2.8 3.2 3.6 4.0 44 4.8 



X 



Figure 3(a-m). — Optimal policy functions for the Branch River for various assumptions about the relative value of smoothing costs. 



(See text for details.) 



45 



