Li et al.: A comparison of 4 age-structured stock assessment models 
159 
Table 5 
Median absolute relative error (%) for the model parameters unfished recruitment (RO) and 
catchability of survey abundance index (q) from each simulated case used to compare the 4 
age-structured stock assessment models used most commonly in the United States: the Assess- 
ment Model for Alaska (AMAK), the Age Structured Assessment Program (ASAP), the Beaufort 
Assessment Model (BAM), and Stock Synthesis (SS). 
RO 
Case 
Case 0 
Case 1 
Case 2 
Case 3 
Case 4 
Case 5 
Case 6 
Case 7 
Case 8 
Case 9 
Case 9 
Case 10 
Case 11 
Case 12 
The range of RE in MSY-based reference points became 
wider when recruitment variability increased (Fig. 5). Fur- 
thermore, the increased variability in SSBysy induced a 
wider range of RE in relative SSB (Table 6, Fig. 6). The accu- 
racy of determining overfishing status for all EMs was not 
greatly affected by changes in recruitment variability, and 
the same trends were retained over time compared with the 
accuracy trends for the null case (Fig. 7, Suppl. Figs. 7 and 8 
[online only]). The accuracy of determining overfished status 
was 100% over time for all EMs. 
Process error in fishing mortality There were no consider- 
able differences found in key parameter estimates when 
examining the effect of process error in F (case 3 versus 
case 0 in Figures 5 and 6). For all 4 EMs, RE patterns and 
estimates of key parameters were almost identical. How- 
ever, the accuracy of the overfishing determination from 
case 3 fluctuated at levels under 100% for a longer period 
than that from case 0, although the true F values were not 
always close to the true Fysy (Fig. 7). This result indicates 
that incorporating stochastic sets of annual deviations in 
F induces sensitivity in the determination of overfishing, 
but the overall accuracy of the overfishing determination 
was still high with values often exceeding 94%. The accu- 
racy of determining an overfished status was still 100% 
over time for all EMs. 
Fishing mortality patterns The EMs produced low MARE 
in estimates of key parameters when patterns of F were 
different. The variability of RE in all 4 EMs was consis- 
tently higher than in case 0 when the F pattern was a 
1.63 
Po AMe) 
2.14 
1.93 
1.83 
4.11 
2.05 
2.11 
3.42 
1.80 (survey1) 
1.73 (survey2) 
1.88 
2.01 
2.14 
constant F’,,,, indicating that the estimates had greater 
bias when there was not much contrast between initial F 
and the later period of constant F (case 5 versus case 0 
in Table 6 and Figures 5 and 6). When there was a fair 
amount of contrast in F across time, the MARE among 
the 4 models remained similar in trends and magnitude. 
Although accuracy in determining an overfished status 
was 100% for all EMs, the accuracy in determining over- 
fishing status was consistently lower compared with that 
for other cases when F fluctuated around Fysy (case 6 in 
Figure 7). 
Selectivity patterns When both the fishing fleet and sur- 
vey had double-logistic selectivity, the estimated median 
selectivity at age over 100 iterations from all 4 EMs was 
close to the true selectivity at age. The estimates of key 
parameters from all EMs were accurate compared with 
the true values from the OM (Table 5). The range of RE 
in key estimates became wider when the fishery and sur- 
vey had double-logistic selectivity (case 8 versus case 0 in 
Table 6, Figure 6, and Supplementary Figures 3-8 [online 
only]), especially in the early years. For years when fishing 
was not near Fysy, the overfishing status determination 
was 100% accurate, and the overfished status determina- 
tion was 100% accurate over all years. 
Multiple surveys The differences in key parameter esti- 
mates between case 9 and case 0 were not considerable 
(Fig. 5). However, the range of RE in SSB, recruitment, F, 
relative SSB, and relative F became narrower after an addi- 
tional survey index with the same level of observation error 
