166 
Fishery Bulletin 119(2—3) 
Having more comparison practices that involve both next- 
generation stock assessment models and existing models 
(inside and outside the United States) will enhance the 
communication among model developers and users, facili- 
tate the interpretation of comparison results among mod- 
els, and improve future assessments. 
Conclusions 
This study was designed to verify if the assessment mod- 
els developed in different regions of the United States can 
produce similar estimates when given the same input 
data and configured similarly. However, it had a second- 
ary objective of informing development of next-generation 
models (Punt et al., 2020). It is clear that all 4 models 
tested in this study provide similar and accurate esti- 
mates of quantities of interest under the tested cases. This 
outcome was expected given that the 4 EMs share similar 
mathematical and statistical attributes and that the sim- 
ulated data were very informative. Nevertheless, it was 
expected also because we carefully evaluated the conver- 
sions among models to ensure that model configurations 
were similar to each other and model outputs were com- 
parable. For future model comparison work or ensemble 
work, we recommend comparison of key features in source 
code before any multi-model analysis is done in order to 
identify differences in parameterization that could be 
misleading when results are compared (e.g., selectivity 
function parameters). We also recommend minimizing 
the variations of parameterizations for the same feature 
during development of next-generation stock assessment 
models. Standardized inputs and outputs for common 
parameters would allow easy comparisons of results from 
different models. 
In this study, we have identified the sources of slight 
differences among model estimates under different cases. 
The differences are associated with computation of ini- 
tial numbers at age and bias adjustment of recruitment. 
Improved insights on these key differences should help the 
development of next-generation stock assessment models. 
Key potential areas for future improvements include bet- 
ter clarification of terminology used in assessment reports, 
use of the conversion function developed in this study to 
convert between median-unbiased and mean-unbiased 
spawner-recruit parameters in stock assessment, use of 
the conversion function in other meta-analyses to ensure 
the inputs of meta-analysis are comparable, and devel- 
opment of guidance on which bias adjustment method is 
preferable under which situations. 
Acknowledgments 
This research was performed while the senior author held 
a National Research Council Research Associateship award 
at the Office of Science and Technology, National Marine 
Fisheries Service. We thank K. Doering, C. Stawitz, E. Dick, 
M. Masi, K. Johnson, and C. Bassin for their comments and 
suggestions for improving the development of the operating 
model in this study and this paper. 
Literature cited 
AFSC (Alaska Fisheries Science Center). 
2015. Assessment Model for Alaska description of GUI and 
instructions, 50 p. [Available from website.] 
Barbeaux, S. J., J. Ianelli, and W. Palsson. 
2019. Assessment of the pollock stock in the Aleutian Islands, 
3 p. In Stock assessment and fishery evaluation report 
for the groundfish resources of the Bering Sea/Aleutian 
Islands regions. North Pac. Fish. Manage. Counc., Anchor- 
age, AK. [Available from website.] 
Beverton, R. J. H., and S. J. Holt. 
1957. On the dynamics of exploited fish populations. Fish. 
Investig. (G.B. Minist. Agric. Fish. Food), Ser. 2, vol. 19, 
533 p. HMSO, London. 
Brodziak, J., and C. M. Legault. 
2005. Model averaging to estimate rebuilding targets for over- 
fished stocks. Can. J. Fish. Aquat. Sci. 62:544-562. 
Brodziak, J., and K. Piner. 
2010. Model averaging and probable status of North Pacific 
striped marlin, Tetrapturus audax. Can. J. Fish. Aquat. Sci. 
67:793-805. 
Cadrin, S. X., and M. Dickey-Collas. 
2015. Stock assessment methods for sustainable fisheries. 
ICES J. Mar. Sci. 72:1-6. 
Chang, Y.-J., J. Brodziak, J. O’Malley, H.-H. Lee, G. DiNardo, and 
C.-L. Sun. 
2015. Model selection and multi-model inference for Bayesian 
surplus production models: a case study for Pacific blue and 
striped marlin. Fish. Res. 166:129-139. 
Chen, D. G. 
2004. Bias and bias correction in fish recruitment prediction. 
North Am. J. Fish. Manage. 24:724—730. 
Connors, B. M., and A. B. Cooper. 
2014. Determining decision thresholds and evaluating indi- 
cators when conservation status is measured as a contin- 
uum. Conserv. Biol. 28:1626—1635. 
Cortés, E., and E. N. Brooks. 
2018. Stock status and reference points for sharks using 
data-limited methods and life history. Fish Fish. 
19:1110-1129. 
Deroba, J. J., D.S. Butterworth, R. D. Methot Jr., J. A. A. De Oliveira, 
C. Fernandez, A. Nielsen, S. X. Cadrin, M. Dickey-Collas, 
C. M. Legault, J. Ianelli, et al. 
2015. Simulation testing the robustness of stock assessment 
models to error: some results from the ICES strategic ini- 
tiative on stock assessment methods. ICES J. Mar. Sci. 
72:19-30. 
Dichmont, C. M., R. A. Deng, A. E. Punt, J. Brodziak, Y.-J. Chang, 
J. M. Cope, J. N. Ianelli, C. M. Legault, R. D. Methot Jr., 
C. E. Porch, et al. 
2016. A review of stock assessment packages in the United 
States. Fish. Res. 183:447—460. 
Federal Register. 
1998. Magnuson-Stevens Act provisions; national standard 
guidelines. Fed. Regist. 63:24212—-24237. 
Fournier, D. A., H. J. Skaug, J. Ancheta, J. Ianelli, A. Magnusson, 
M. N. Maunder, A. Nielsen, and J. Sibert. 
2012. AD Model Builder: using automatic differentiation for 
statistical inference of highly parameterized complex non- 
linear models. Optim. Methods Softw. 27:233-249. 
