Validation of stock assessment methods: Is it me or my model talking?

Laurence T. Kell*, Rishi Sharma, Toshihide Kitakado, Henning Winker, Iago Mosqueira, Massimiliano Cardinale, Dan Fu

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

11 Citations (Scopus)

Abstract

The adoption of the Precautionary Approach requires providing advice that is robust to uncertainty. Therefore, when conducting stock assessment alternative, model structures and data sets are commonly considered. The primary diagnostics used to compare models are to examine residuals patterns to check goodness-of-fit and to conduct retrospective analysis to check the stability of estimates. However, residual patterns can be removed by adding more parameters than justified by the data, and retrospective patterns removed by ignoring the data. Therefore, neither alone can be used for validation, which requires assessing whether it is plausible that a system identical to the model generated the data. Therefore, we use hindcasting to estimate prediction skill, a measure of the accuracy of a predicted value unknown by the model relative to its observed value, to explore model misspecification and data conflicts. We compare alternative model structures based on integrated statistical and Bayesian state-space biomass dynamic models using, as an example, Indian Ocean yellowfin tuna. Validation is not a binary process (i.e. pass or fail) but a continuum; therefore, we discuss the use of prediction skill to identify alternative hypotheses, weight ensemble models and agree on reference sets of operating models when conducting Management Strategy Evaluation.

Original languageEnglish
Pages (from-to)2244-2255
Number of pages12
JournalICES Journal of Marine Science
Volume78
Issue number6
DOIs
Publication statusPublished - 10 Jun 2021

Keywords

  • diagnostics
  • hindcast
  • prediction skill
  • retrospective analysis
  • stock assessment
  • validation

Fingerprint

Dive into the research topics of 'Validation of stock assessment methods: Is it me or my model talking?'. Together they form a unique fingerprint.

Cite this