A New Statistical Method to Determine the Degree of Validity of Health Economic Model Outcomes against Empirical Data

Isaac Corro Ramos*, George A.K. van Voorn, Pepijn Vemer, Talitha L. Feenstra, Maiwenn J. Al

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

4 Citations (Scopus)

Abstract

Background: The validation of health economic (HE) model outcomes against empirical data is of key importance. Although statistical testing seems applicable, guidelines for the validation of HE models lack guidance on statistical validation, and actual validation efforts often present subjective judgment of graphs and point estimates. Objectives: To discuss the applicability of existing validation techniques and to present a new method for quantifying the degrees of validity statistically, which is useful for decision makers. Methods: A new Bayesian method is proposed to determine how well HE model outcomes compare with empirical data. Validity is based on a pre-established accuracy interval in which the model outcomes should fall. The method uses the outcomes of a probabilistic sensitivity analysis and results in a posterior distribution around the probability that HE model outcomes can be regarded as valid. Results: We use a published diabetes model (Modelling Integrated Care for Diabetes based on Observational data) to validate the outcome "number of patients who are on dialysis or with end-stage renal disease." Results indicate that a high probability of a valid outcome is associated with relatively wide accuracy intervals. In particular, 25% deviation from the observed outcome implied approximately 60% expected validity. Conclusions: Current practice in HE model validation can be improved by using an alternative method based on assessing whether the model outcomes fit to empirical data at a predefined level of accuracy. This method has the advantage of assessing both model bias and parameter uncertainty and resulting in a quantitative measure of the degree of validity that penalizes models predicting the mean of an outcome correctly but with overly wide credible intervals.

Original languageEnglish
Pages (from-to)1041-1047
JournalValue in Health
Volume20
Issue number8
DOIs
Publication statusPublished - 2017

Keywords

  • Decision making
  • Health economics methods
  • Statistics
  • Validation

Fingerprint

Dive into the research topics of 'A New Statistical Method to Determine the Degree of Validity of Health Economic Model Outcomes against Empirical Data'. Together they form a unique fingerprint.

Cite this