TY - JOUR
T1 - The effects of feeding back experts’ own initial ratings in Delphi studies
T2 - A randomized trial
AU - Meijering, Jurian Vincent
AU - Tobi, Hilde
PY - 2018/4/1
Y1 - 2018/4/1
N2 - This study examined the effects of feeding back experts’ initial ratings on three Delphi outcome measures: (1) the percentage of items on which experts changed their opinion; (2) the degree to which experts changed their ratings towards the group response; and (3) the increase in the level of agreement among experts. Additionally, two conformity indices were developed. Within a real-world Delphi study, experts were randomly assigned to one of two conditions: either their initial ratings were included in feedback (IN) or excluded from feedback (EX). Results showed that experts in the EX condition changed their opinion relatively more often than experts in the IN condition. Results also suggested that experts in the EX condition changed their ratings to a greater degree towards the group response than experts in the IN condition. No difference between conditions was found regarding the increase in the level of agreement.
AB - This study examined the effects of feeding back experts’ initial ratings on three Delphi outcome measures: (1) the percentage of items on which experts changed their opinion; (2) the degree to which experts changed their ratings towards the group response; and (3) the increase in the level of agreement among experts. Additionally, two conformity indices were developed. Within a real-world Delphi study, experts were randomly assigned to one of two conditions: either their initial ratings were included in feedback (IN) or excluded from feedback (EX). Results showed that experts in the EX condition changed their opinion relatively more often than experts in the IN condition. Results also suggested that experts in the EX condition changed their ratings to a greater degree towards the group response than experts in the IN condition. No difference between conditions was found regarding the increase in the level of agreement.
U2 - 10.1016/j.ijforecast.2017.11.010
DO - 10.1016/j.ijforecast.2017.11.010
M3 - Article
AN - SCOPUS:85041568825
SN - 0169-2070
VL - 34
SP - 216
EP - 224
JO - International Journal of Forecasting
JF - International Journal of Forecasting
IS - 2
ER -