The effect of controlled opinion feedback on Delphi features: Mixed messages from a real-world Delphi experiment

Jurian V. Meijering*, Hilde Tobi

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

29 Citations (Scopus)


A real-world Delphi experiment was conducted to investigate the effect of two controlled opinion feedback conditions on the drop-out rate, experts' degree of opinion change, and the increase in the level of agreement among experts. Additionally, experts' perceived usefulness of feedback was explored. In the first and second Delphi round experts received a questionnaire which consisted of two sections. Within each section experts were asked to rate several items. In round 2, experts in one condition received feedback consisting of summary statistics and rationales (S&R condition), whereas experts in the other condition received rationales only (R condition). Results showed that drop-out of experts was greater in the S&. R condition than in the R condition. No difference between conditions was found concerning experts' degree of opinion change. The increase in the level of agreement across the items in the second section of the questionnaire differed significantly between conditions. This difference was mainly due to a decrease in agreement in the R condition, suggesting that feedback of rationales may increase disagreement among experts. In round 3 experts preferred to receive both summary statistics and rationales, although they tended to perceive rationales as somewhat more useful than summary statistics.

Original languageEnglish
Pages (from-to)166-173
JournalTechnological Forecasting and Social Change
Publication statusPublished - 1 Feb 2016


  • Agreement
  • Controlled opinion feedback
  • Delphi experiment
  • Drop-out
  • Feedback perception
  • Opinion change


Dive into the research topics of 'The effect of controlled opinion feedback on Delphi features: Mixed messages from a real-world Delphi experiment'. Together they form a unique fingerprint.

Cite this