Sustainability assessment of agricultural systems

The validity of expert opinion and robustness of a multi-criteria analysis

Farahnaz Pashaei Kamali*, João A.R. Borges, Miranda P.M. Meuwissen, Imke J.M. de Boer, Alfons G.J.M. Oude Lansink

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

10 Citations (Scopus)

Abstract

Sustainability assessment of agricultural systems is frequently hampered by data availability. Elicitation of expert opinions combined with multi-criteria assessment (MCA) could be a useful approach for sustainability assessments in data-scarce situations. To our knowledge, the validity of expert opinion used to score sustainability performance of agricultural systems, however, has not been addressed. Also, robustness of the overall outcome of MCA to uncertainty about scores obtained from expert elicitation and weights used to aggregate scores is generally not addressed. The objectives of this study were to evaluate the validity of expert opinion, and to evaluate the robustness of the overall MCA outcome to uncertainty about scores and weights. The case study considers three soybean agricultural systems in Latin America: conventional agricultural system, with either genetically modified (GM) or non-genetically modified (non-GM) soybeans, and organic agricultural system. The validation was carried out by comparing the sustainability scores of experts with values from scientific studies. The robustness of the overall outcome of the MCA to uncertainty about scores and weights was assessed using Monte Carlo simulation. The comparison of expert opinion with reviewed studies showed that expert opinions are a potential alternative to extensive data-rich methods. The validity of expert opinions can be increased by considering a larger group of experts, with a high level of knowledge about agricultural systems and sustainability issues. With regard to robustness, the overall outcome of the MCA showed higher variation for organic soybean agricultural systems compared with GM and non-GM, in both Brazil and Argentina.
Original languageEnglish
Pages (from-to)118-128
JournalAgricultural Systems
Volume157
DOIs
Publication statusPublished - 2017

Fingerprint

expert opinion
uncertainty
soybeans
Latin America
Argentina
case studies
Brazil

Keywords

  • Agricultural systems
  • Monte Carlo simulation
  • Multi-criteria assessment
  • Robustness
  • Sustainability
  • Validation

Cite this

@article{0dedc4c506964ad9b4d4ecb2838f6400,
title = "Sustainability assessment of agricultural systems: The validity of expert opinion and robustness of a multi-criteria analysis",
abstract = "Sustainability assessment of agricultural systems is frequently hampered by data availability. Elicitation of expert opinions combined with multi-criteria assessment (MCA) could be a useful approach for sustainability assessments in data-scarce situations. To our knowledge, the validity of expert opinion used to score sustainability performance of agricultural systems, however, has not been addressed. Also, robustness of the overall outcome of MCA to uncertainty about scores obtained from expert elicitation and weights used to aggregate scores is generally not addressed. The objectives of this study were to evaluate the validity of expert opinion, and to evaluate the robustness of the overall MCA outcome to uncertainty about scores and weights. The case study considers three soybean agricultural systems in Latin America: conventional agricultural system, with either genetically modified (GM) or non-genetically modified (non-GM) soybeans, and organic agricultural system. The validation was carried out by comparing the sustainability scores of experts with values from scientific studies. The robustness of the overall outcome of the MCA to uncertainty about scores and weights was assessed using Monte Carlo simulation. The comparison of expert opinion with reviewed studies showed that expert opinions are a potential alternative to extensive data-rich methods. The validity of expert opinions can be increased by considering a larger group of experts, with a high level of knowledge about agricultural systems and sustainability issues. With regard to robustness, the overall outcome of the MCA showed higher variation for organic soybean agricultural systems compared with GM and non-GM, in both Brazil and Argentina.",
keywords = "Agricultural systems, Monte Carlo simulation, Multi-criteria assessment, Robustness, Sustainability, Validation",
author = "{Pashaei Kamali}, Farahnaz and Borges, {Jo{\~a}o A.R.} and Meuwissen, {Miranda P.M.} and {de Boer}, {Imke J.M.} and {Oude Lansink}, {Alfons G.J.M.}",
year = "2017",
doi = "10.1016/j.agsy.2017.07.013",
language = "English",
volume = "157",
pages = "118--128",
journal = "Agricultural Systems",
issn = "0308-521X",
publisher = "Elsevier",

}

Sustainability assessment of agricultural systems : The validity of expert opinion and robustness of a multi-criteria analysis. / Pashaei Kamali, Farahnaz; Borges, João A.R.; Meuwissen, Miranda P.M.; de Boer, Imke J.M.; Oude Lansink, Alfons G.J.M.

In: Agricultural Systems, Vol. 157, 2017, p. 118-128.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Sustainability assessment of agricultural systems

T2 - The validity of expert opinion and robustness of a multi-criteria analysis

AU - Pashaei Kamali, Farahnaz

AU - Borges, João A.R.

AU - Meuwissen, Miranda P.M.

AU - de Boer, Imke J.M.

AU - Oude Lansink, Alfons G.J.M.

PY - 2017

Y1 - 2017

N2 - Sustainability assessment of agricultural systems is frequently hampered by data availability. Elicitation of expert opinions combined with multi-criteria assessment (MCA) could be a useful approach for sustainability assessments in data-scarce situations. To our knowledge, the validity of expert opinion used to score sustainability performance of agricultural systems, however, has not been addressed. Also, robustness of the overall outcome of MCA to uncertainty about scores obtained from expert elicitation and weights used to aggregate scores is generally not addressed. The objectives of this study were to evaluate the validity of expert opinion, and to evaluate the robustness of the overall MCA outcome to uncertainty about scores and weights. The case study considers three soybean agricultural systems in Latin America: conventional agricultural system, with either genetically modified (GM) or non-genetically modified (non-GM) soybeans, and organic agricultural system. The validation was carried out by comparing the sustainability scores of experts with values from scientific studies. The robustness of the overall outcome of the MCA to uncertainty about scores and weights was assessed using Monte Carlo simulation. The comparison of expert opinion with reviewed studies showed that expert opinions are a potential alternative to extensive data-rich methods. The validity of expert opinions can be increased by considering a larger group of experts, with a high level of knowledge about agricultural systems and sustainability issues. With regard to robustness, the overall outcome of the MCA showed higher variation for organic soybean agricultural systems compared with GM and non-GM, in both Brazil and Argentina.

AB - Sustainability assessment of agricultural systems is frequently hampered by data availability. Elicitation of expert opinions combined with multi-criteria assessment (MCA) could be a useful approach for sustainability assessments in data-scarce situations. To our knowledge, the validity of expert opinion used to score sustainability performance of agricultural systems, however, has not been addressed. Also, robustness of the overall outcome of MCA to uncertainty about scores obtained from expert elicitation and weights used to aggregate scores is generally not addressed. The objectives of this study were to evaluate the validity of expert opinion, and to evaluate the robustness of the overall MCA outcome to uncertainty about scores and weights. The case study considers three soybean agricultural systems in Latin America: conventional agricultural system, with either genetically modified (GM) or non-genetically modified (non-GM) soybeans, and organic agricultural system. The validation was carried out by comparing the sustainability scores of experts with values from scientific studies. The robustness of the overall outcome of the MCA to uncertainty about scores and weights was assessed using Monte Carlo simulation. The comparison of expert opinion with reviewed studies showed that expert opinions are a potential alternative to extensive data-rich methods. The validity of expert opinions can be increased by considering a larger group of experts, with a high level of knowledge about agricultural systems and sustainability issues. With regard to robustness, the overall outcome of the MCA showed higher variation for organic soybean agricultural systems compared with GM and non-GM, in both Brazil and Argentina.

KW - Agricultural systems

KW - Monte Carlo simulation

KW - Multi-criteria assessment

KW - Robustness

KW - Sustainability

KW - Validation

U2 - 10.1016/j.agsy.2017.07.013

DO - 10.1016/j.agsy.2017.07.013

M3 - Article

VL - 157

SP - 118

EP - 128

JO - Agricultural Systems

JF - Agricultural Systems

SN - 0308-521X

ER -