Evaluating the characteristics of a non-standardised Model Requirements Analysis (MRA) for the development of policy impact assessment tools

S. Sieber, T.S. Amjath-Babu, B.S. McIntosh, K. Tscherning, K. Mulller, K. Helming, D. Pohle, K. Fricke, P.J.F.M. Verweij, C. Pacini, T. Jansson, S.G.Y. Paloma

Research output: Contribution to journalArticleAcademicpeer-review

6 Citations (Scopus)

Abstract

The aim of this paper is to provide a critical analysis of the strengths and weaknesses of a non-standardised Model Requirements Analysis (MRA) used for the purpose of developing the Sustainability Impact Assessment Tool (SIAT). By 'non-standardised' we mean not strictly following a published MRA method. The underlying question we are interested in addressing is how non-standardised methods, often employed in research driven projects, compare to defined methods with more standardised structure, with regards their ability to capture model requirements effectively, and with regards their overall usability. Through describing and critically assessing the specific features of the non-standardised MRA employed, the ambition of this paper is to provide insights useful for impact assessment tool (IAT) development. Specifically, the paper will (i) characterise kinds of user requirements relevant to the functionality and design of IATs; (ii) highlight the strengths and weaknesses of non-standardised MRA for user requirements capture, analysis and reflection in the context of IAT; (iii) critically reflect on the process and outcomes of having used a non-standardised MRA in comparison with other more standardised approaches. To accomplish these aims, we first review methods available for IAT development before describing the SIAT development process, including the MRA employed. Major strengths and weaknesses of the MRA method are then discussed in terms of user identification and characterisation, organisational characterisation and embedding, and ability to capture design options for ensuring usability and usefulness. A detailed assessment on the structural differences of MRA with two advanced approaches (Integrated DSS design and goal directed design) and their role in performance of the MRA tool is used to critique the approach employed. The results show that MRA is able to bring thematic integration, establish system performance and technical thresholds as well as detailing quality and transparency guidelines. Nevertheless the discussion points out to a number of deficiencies in application - (i) a need to more effectively characterise potential users, and; (ii) a need to better foster communication among the distinguished roles in the development process. If addressed these deficiencies, SIAT non-standardised MRA could have brought out better outcomes in terms of tool usability and usefulness, and improved embedding of the tool into conditions of targeted end-users. (C) 2013 Elsevier Ltd. All rights reserved.
Original languageEnglish
Pages (from-to)53-63
JournalEnvironmental Modelling & Software
Volume49
DOIs
Publication statusPublished - 2013

Fingerprint

Sustainable development
sustainability
analysis
policy
impact assessment
critical analysis
integrated approach
transparency
Transparency
method
communication
Communication
comparison
project

Keywords

  • support
  • systems
  • design
  • methodology
  • management
  • decision
  • science

Cite this

Sieber, S. ; Amjath-Babu, T.S. ; McIntosh, B.S. ; Tscherning, K. ; Mulller, K. ; Helming, K. ; Pohle, D. ; Fricke, K. ; Verweij, P.J.F.M. ; Pacini, C. ; Jansson, T. ; Paloma, S.G.Y. / Evaluating the characteristics of a non-standardised Model Requirements Analysis (MRA) for the development of policy impact assessment tools. In: Environmental Modelling & Software. 2013 ; Vol. 49. pp. 53-63.
@article{5767507343b1488ab1c28686def38b71,
title = "Evaluating the characteristics of a non-standardised Model Requirements Analysis (MRA) for the development of policy impact assessment tools",
abstract = "The aim of this paper is to provide a critical analysis of the strengths and weaknesses of a non-standardised Model Requirements Analysis (MRA) used for the purpose of developing the Sustainability Impact Assessment Tool (SIAT). By 'non-standardised' we mean not strictly following a published MRA method. The underlying question we are interested in addressing is how non-standardised methods, often employed in research driven projects, compare to defined methods with more standardised structure, with regards their ability to capture model requirements effectively, and with regards their overall usability. Through describing and critically assessing the specific features of the non-standardised MRA employed, the ambition of this paper is to provide insights useful for impact assessment tool (IAT) development. Specifically, the paper will (i) characterise kinds of user requirements relevant to the functionality and design of IATs; (ii) highlight the strengths and weaknesses of non-standardised MRA for user requirements capture, analysis and reflection in the context of IAT; (iii) critically reflect on the process and outcomes of having used a non-standardised MRA in comparison with other more standardised approaches. To accomplish these aims, we first review methods available for IAT development before describing the SIAT development process, including the MRA employed. Major strengths and weaknesses of the MRA method are then discussed in terms of user identification and characterisation, organisational characterisation and embedding, and ability to capture design options for ensuring usability and usefulness. A detailed assessment on the structural differences of MRA with two advanced approaches (Integrated DSS design and goal directed design) and their role in performance of the MRA tool is used to critique the approach employed. The results show that MRA is able to bring thematic integration, establish system performance and technical thresholds as well as detailing quality and transparency guidelines. Nevertheless the discussion points out to a number of deficiencies in application - (i) a need to more effectively characterise potential users, and; (ii) a need to better foster communication among the distinguished roles in the development process. If addressed these deficiencies, SIAT non-standardised MRA could have brought out better outcomes in terms of tool usability and usefulness, and improved embedding of the tool into conditions of targeted end-users. (C) 2013 Elsevier Ltd. All rights reserved.",
keywords = "support, systems, design, methodology, management, decision, science",
author = "S. Sieber and T.S. Amjath-Babu and B.S. McIntosh and K. Tscherning and K. Mulller and K. Helming and D. Pohle and K. Fricke and P.J.F.M. Verweij and C. Pacini and T. Jansson and S.G.Y. Paloma",
note = "WOS:000327575600005",
year = "2013",
doi = "10.1016/j.envsoft.2013.07.007",
language = "English",
volume = "49",
pages = "53--63",
journal = "Environmental Modelling & Software",
issn = "1364-8152",
publisher = "Elsevier",

}

Sieber, S, Amjath-Babu, TS, McIntosh, BS, Tscherning, K, Mulller, K, Helming, K, Pohle, D, Fricke, K, Verweij, PJFM, Pacini, C, Jansson, T & Paloma, SGY 2013, 'Evaluating the characteristics of a non-standardised Model Requirements Analysis (MRA) for the development of policy impact assessment tools', Environmental Modelling & Software, vol. 49, pp. 53-63. https://doi.org/10.1016/j.envsoft.2013.07.007

Evaluating the characteristics of a non-standardised Model Requirements Analysis (MRA) for the development of policy impact assessment tools. / Sieber, S.; Amjath-Babu, T.S.; McIntosh, B.S.; Tscherning, K.; Mulller, K.; Helming, K.; Pohle, D.; Fricke, K.; Verweij, P.J.F.M.; Pacini, C.; Jansson, T.; Paloma, S.G.Y.

In: Environmental Modelling & Software, Vol. 49, 2013, p. 53-63.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Evaluating the characteristics of a non-standardised Model Requirements Analysis (MRA) for the development of policy impact assessment tools

AU - Sieber, S.

AU - Amjath-Babu, T.S.

AU - McIntosh, B.S.

AU - Tscherning, K.

AU - Mulller, K.

AU - Helming, K.

AU - Pohle, D.

AU - Fricke, K.

AU - Verweij, P.J.F.M.

AU - Pacini, C.

AU - Jansson, T.

AU - Paloma, S.G.Y.

N1 - WOS:000327575600005

PY - 2013

Y1 - 2013

N2 - The aim of this paper is to provide a critical analysis of the strengths and weaknesses of a non-standardised Model Requirements Analysis (MRA) used for the purpose of developing the Sustainability Impact Assessment Tool (SIAT). By 'non-standardised' we mean not strictly following a published MRA method. The underlying question we are interested in addressing is how non-standardised methods, often employed in research driven projects, compare to defined methods with more standardised structure, with regards their ability to capture model requirements effectively, and with regards their overall usability. Through describing and critically assessing the specific features of the non-standardised MRA employed, the ambition of this paper is to provide insights useful for impact assessment tool (IAT) development. Specifically, the paper will (i) characterise kinds of user requirements relevant to the functionality and design of IATs; (ii) highlight the strengths and weaknesses of non-standardised MRA for user requirements capture, analysis and reflection in the context of IAT; (iii) critically reflect on the process and outcomes of having used a non-standardised MRA in comparison with other more standardised approaches. To accomplish these aims, we first review methods available for IAT development before describing the SIAT development process, including the MRA employed. Major strengths and weaknesses of the MRA method are then discussed in terms of user identification and characterisation, organisational characterisation and embedding, and ability to capture design options for ensuring usability and usefulness. A detailed assessment on the structural differences of MRA with two advanced approaches (Integrated DSS design and goal directed design) and their role in performance of the MRA tool is used to critique the approach employed. The results show that MRA is able to bring thematic integration, establish system performance and technical thresholds as well as detailing quality and transparency guidelines. Nevertheless the discussion points out to a number of deficiencies in application - (i) a need to more effectively characterise potential users, and; (ii) a need to better foster communication among the distinguished roles in the development process. If addressed these deficiencies, SIAT non-standardised MRA could have brought out better outcomes in terms of tool usability and usefulness, and improved embedding of the tool into conditions of targeted end-users. (C) 2013 Elsevier Ltd. All rights reserved.

AB - The aim of this paper is to provide a critical analysis of the strengths and weaknesses of a non-standardised Model Requirements Analysis (MRA) used for the purpose of developing the Sustainability Impact Assessment Tool (SIAT). By 'non-standardised' we mean not strictly following a published MRA method. The underlying question we are interested in addressing is how non-standardised methods, often employed in research driven projects, compare to defined methods with more standardised structure, with regards their ability to capture model requirements effectively, and with regards their overall usability. Through describing and critically assessing the specific features of the non-standardised MRA employed, the ambition of this paper is to provide insights useful for impact assessment tool (IAT) development. Specifically, the paper will (i) characterise kinds of user requirements relevant to the functionality and design of IATs; (ii) highlight the strengths and weaknesses of non-standardised MRA for user requirements capture, analysis and reflection in the context of IAT; (iii) critically reflect on the process and outcomes of having used a non-standardised MRA in comparison with other more standardised approaches. To accomplish these aims, we first review methods available for IAT development before describing the SIAT development process, including the MRA employed. Major strengths and weaknesses of the MRA method are then discussed in terms of user identification and characterisation, organisational characterisation and embedding, and ability to capture design options for ensuring usability and usefulness. A detailed assessment on the structural differences of MRA with two advanced approaches (Integrated DSS design and goal directed design) and their role in performance of the MRA tool is used to critique the approach employed. The results show that MRA is able to bring thematic integration, establish system performance and technical thresholds as well as detailing quality and transparency guidelines. Nevertheless the discussion points out to a number of deficiencies in application - (i) a need to more effectively characterise potential users, and; (ii) a need to better foster communication among the distinguished roles in the development process. If addressed these deficiencies, SIAT non-standardised MRA could have brought out better outcomes in terms of tool usability and usefulness, and improved embedding of the tool into conditions of targeted end-users. (C) 2013 Elsevier Ltd. All rights reserved.

KW - support

KW - systems

KW - design

KW - methodology

KW - management

KW - decision

KW - science

U2 - 10.1016/j.envsoft.2013.07.007

DO - 10.1016/j.envsoft.2013.07.007

M3 - Article

VL - 49

SP - 53

EP - 63

JO - Environmental Modelling & Software

JF - Environmental Modelling & Software

SN - 1364-8152

ER -