Measuring what matters? Exploring the use of values-based indicators in assessing Education for Sustainability

Evaluating the success or failure of educational initiatives, whether at the level of individual students, teachers and institutions or at national and global levels, relies on a shared understanding of what ‘success’ or ‘failure’ might look like in practice. This, in turn, cannot be defined in an objective way because it depends on the values, mindsets and priorities of diverse stakeholders - which may be very different from one another, or even mutually conflicting. In order to design assessment tools to ‘measure what we treasure’, as called for in the 2015 Millennium Development Report, we must first embark on the challenging task of creating appropriate indicators.

In this thesis, I have described four separate research studies conducted as part of the ESDinds project (‘Creating Indicators and Assessment Tools for Civil Society Organizations Promoting Education for Sustainable Development’), which was led by a consortium of two universities and four civil society organizations from 2008-2011. The project aimed to explore inductive ways of developing indicators in the context of non-formal education for sustainability – drawing them out from participants’ comments about what they find valuable, meaningful and worthwhile within a shared context of practical action, rather than deducing a framework from theory. These approaches were also intersubjective, in the sense of seeking to understand and empathise with each other’s perspectives, identify a common core if applicable, and build consensus about how to represent what matters to the group as a whole within the specified context. The indicators created in this way can be described as ‘values-based’.

In the first of the research studies presented here, I describe the organisational impacts of conducting field trials with a ‘values-based evaluation’ approach in eight different organisations. These preliminary findings are discussed in relation to wider academic discourse on values communication in organisations. Having established the potential for organisational benefits of values-based evaluation in practice, I proceed to investigate its theoretical validity in more depth. Specifically, I refute the theoretical argument which is often used to suggest that values enactment is impossible to measure, and replace it with a ‘context-specific measurability’ argument which suggests that intersubjective conceptualization of values can be achieved within clearly-defined practical contexts. This is followed up with a short case study of the use of inductive/intersubjective approaches to evaluate an online course in sustainability leadership, a rigorous investigation of how such approaches might contribute to conversations around indicator design in relation to the Education for Sustainable Development (ESD) target of the Sustainable Development Goals, and an exploration of values-based reflection at the level of individual schools or classes.  In the final chapter, I discuss my research processes and findings in the light of recent explorations of transformative, transgressive and transdisciplinary (which I term ‘triple-T’) learning initiatives.