Meet the peer and the cook: Design and evaluation of persuasive Embodied Conversational Agents to support healthy ageing

The ageing of the global population is the most important medical and social demographic problem worldwide. Promoting health, especially regarding nutrition and loneliness, among older adults can contribute to an increase in healthy life years and life expectancy. eHealth services can be used to deliver such interventions. However, they are often not tailored and persuasive enough to be effective. Embodied Conversational Agents (ECAs) are known to improve the persuasiveness of eHealth services. Studies among older adults show high ratings of acceptance, enjoyment, and usability, but evidence regarding their effectiveness and working mechanisms is limited. Therefore, the aim of this thesis was to provide insight into how to design and evaluate ECAs that support healthy living. The following sub-questions were formulated:

• How are ECAs designed and evaluated?
• How should ECAs support older adults with healthy eating?
• Do matching topics and appearances effect ECAs’ personality characteristics and persuasiveness?
• How should PACO be evaluated?
• What factors influence the use and effect of ECAs?

Chapter 2 explored the current practices in designing and evaluating ECAs in the health domain. The Arksey and O’Malley framework was used to conduct a scoping review. The results showed that ECAs most often targeted physical activity and had the appearance of a middle-aged African–American woman. Regarding content, multiple behavior change techniques and theories or principles were applied, but their interpretation and application were usually not reported. During the development process, human-centered and stakeholder-inclusive design approaches tended not to be used. Regarding evaluation, a combination of efficacy and use-related outcomes were assessed, usually in an RCT. However, rather than evaluating specific components, the interventions were evaluated as a whole. Overall, the studies suggested that ECAs for coaching people in relation to a healthy lifestyle can make an intervention more engaging, although evidence on their effectiveness on health remained inconclusive.

Chapter 3 marks the start of the development of PACO. This chapter described a co-creation process with older adults that informed both the content and the appearance of the ECA. Data were gathered through three consecutive iterations of co-design sessions with two groups of community-dwelling older adults in the Netherlands. The first main finding was that older adults approach eating from a holistic perspective. This meant that they evaluated eating not only in terms of nutrients, ingredients, or components, but also in terms of eating mindfully and in a well-balanced way. Second, action planning and self-monitoring were the preferred approaches towards changing eating behavior among older adults. The third and last main finding was that ECAs have the potential to support older adults with healthy eating behaviors in an engaging manner. Next, the requirements for the ECA were discussed. There were five preferred personality characteristics: friendly, warm, trustworthy, concerned, and competent. The communication style should contain some humor and non-judgmental language.

The design of the ECA was further discussed in Chapter 4, which described a study with the goal of identifying the effect of personality characteristics and persuasiveness on a match between the ECA and the health topic. Via an online experiment, three different ECAs (a peer, a cook, and a fantasy figure) and three different health topics (cooking, food, and loneliness) were tested, and the results were triangulated with qualitative insights from a focus group. The results revealed that older adults preferred an ECA whose appearance matched a certain health topic, and this design scored high ratings on persuasiveness and intention to use. However, positive personality traits were not appreciated better as a result of a match. In a focus group, it was explored how both design and personality evaluations should be further improved. Multiple specific design changes were made, and two background stories were developed, together with a list of requirements for the tone of voice in the dialogue script.

The protocol to evaluate the use and the effect on health behavior change of PACO, an extensive description of PACO, and the design process were presented in Chapter 5. In addition, two conceptual models were created – one for use and one for effect. The conceptual model explaining ECA use has usability and perceived usefulness as the main predictors for use, which is expected to lead to an improved relationship with the ECA. Good aesthetics are further expected to lead to an improved usability. Enjoyment, privacy concerns, and perceived control are expected to lead to a higher score on perceived usefulness. The conceptual model explaining health effects has the different behavior change techniques as predictors for the three basis psychological needs (autonomy, competence, and relatedness). An improvement in satisfying these needs is expected to lead to improved eating behavior and fewer feelings of loneliness, all of which is expected to lead to an overall improvement in quality of life.

The results of the summative evaluation and the verification of the conceptual models were described in Chapter 6. The results showed a significant correlation between use in minutes on the one hand and perceived usefulness and enjoyment on the other. However, these did not predict use in the full regression model. Additionally, PACO use did not lead to improvements in eating behavior or a decrease in loneliness. The study did not provide conclusive evidence about factors that were linked to the use or health effects of ECAs.

Chapter 7 presented the main conclusion and practical guidelines and reflected critically on the work presented in this thesis. In conclusion, this thesis showed that an ECA can be used as a tool to increase users’ engagement with an eHealth intervention. Moreover, this thesis provided a use-case of how to develop such an ECA: via a human-centered and stakeholder- inclusive design approach, incorporating grounded behavioral theory, operationalized via various behavior change techniques. Furthermore, this thesis showed how to evaluate an ECA intervention: by developing conceptual models, focusing on both the use and the effect, and combining qualitative, quantitative, and log data.