Nudging Towards Health in a Conversational Food Recommender System Using Multi-Modal Interactions and Nutrition Labels

G. Castiglia; Ayoub El Majjodi; F. Calò; Y. Deldjoo; F. Narducci; A.D. Starke; Christoph Trattner

Nudging Towards Health in a Conversational Food Recommender System Using Multi-Modal Interactions and Nutrition Labels

G. Castiglia^*, Ayoub El Majjodi, F. Calò, Y. Deldjoo, F. Narducci, A.D. Starke, Christoph Trattner

^*Corresponding author for this work

Marketing and Consumer Behaviour

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › Academic › peer-review

3 Citations (Scopus)

Abstract

Humans engage with other humans and their surroundings through various modalities, most notably speech, sight, and touch. In a conversation, all these inputs provide an overview of how another person is feeling. When translating these modalities to a digital context, most of them are unfortunately lost. The majority of existing conversational recommender systems (CRSs) rely solely on natural language or basic click-based interactions. This work is one of the first studies to examine the influence of multi-modal interactions in a conversational food recommender system. In particular, we examined the effect of three distinct interaction modalities: pure textual, multi-modal (text plus visuals), and multi-modal supplemented with nutritional labeling. We conducted a user study (𝑁=195) to evaluate the three interaction modalities in terms of how effectively they supported users in selecting healthier foods. Structural equation modelling revealed that users engaged more extensively with the multi-modal system that was annotated with labels, compared to the system with a single modality, and in turn evaluated it as more effective.

Original language	English
Title of host publication	Proceedings of the Fourth Knowledge-aware and Conversational Recommender Systems Workshop co-located with 16th ACM Conference on Recommender Systems (RecSys 2022)
Editors	V.W. Anelli, P. Basile, G. de Melo, F.M. Donini, A. Ferrara, C. Musto, F. Narducci, A. Ragone, M. Zanker
Publisher	CEUR Workshop Proceedings (CEUR-WS.org)
Pages	29-35
Number of pages	8
Volume	3294
Publication status	Published - 2022
Event	4th Edition of Knowledge-aware and Conversational Recommender Systems (KaRS) Workshop @ RecSys 2022 - Seattle, United States Duration: 18 Sept 2022 → 23 Sept 2022 https://recsys.acm.org/recsys22/

Publication series

Name	CEUR Workshop Proceedings
ISSN (Electronic)	1613-0073

Conference/symposium

Conference/symposium	4th Edition of Knowledge-aware and Conversational Recommender Systems (KaRS) Workshop @ RecSys 2022
Abbreviated title	RecSys 2022
Country/Territory	United States
City	Seattle
Period	18/09/22 → 23/09/22
Internet address	https://recsys.acm.org/recsys22/

Keywords

Personalization
Health
Food recommendation
Digital Nudges
Nutrition labels

Access to Document

https://edepot.wur.nl/581683Licence: CC BY
https://ceur-ws.org/Vol-3294/long1.pdfLicence: CC BY

Cite this

Castiglia, G., El Majjodi, A., Calò, F., Deldjoo, Y., Narducci, F., Starke, A. D., & Trattner, C. (2022). Nudging Towards Health in a Conversational Food Recommender System Using Multi-Modal Interactions and Nutrition Labels. In V. W. Anelli, P. Basile, G. de Melo, F. M. Donini, A. Ferrara, C. Musto, F. Narducci, A. Ragone, & M. Zanker (Eds.), Proceedings of the Fourth Knowledge-aware and Conversational Recommender Systems Workshop co-located with 16th ACM Conference on Recommender Systems (RecSys 2022) (Vol. 3294, pp. 29-35). (CEUR Workshop Proceedings). CEUR Workshop Proceedings (CEUR-WS.org). https://edepot.wur.nl/581683

Castiglia, G. ; El Majjodi, Ayoub ; Calò, F. et al. / Nudging Towards Health in a Conversational Food Recommender System Using Multi-Modal Interactions and Nutrition Labels. Proceedings of the Fourth Knowledge-aware and Conversational Recommender Systems Workshop co-located with 16th ACM Conference on Recommender Systems (RecSys 2022). editor / V.W. Anelli ; P. Basile ; G. de Melo ; F.M. Donini ; A. Ferrara ; C. Musto ; F. Narducci ; A. Ragone ; M. Zanker. Vol. 3294 CEUR Workshop Proceedings (CEUR-WS.org), 2022. pp. 29-35 (CEUR Workshop Proceedings).

@inproceedings{a1ecf6f7c1f249af8602f6f69e2ea76d,

title = "Nudging Towards Health in a Conversational Food Recommender System Using Multi-Modal Interactions and Nutrition Labels",

abstract = "Humans engage with other humans and their surroundings through various modalities, most notably speech, sight, and touch. In a conversation, all these inputs provide an overview of how another person is feeling. When translating these modalities to a digital context, most of them are unfortunately lost. The majority of existing conversational recommender systems (CRSs) rely solely on natural language or basic click-based interactions. This work is one of the first studies to examine the influence of multi-modal interactions in a conversational food recommender system. In particular, we examined the effect of three distinct interaction modalities: pure textual, multi-modal (text plus visuals), and multi-modal supplemented with nutritional labeling. We conducted a user study (푁=195) to evaluate the three interaction modalities in terms of how effectively they supported users in selecting healthier foods. Structural equation modelling revealed that users engaged more extensively with the multi-modal system that was annotated with labels, compared to the system with a single modality, and in turn evaluated it as more effective. ",

keywords = "Personalization, Health, Food recommendation, Digital Nudges, Nutrition labels",

author = "G. Castiglia and {El Majjodi}, Ayoub and F. Cal{\`o} and Y. Deldjoo and F. Narducci and A.D. Starke and Christoph Trattner",

year = "2022",

language = "English",

volume = "3294",

series = "CEUR Workshop Proceedings",

publisher = "CEUR Workshop Proceedings (CEUR-WS.org)",

pages = "29--35",

editor = "V.W. Anelli and P. Basile and {de Melo}, G. and F.M. Donini and A. Ferrara and C. Musto and F. Narducci and A. Ragone and M. Zanker",

booktitle = "Proceedings of the Fourth Knowledge-aware and Conversational Recommender Systems Workshop co-located with 16th ACM Conference on Recommender Systems (RecSys 2022)",

note = "4th Edition of Knowledge-aware and Conversational Recommender Systems (KaRS) Workshop @ RecSys 2022, RecSys 2022 ; Conference date: 18-09-2022 Through 23-09-2022",

url = "https://recsys.acm.org/recsys22/",

}

Castiglia, G, El Majjodi, A, Calò, F, Deldjoo, Y, Narducci, F, Starke, AD & Trattner, C 2022, Nudging Towards Health in a Conversational Food Recommender System Using Multi-Modal Interactions and Nutrition Labels. in VW Anelli, P Basile, G de Melo, FM Donini, A Ferrara, C Musto, F Narducci, A Ragone & M Zanker (eds), Proceedings of the Fourth Knowledge-aware and Conversational Recommender Systems Workshop co-located with 16th ACM Conference on Recommender Systems (RecSys 2022). vol. 3294, CEUR Workshop Proceedings, CEUR Workshop Proceedings (CEUR-WS.org), pp. 29-35, 4th Edition of Knowledge-aware and Conversational Recommender Systems (KaRS) Workshop @ RecSys 2022, Seattle, Washington, United States, 18/09/22. <https://edepot.wur.nl/581683>

Nudging Towards Health in a Conversational Food Recommender System Using Multi-Modal Interactions and Nutrition Labels. / Castiglia, G.; El Majjodi, Ayoub; Calò, F. et al.
Proceedings of the Fourth Knowledge-aware and Conversational Recommender Systems Workshop co-located with 16th ACM Conference on Recommender Systems (RecSys 2022). ed. / V.W. Anelli; P. Basile; G. de Melo; F.M. Donini; A. Ferrara; C. Musto; F. Narducci; A. Ragone; M. Zanker. Vol. 3294 CEUR Workshop Proceedings (CEUR-WS.org), 2022. p. 29-35 (CEUR Workshop Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › Academic › peer-review

TY - GEN

T1 - Nudging Towards Health in a Conversational Food Recommender System Using Multi-Modal Interactions and Nutrition Labels

AU - Castiglia, G.

AU - El Majjodi, Ayoub

AU - Calò, F.

AU - Deldjoo, Y.

AU - Narducci, F.

AU - Starke, A.D.

AU - Trattner, Christoph

PY - 2022

Y1 - 2022

N2 - Humans engage with other humans and their surroundings through various modalities, most notably speech, sight, and touch. In a conversation, all these inputs provide an overview of how another person is feeling. When translating these modalities to a digital context, most of them are unfortunately lost. The majority of existing conversational recommender systems (CRSs) rely solely on natural language or basic click-based interactions. This work is one of the first studies to examine the influence of multi-modal interactions in a conversational food recommender system. In particular, we examined the effect of three distinct interaction modalities: pure textual, multi-modal (text plus visuals), and multi-modal supplemented with nutritional labeling. We conducted a user study (푁=195) to evaluate the three interaction modalities in terms of how effectively they supported users in selecting healthier foods. Structural equation modelling revealed that users engaged more extensively with the multi-modal system that was annotated with labels, compared to the system with a single modality, and in turn evaluated it as more effective.

AB - Humans engage with other humans and their surroundings through various modalities, most notably speech, sight, and touch. In a conversation, all these inputs provide an overview of how another person is feeling. When translating these modalities to a digital context, most of them are unfortunately lost. The majority of existing conversational recommender systems (CRSs) rely solely on natural language or basic click-based interactions. This work is one of the first studies to examine the influence of multi-modal interactions in a conversational food recommender system. In particular, we examined the effect of three distinct interaction modalities: pure textual, multi-modal (text plus visuals), and multi-modal supplemented with nutritional labeling. We conducted a user study (푁=195) to evaluate the three interaction modalities in terms of how effectively they supported users in selecting healthier foods. Structural equation modelling revealed that users engaged more extensively with the multi-modal system that was annotated with labels, compared to the system with a single modality, and in turn evaluated it as more effective.

KW - Personalization

KW - Health

KW - Food recommendation

KW - Digital Nudges

KW - Nutrition labels

M3 - Conference paper

VL - 3294

T3 - CEUR Workshop Proceedings

SP - 29

EP - 35

BT - Proceedings of the Fourth Knowledge-aware and Conversational Recommender Systems Workshop co-located with 16th ACM Conference on Recommender Systems (RecSys 2022)

A2 - Anelli, V.W.

A2 - Basile, P.

A2 - de Melo, G.

A2 - Donini, F.M.

A2 - Ferrara, A.

A2 - Musto, C.

A2 - Narducci, F.

A2 - Ragone, A.

A2 - Zanker, M.

PB - CEUR Workshop Proceedings (CEUR-WS.org)

T2 - 4th Edition of Knowledge-aware and Conversational Recommender Systems (KaRS) Workshop @ RecSys 2022

Y2 - 18 September 2022 through 23 September 2022

ER -

Castiglia G, El Majjodi A, Calò F, Deldjoo Y, Narducci F, Starke AD et al. Nudging Towards Health in a Conversational Food Recommender System Using Multi-Modal Interactions and Nutrition Labels. In Anelli VW, Basile P, de Melo G, Donini FM, Ferrara A, Musto C, Narducci F, Ragone A, Zanker M, editors, Proceedings of the Fourth Knowledge-aware and Conversational Recommender Systems Workshop co-located with 16th ACM Conference on Recommender Systems (RecSys 2022). Vol. 3294. CEUR Workshop Proceedings (CEUR-WS.org). 2022. p. 29-35. (CEUR Workshop Proceedings).

Nudging Towards Health in a Conversational Food Recommender System Using Multi-Modal Interactions and Nutrition Labels

Abstract

Publication series

Conference/symposium

Keywords

Access to Document

Fingerprint

Cite this