A New Approach to Measure User Experience with Voice-Controlled Intelligent Assistants: A Pilot Study

Félix Le Pailleur; Bo Huang; Pierre Majorique Léger; Sylvain Sénécal

doi:10.1007/978-3-030-49062-1_13

A New Approach to Measure User Experience with Voice-Controlled Intelligent Assistants: A Pilot Study

Félix Le Pailleur, Bo Huang, Pierre Majorique Léger, Sylvain Sénécal

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

10 Citations (Scopus)

Abstract

Voice-controlled intelligent assistants use a conversational user interface (CUI), a system that relies on natural language processing and artificial intelligence to have verbal interactions with end-users. In this research, we propose a multi-method approach to assess user experience with a smart voice assistant through triangulation of psychometric and psychophysiological measures. The approach aims to develop a richer understanding of what the users experience during the interaction, which could provide new insights to researchers and developers in the field of voice assistant. We apply this new approach in a pilot study, and we show that each method captures a part of emotional variance during the interaction. Results suggest that emotional valence is better captured with psychometric measures, whereas arousal is better detected with psychophysiological measures.

Original language	English
Title of host publication	Human-Computer Interaction. Multimodal and Natural Interaction - Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings
Editors	Masaaki Kurosu
Publisher	Springer
Pages	197-208
Number of pages	12
ISBN (Print)	9783030490614
DOIs	https://doi.org/10.1007/978-3-030-49062-1_13
Publication status	Published - 2020
Externally published	Yes
Event	Thematic Area on Human Computer Interaction, HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020 - Copenhagen, Denmark Duration: 19 Jul 2020 → 24 Jul 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12182 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Thematic Area on Human Computer Interaction, HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020
Country/Territory	Denmark
City	Copenhagen
Period	19/07/20 → 24/07/20

Keywords

Arousal
Conversational user interface
Emotion
Human-Computer Interactions
User experience
Valence
Vocal assistant

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-49062-1_13

Cite this

Le Pailleur, F., Huang, B., Léger, P. M., & Sénécal, S. (2020). A New Approach to Measure User Experience with Voice-Controlled Intelligent Assistants: A Pilot Study. In M. Kurosu (Ed.), Human-Computer Interaction. Multimodal and Natural Interaction - Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings (pp. 197-208). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12182 LNCS). Springer. https://doi.org/10.1007/978-3-030-49062-1_13

Le Pailleur, Félix ; Huang, Bo ; Léger, Pierre Majorique et al. / A New Approach to Measure User Experience with Voice-Controlled Intelligent Assistants : A Pilot Study. Human-Computer Interaction. Multimodal and Natural Interaction - Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings. editor / Masaaki Kurosu. Springer, 2020. pp. 197-208 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a52da4afc966426a9283d900160dbd9f,

title = "A New Approach to Measure User Experience with Voice-Controlled Intelligent Assistants: A Pilot Study",

abstract = "Voice-controlled intelligent assistants use a conversational user interface (CUI), a system that relies on natural language processing and artificial intelligence to have verbal interactions with end-users. In this research, we propose a multi-method approach to assess user experience with a smart voice assistant through triangulation of psychometric and psychophysiological measures. The approach aims to develop a richer understanding of what the users experience during the interaction, which could provide new insights to researchers and developers in the field of voice assistant. We apply this new approach in a pilot study, and we show that each method captures a part of emotional variance during the interaction. Results suggest that emotional valence is better captured with psychometric measures, whereas arousal is better detected with psychophysiological measures.",

keywords = "Arousal, Conversational user interface, Emotion, Human-Computer Interactions, User experience, Valence, Vocal assistant",

author = "{Le Pailleur}, F{\'e}lix and Bo Huang and L{\'e}ger, {Pierre Majorique} and Sylvain S{\'e}n{\'e}cal",

note = "Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; Thematic Area on Human Computer Interaction, HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020 ; Conference date: 19-07-2020 Through 24-07-2020",

year = "2020",

doi = "10.1007/978-3-030-49062-1_13",

language = "English",

isbn = "9783030490614",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "197--208",

editor = "Masaaki Kurosu",

booktitle = "Human-Computer Interaction. Multimodal and Natural Interaction - Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings",

}

Le Pailleur, F, Huang, B, Léger, PM & Sénécal, S 2020, A New Approach to Measure User Experience with Voice-Controlled Intelligent Assistants: A Pilot Study. in M Kurosu (ed.), Human-Computer Interaction. Multimodal and Natural Interaction - Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12182 LNCS, Springer, pp. 197-208, Thematic Area on Human Computer Interaction, HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020, Copenhagen, Denmark, 19/07/20. https://doi.org/10.1007/978-3-030-49062-1_13

A New Approach to Measure User Experience with Voice-Controlled Intelligent Assistants: A Pilot Study. / Le Pailleur, Félix; Huang, Bo; Léger, Pierre Majorique et al.
Human-Computer Interaction. Multimodal and Natural Interaction - Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings. ed. / Masaaki Kurosu. Springer, 2020. p. 197-208 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12182 LNCS).

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A New Approach to Measure User Experience with Voice-Controlled Intelligent Assistants

T2 - Thematic Area on Human Computer Interaction, HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020

AU - Le Pailleur, Félix

AU - Huang, Bo

AU - Léger, Pierre Majorique

AU - Sénécal, Sylvain

PY - 2020

Y1 - 2020

N2 - Voice-controlled intelligent assistants use a conversational user interface (CUI), a system that relies on natural language processing and artificial intelligence to have verbal interactions with end-users. In this research, we propose a multi-method approach to assess user experience with a smart voice assistant through triangulation of psychometric and psychophysiological measures. The approach aims to develop a richer understanding of what the users experience during the interaction, which could provide new insights to researchers and developers in the field of voice assistant. We apply this new approach in a pilot study, and we show that each method captures a part of emotional variance during the interaction. Results suggest that emotional valence is better captured with psychometric measures, whereas arousal is better detected with psychophysiological measures.

AB - Voice-controlled intelligent assistants use a conversational user interface (CUI), a system that relies on natural language processing and artificial intelligence to have verbal interactions with end-users. In this research, we propose a multi-method approach to assess user experience with a smart voice assistant through triangulation of psychometric and psychophysiological measures. The approach aims to develop a richer understanding of what the users experience during the interaction, which could provide new insights to researchers and developers in the field of voice assistant. We apply this new approach in a pilot study, and we show that each method captures a part of emotional variance during the interaction. Results suggest that emotional valence is better captured with psychometric measures, whereas arousal is better detected with psychophysiological measures.

KW - Arousal

KW - Conversational user interface

KW - Emotion

KW - Human-Computer Interactions

KW - User experience

KW - Valence

KW - Vocal assistant

UR - http://www.scopus.com/inward/record.url?scp=85088753951&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-49062-1_13

DO - 10.1007/978-3-030-49062-1_13

M3 - Conference contribution

AN - SCOPUS:85088753951

SN - 9783030490614

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 197

EP - 208

BT - Human-Computer Interaction. Multimodal and Natural Interaction - Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings

A2 - Kurosu, Masaaki

PB - Springer

Y2 - 19 July 2020 through 24 July 2020

ER -

Le Pailleur F, Huang B, Léger PM, Sénécal S. A New Approach to Measure User Experience with Voice-Controlled Intelligent Assistants: A Pilot Study. In Kurosu M, editor, Human-Computer Interaction. Multimodal and Natural Interaction - Thematic Area, HCI 2020, Held as Part of the 22nd International Conference, HCII 2020, Proceedings. Springer. 2020. p. 197-208. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-49062-1_13

A New Approach to Measure User Experience with Voice-Controlled Intelligent Assistants: A Pilot Study

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this