Visualising emotion in support of patient-physician communication: an empirical study

Patient-physician communication is a crucial aspect of clinical diagnoses and treatments. However, there are barriers to effective empathic practices, including consciousness, busy working rhythms, and difficulties recognising patients’ implicit emotional expressions. While previous research has attempted to support asynchronous medical conversations, this study has explored the use of emotion visualisation techniques for synchronous, face-to-face medical encounters. After interviewing doctors to understand user requirements, an emotion-visualisation prototype, EMVIS, was created. The prototype was evaluated in a study with 31 patients and 37 healthcare providers within different specialist groups using a contextualised Technology Acceptance Model (TAM) and follow-up interviews. The results indicated that patients and physicians were generally accepting of emotion visualisation for medical encounters. Patients were more interested in their physicians’ attitudes and intentions, while physicians accepted the visualisation, but their requirements differed according to their skill levels and specialities. Hence, four supportive factors - emotional empathy, careful attention, human connection, and reflective conversation - elicited information on how EMVIS contributed to medical conversations. Five future opportunities for the emotion visualisation of medical conversations were discussed in respect of the human factors and potential requirements. These include communicating uncertainty, addressing user diversity, providing explanatory information, managing attention, and supporting negotiations.