Posted on

Enhancing Social Messaging with Mediated Social Touch

Qianhui Wei, Jun Hu & Min Li

Mediated social touch (MST) is a popular way to communicate emotion and connect people in mobile communication. This article applies MST gestures with vibrotactile stimuli in two online communication modes—asynchronous and synchronous communication (texting and video calling) to enhance social presence for mobile communication. We first designed the application that included the visual design of MST gestures, the vibrotactile stimuli design for MST gestures, and the interface design for texting and video calling. Then, we conducted a user study to explore if the MST gestures with vibrotactile stimuli could increase social presence in texting and video calling compared to MST gestures without vibrotactile stimuli. We also explored if the communication modes affected the social presence significantly when applying MST signals. The quantitative data analysis shows that adding vibrotactile stimuli to MST gestures helps to increase social presence in the aspects of co-presence, perceived behavior interdependence, perceived affective understanding, and perceived emotional interdependence. Adding vibrotactile stimuli to MST gestures causes no significant differences in attentional allocation and perceived message understanding. There is no significant difference between texting and video calling when applying MST signals in mobile communication. The qualitative data analysis shows that participants think MST gestures with vibrotactile stimuli are interesting, and they are willing to use them in mobile communication, but the application design should be iterated based on their feedback.

Q. Wei, J. Hu, and M. Li, “Enhancing Social Messaging with Mediated Social Touch,” International Journal of Human–Computer Interaction, vol. 40, no. 7, pp. 1669–1688, 2024.
FULLTEXT: PDF REFERENCE: BibTeX EndNote
DOI: 10.1080/10447318.2022.2148883

Posted on

I’m not upset–I get it

  • Help-seekers adjust their diction to be more empathic and tactful when they perceive co-workers as stressed.
  • Help-seekers experience less negative emotional shifts when rejected by stressed rather than relaxed co-workers, suggesting empathy towards stressed individuals.
  • Social messaging applications could boost workplace empathy by enabling stress status sharing, albeit with caution for relaxed status displays.
  • For privacy and reduced stigmatization, using subtle haptic vibrations could be considered for signaling users’ negative or sensitive emotional states.
N. Zhang, B. Yu, J. Hu, M. Li, and P. An, “I’m Not Upset–I Get It: Effects of Co-workers’ Stress Cues on Help-seekers’ Social Diction and Empathy in Telecommuting,” International Journal of Human-Computer Studies, pp. 103218, 2024/01/12/, 2024. FULLTEXT: PDF REFERENCE: BibTeX EndNote DOI: https://doi.org/10.1016/j.ijhcs.2024.103218
Posted on

Mediated Social Touch with Mobile Devices: A Review of Designs and Evaluations

Qianhui Wei, Min Li, Jun Hu

Background: Mediated social touch has been widely studied for remote affective communication in the field of human-computer interaction. Goal: We conducted this literature review to comprehensively understand the state of the art of the designs and evaluations of mediated social touch for mobile devices. Method: We selected 52 articles based on related keywords from four main digital libraries, i.e., ACM, IEEE, Springer, and Scopus. Results: We summarized from these articles how mediated social touch signal is designed, prototyped, and evaluated, and what the main research findings are. Based on the analysis, we identified opportunities for later work.

Q. Wei, M. Li, and J. Hu, “Mediated Social Touch with Mobile Devices: A Review of Designs and Evaluations,” IEEE Transactions on Haptics, vol. Early Access, Oct 25, 2023, 2023. FULLTEXT: PDF REFERENCE: BibTeX EndNote DOI: 10.1109/TOH.2023.3327506
Posted on

Enhanced Presence Evaluation in Virtual Reality Feedback System with TOPSIS Model

Shuo Li et al.

Researcher has been trying to optimize the method for evaluating presence in virtual reality (VR) to address variability and uncertainty in quick evaluations using questionnaires. We recommend using the Technique for Order Performance by Similarity to Ideal Solution (TOPSIS) model to calculate the iGroup Presence Questionnaire to measure users’ overall presence in VR prototyping, instead of the weighted sum approach. The effects of two presence factors (scene density and motion trajectory technology) on a self-developed VR swimming virtual system were tested using the TOPSIS model with 20 participants each 12 tasks in a user experiment. The results were compared using two different weighting methods, fuzzy hierarchical analysis and uniform weighting methods. TOPSIS had a narrower range of data within the 95% confidence interval and a significantly lower coefficient of variation (CV). This indicates enhanced precision in evaluating presence and can be used to compare different technique setting of virtual systems.

S. Li, H. Zheng, T. Han, J. Hu, C. Zhang, and C. Yu, “Enhanced Presence Evaluation in Virtual Reality Feedback System with TOPSIS Model,” International Journal of Human–Computer Interaction, vol. Published online, pp. 1-15, 2023. FULLTEXT: PDF REFERENCE: BibTeX EndNote DOI: 10.1080/10447318.2023.2263697
Posted on

Robot Design Competition Award at IEEE RO-MAN 2023

Best Robot Design Process Award @ IEEE RO-MAN 2023 Robot Design Competition, for the design Petting Pen for Stress Awareness and Management in Children. We are very proud of our PhD candidates Jing Li and Pinhao Wang!

J. Li, P. Wang, E. Barakova, and J. Hu, “Petting Pen for Stress Awareness and Management in Children,” in 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Robot Design Competition Award – Robot Design Process, 2023. FULLTEXT: PDF REFERENCE: BibTeX EndNote
Posted on

Dutch Design Engineering Science at Dutch Design Week: Examples of Design Research on Social Computing

Together with our former PhDs, we were invited by a top Chinese design journal (Zhuangshi) to share our view on Dutch Design Engineering Science, with examples of our Dutch Design Week entries over the years, from the perspective of design research on social computing. The link is a Google-translated version of the article shared by the journal on WeChat.

The same issue features articles about Dutch design by J.W. Drukker, Herman van Bergeijk, and colleagues from TU Delft. 

S. Qiu, K. Kang, C. Li, C. Wang, M. Hoekstra, Y. Feng, and J. Hu, “Dutch Design Engineering Science at Dutch Design Week: Examples of Design Research on Social Computing,” Zhuangshi, vol. 2023, no. 4(360), pp. 23-28, 2023. FULLTEXT: PDF REFERENCE: BibTeX EndNote

WeChat post by the journal; and see also a Google translated version.

Continue reading Dutch Design Engineering Science at Dutch Design Week: Examples of Design Research on Social Computing
Posted on

ApEn: A Stress-Aware Pen for Children with Autism Spectrum Disorder

Jing Li, Emilia Barakova, Jun Hu, Wouter Staal & Martine van Dongen-Boomsma

See also the related Patent: https://patents.google.com/patent/NL2029559B1/en?oq=NL2029559B1

Children with Autism Spectrum disorder (ASD) often experience high levels of anxiety and stress. Many children with ASD have difficulty in being aware of their stress and communicating distress to family and caregivers. Stress detection and regulation are vital for their mental well-being. This paper presents a stress-aware pen (ApEn) that detects real-time stress-related behaviors and interacts with users with vibrotactile and light as a feedback indication of interpreted stress levels. ApEn is a context-aware tool for collecting behavioral data related to the expression of stress and can increase users’ stress awareness. A pilot test was conducted with typical developed children to investigate how to detect stress in their daily environment. The pilot test results indicate that ApEn is a promising tool for detecting stress-related behaviors and can attend the user about the detected stress through designed sensory feedback.

J. Li, E. Barakova, J. Hu, M. van Dongen-Boomsma, and W. Staal, Drawing instrument for biofeedback relating to stress, The Nehterlands NL2029559B1, Octrooicentrum Nederland, 2023. FULLTEXT: PDF REFERENCE: BibTeX EndNote
J. Li, E. Barakova, J. Hu, W. Staal, and M. van Dongen-Boomsma, “ApEn: A Stress-Aware Pen for Children with Autism Spectrum Disorder,” in International Work-Conference on the Interplay Between Natural and Artificial Computation, Cham, 2022, pp. 281-290. FULLTEXT: PDF REFERENCE: BibTeX EndNote DOI: 10.1007/978-3-031-06242-1_28
Posted on

Co-constructing Stories Based on Users Lived Experiences to Investigate Visualization Design for Collective Stress Management

Mengru Xue, Pengcheng An, Rong-Hao Liang, Zengrong Guo, Jun Hu, Preben Hansen,
Loe Feijs

Collective stress is the stress within a group or an organization. It affects individuals’ well-being and group productivity. HCI research has started exploring collective stress visualization to facilitate group awareness and collective coping via testing prototypes in controlled settings. However, an in-depth understanding of users’ needs and envisaged scenarios based on their authentic experiences are still lacking. In this study, we utilized a participatory approach called co-constructing stories to investigate how a collective stress visualization would be used in office workers’ authentic workday routines. We constructed use case stories with a group of office workers separately based on their personal lived experiences, using a design probe called AffectiveGarden. Our results categorized six clusters of benefits for collective coping through visualization and their implications for future design practice.

M. Xue, P. An, R.-H. Liang, Z. Guo, J. Hu, P. Hansen, and L. Feijs, “Co-constructing Stories Based on Users Lived Experiences to Investigate Visualization Design for Collective Stress Management,” in Proceedings of the 2023 ACM Designing Interactive Systems Conference, Pittsburgh, PA, USA, 2023, pp. 652–663. FULLTEXT: PDF REFERENCE: BibTeX EndNote DOI: 10.1145/3563657.3596118
Posted on

Can blindfolded users replace blind ones in product testing? an empirical study

Shi Qiu, Jun Hu, Ting Han &Matthias Rauterberg

During the design, it is important to evaluate the user experience of representative users in many human product interactions. But, in some cases, it is difficult or even impossible to recruit representative users because they have disabilities that do not allow them to take part in such investigations. Thus, alternative populations are widely studied. The most common way to replace real blind people is to use sighted but blindfolded users when studying design solutions. To test whether such alternative or proxy users can be used to represent blind people in social interactions, we examined the communication quality of 20 blind-sighted pairs and 20 blindfolded-sighted pairs in two different experiments. A prototype named E-Gaze glasses was evaluated as the testing tool. Results clearly show that the blindfolded participants achieved significantly higher communication quality than the blind participants. In qualitative data analysis, the blindfolded participants also reported their user experience of being blindfolded in conversations. Our qualitative results strengthen the conclusion that blindfolded users’ behaviour is different from real blind users’ behaviour. We recommend that blind users should not be substituted for blindfolded users in human product evaluations when communication quality is measured.

S. Qiu, J. Hu, T. Han, and M. Rauterberg, “Can Blindfolded Users Replace Blind Ones in Product Testing? An Empirical Study,” Behaviour & Information Technology, vol. Online, 06 Jul 2023, 2023. FULLTEXT: PDF REFERENCE: BibTeX EndNote DOI: 10.1080/0144929X.2023.2226768

Posted on

Context-Enhanced Human-Robot Interaction: Exploring the Role of System Interactivity and Multimodal Stimuli on the Engagement of People with Dementia

Yuan Feng et al.

Engaging people with dementia (PWD) in meaningful activities is the key to promote their quality of life. Design towards a higher level of user engagement has been extensively studied within the human-computer interaction community, however, few extend to PWD. It is generally considered that increased richness of experiences can lead to enhanced engagement. Therefore, this paper explores the effects of rich interaction in terms of the role of system interactivity and multimodal stimuli by engaging participants in context-enhanced human-robot interaction activities. The interaction with a social robot was considered context-enhanced due to the additional responsive sensory feedback from an augmented reality display. A field study was conducted in a Dutch nursing home with 16 residents. The study followed a two by two mixed factorial design with one within-subject variable – multimodal stimuli – and one between-subject variable – system interactivity. A mixed method of video coding analysis and observational rating scales was adopted to assess user engagement comprehensively. Results disclose that when additional auditory modality was included besides the visual-tactile stimuli, participants had significantly higher scores on attitude, more positive behavioral engagement during activity, and a higher percentage of communications displayed. The multimodal stimuli also promoted social interaction between participants and the facilitator. The findings provide sufficient evidence regarding the significant role of multimodal stimuli in promoting PWD’s engagement, which could be potentially used as a motivation strategy in future research to improve emotional aspects of activity-related engagement and social interaction with the human partner.

Y. Feng, G. Perugia, S. Yu, E. I. Barakova, J. Hu, and G. W. M. Rauterberg, “Context-Enhanced Human-Robot Interaction: Exploring the Role of System Interactivity and Multimodal Stimuli on the Engagement of People with Dementia,” International Journal of Social Robotics, vol. 14, no. 3, pp. 807-826, 2022/04/01, 2022.
FULLTEXT: PDF REFERENCE: BibTeX EndNote
DOI: 10.1007/s12369-021-00823-4