Research | School of Design Has Eight Papers Accepted by Top-tier HCI Conference ACM CHI

Finger wearable haptic devices enrich virtual reality experiences by offering haptic feedback corresponding to the virtual environment. However, despite the effectiveness of current finger wearable haptic devices in delivering haptic feedback, many are often constrained in their ability to provide force feedback across a diverse range of directions or to sustain it. Therefore, we present AirPush, a finger wearable haptic device capable of generating continuously adjustable force feedback in multiple directions using compressed air. To evaluate its usability, we conducted a technical evaluation and four user studies: (1) we obtained the user’s perceptual thresholds of angles under different directions on horizontal and vertical planes, (2) in perception studies, we found that users can identify five different magnitudes of force and eight different motion when using AirPush, and (3) using it in VR applications, we confirmed that users felt more realistic and immersed when using AirPush than the HTC VIVE Controller or AirPush with a fixed nozzle.

With the digitalization and mobilization of the society, older people face the challenge of maintaining high-quality communication with their younger family members who move out and lead separate lives at a distance. In HCI, little work is done to understand the social dynamics between distributed families and their remote communication mediated by the technologies. To identify design opportunities to support their remote communication, we conducted interviews with nine family pairs composed of distributed intergenerational family members. In addition, we interviewed eight community volunteers to formulate a perspective of social service providers. Our paper contributes to the HCI community by providing an account of the social dynamics mediated by communication technologies between older adults and their remote families, and opportunities to promote their social connections from a multi-stakeholder perspective. This paper presents valuable insights for designers aiming to enhance wellbeing of older adults within the context of distributed families.

Storymaking, as an integrative form of expressive arts therapy, is an effective means to foster family communication. Yet, the integration of generative AI as expressive materials in therapeutic storymaking remains underexplored. And there is a lack of HCI implications on how to support families and therapists in this context. Addressing this, our study involved five weeks of storymaking sessions with seven families guided by a professional therapist. In these sessions, the families used both traditional art-making materials and image-based generative AI to create and evolve their family stories. Via the rich empirical data and commentaries from four expert therapists, we contextualize how families creatively melded AI and traditional expressive materials to externalize their ideas and feelings. Through the lens of Expressive Therapies Continuum (ETC), we characterize the therapeutic implications of AI as expressive materials. Desirable interaction qualities to support children, parents, and therapists are distilled for future HCI research.

Voice messages, by nature, prevent users from gauging the emotional tone without fully diving into the audio content. This hinders the shared emotional experience at the pre-retrieval stage. Research scarcely explored "Emotional Teasers"-pre-retrieval cues offering a glimpse into an awaiting message's emotional tone without disclosing its content. We introduce EmoWear, a smartwatch voice messaging system enabling users to apply 30 animation teasers on message bubbles to reflect emotions. EmoWear eases senders' choice by prioritizing emotions based on semantic and acoustic processing. EmoWear was evaluated in comparison with a mirroring system using color-coded message bubbles as emotional cues (N=24). Results showed EmoWear significantly enhanced emotional communication experience in both receiving and sending messages. The animated teasers were considered intuitive and valued for diverse expressions. Desirable interaction qualities and practical implications are distilled for future design. We thereby contribute both a novel system and empirical knowledge concerning emotional teasers for voice messaging.

This Late-Breaking Work explores the significance of socio-emotional learning (SEL) and the challenges inherent in designing child-appropriate technologies, namely storytelling agents, to support SEL. We aim to probe their needs and preferences regarding agents for SEL by conducting co-design which involves children co-creating characters and social-emotional stories. We conducted collaborative story-making activities with children aged four to six years old. Our findings could inform the design of both verbal and nonverbal interactions of agents, which are to be aligned with children's understanding and interest. Based on the child-led peer co-design, our work enhances the understanding of SEL agent designs and behaviors tailored to children's socio-emotional needs, thereby offering practical implications for more effective SEL tools in future HCI research and practice.

Studies of Generative AI (GenAI)-assisted creative workflows have focused on individuals overcoming challenges of prompting to produce what they envisioned. When designers work in teams, how do collaboration and prompting influence each other, and how do users perceive generative AI and their collaborators during the co-prompting process? We engaged students with design or performance backgrounds, and little exposure to GenAI, to work in pairs with GenAI to create stage designs based on a creative theme. We found two patterns of collaborative prompting focused on generating story descriptions first, or visual imagery first. GenAI tools helped participants build consensus in the task, and allowed for discussion of the prompting strategies. Participants perceived GenAI as efficient tools rather than true collaborators, suggesting that human partners reduced the reliance on their use. This work highlights the importance of human-human collaboration when working with GenAI tools, suggesting systems that take advantage of shared human expertise in the prompting process. 

Otome games (also known as romantic video games, or RVGs) are story-based video games that are designed for young women, simulating the experience of a romantic relationship. Players are invited to adopt the female avatar’s perspective in the story and date one or more of the male characters. Our empirical study focuses on the different types of social support among the players of the Chinese otome game Mr. Love: Queen’s Choice. We discovered that although the game was initially designed to be a consumer product aiming to profit from a largely marginalized and stigmatized group of gamers, i.e., young female gamers, the game has created a gaming community in which the players seek and provide each other with social support. We primarily use ethnographic methods, such as participant observation and in-depth interviews. Our study contributes to HCI research on mediated social support in game.