My Projects

In virtual reality (VR), our virtual body can have different characteristics from our real body, such as appearance, size, and even extra body parts. Previous research shows that haptic feedback enhances the user-perceived embodiment of those dissimilar avatars. In particular, weight-shifting devices showed the potential to enhance arm deformation. However, there has been no exploration of using such techniques to enhance embodiment with extra body parts, like wings. We introduce HapticWings, a back-wearable 2D weight-shifting device that provides haptic feedback for wing motions, enhancing the user embodiment of avatars with extra wings. In three user studies, we explored (1) users’ abilities to recognize different weight-shifting motions provided by HapticWings, (2) users’ perceived embodiment of avatars with extra wings when providing haptic feedback for wing motions, and (3) four possible applications and used two of them to evaluate users’ sense of realism and enjoyment in VR.

Accessing auditory information remains challenging for DHH individuals in real-world situations and multiplayer VR interactions. To improve this, we investigated caption designs that specialize in the needs of DHH users in multiplayer VR settings. First, we conducted three co-design workshops with DHH participants, social workers, and designers to gather insights into the specific needs of design directions for DHH users in the context of a room escape game in VR. We further refined our designs with 13 DHH users to determine the most preferred features. Based on this, we developed VRCaptions, a caption prototype for DHH users to better experience multiplayer conversations in VR. We lastly invited two mixed-hearing groups to participate in the VR room escape game with our VRCaptions to validate. The results demonstrate that VRCaptions can enhance the ability of DHH participants to access information and reduce the barrier to communication in VR.

Due to the lack of a universally accepted definition for the term “virtual twin”, there are varying degrees of similarity between physical prototypes and their virtual counterparts across different research papers. This variability complicates the comparison of results from these papers. To bridge this gap, we introduce the Scale of Virtual Twin’s Similarity (SVS), a questionnaire intended to quantify the similarity between a virtual twin and its physical counterpart in simple environments in terms of visual fidelity, physical fidelity, environmental fidelity, and functional fidelity. This paper describes the development process of the SVS questionnaire items and provides an initial evaluation through two between-subjects user studies to validate the items under the categories of visual and functional fidelity. Additionally, we discuss the way to apply it in research and development settings.

In the domain of Human-Computer Interaction, focus groups represent a widely utilised yet resource-intensive methodology, often demanding the expertise of skilled moderators and meticulous preparatory efforts. This study introduces the “Focus Agent,” a Large Language Model (LLM) powered framework that simulates both the focus group (for data collection) and acts as a moderator in a focus group setting with human participants. To assess the data quality derived from the Focus Agent, we ran five focus group sessions with a total of 23 human participants as well as deploying the Focus Agent to simulate these discussions with AI participants. Quantitative analysis indicates that Focus Agent can generate opinions similar to those of human participants. Furthermore, the research exposes some improvements associated with LLMs acting as moderators in focus group discussions that include human participants.

Locating out-of-view vehicles can help pedestrians to avoid critical traffic encounters. Some previous approaches focused solely on visualising out-of-view objects, neglecting their localisation and limitations. Other methods rely on continuous camera-based localisation, raising privacy concerns. Hence, we propose the ARcoustic system, which utilises a microphone array for nearby moving vehicle localisation and visualises nearby out-of-view vehicles to support pedestrians. First, we present the implementation of our sonic-based localisation and discuss the current technical limitations. Next, we present a user study (n = 18) in which we compared two state-of-the-art visualisation techniques (Radar3D, CompassbAR) to a baseline without any visualisation. Results show that both techniques present too much information, resulting in below-average user experience and longer response times. Therefore, we introduce a novel visualisation technique that aligns with the technical localisation limitations and meets pedestrians’ preferences for effective visualisation, as demonstrated in the second user study (n = 16). Lastly, we conduct a small field study (n = 8) testing our ARcoustic system under realistic conditions. Our work shows that out-of-view object visualisations must align with the underlying localisation technology and fit the concrete application scenario.

In this work, we describe our experience developing and evaluating the Cross-Reality Study Tool (CReST), which allows researchers to conduct and observe Virtual Reality (VR) user studies from an Augmented Reality perspective. So far, most research on conducting VR user studies has centred around tools for asynchronous setup, data collection and analysis, or (immersive) replays. Conversely, CReST is centred around supporting the researcher synchronously. We replicated three VR studies as example cases, applied CReST to them, and conducted an interview with one author of each case. We then performed a case study, and recruited 17 participants to take part in a user study where the researchers used CReST to observe participant interaction with virtual drawer and closet artefacts. We make CReST available for other researchers, as a tool to enable direct observation of participants in VR, and perform rapid, qualitative evaluations.

Employing virtual prototypes and immersive Virtual Reality (VR) in usability evaluation can save time and speed up the iteration process during the design process. However, it is still unclear whether we can use conventional usability evaluation methods in VR and obtain results comparable to performing the evaluation on a physical prototype. Hence, we conducted a user study with 24 participants, where we compared the results obtained by using the Think Aloud Protocol to inspect an everyday product and its virtual twin. Results show that more than 60% of the reported usability problems were shared by both the physical and virtual prototype, and the in-depth qualitative analysis further highlights the potential of immersive VR evaluations. We report on the lessons we learned for designing and implementing virtual prototypes in immersive VR evaluations.

Previous works show that virtual reality itself can be used as a medium in which to stage an experimental evaluation. However, it is still unclear whether conventional usability evaluation methods can directly be applied to virtual reality evaluations and whether they will lead to similar insights when compared to equivalent real-world lab studies. Therefore, we conducted a user study with nine participants, comparing Heuristic Evaluation (HE) for the evaluations of a novel smart artefact. We asked participants to evaluate the physical prototype and their virtual counterparts in the real-world and the virtual environment, respectively. Results show the HE have similar performance when evaluating artefacts usability in VR and real-world in terms of identified usability problems. The VR implementation has an impact on the immersive VR evaluation result.

TV experiences are often social, be it at-a-distance (through text) or in-person (through speech). Mixed Reality (MR) headsets offer new opportunities to enhance social communication during TV viewing by placing social artifacts (e.g. text) anywhere the viewer wishes, rather than being constrained to a smartphone or TV display. In this paper, we use VR as a test-bed to evaluate different text locations for MR TV specifically. We introduce the concepts of wall messages, below-screen messages, and egocentric messages in addition to state-of-the-art on-screen messages (i.e., subtitles) and controller messages (i.e., reading text messages on the mobile device) to convey messages to users during TV viewing experiences. Our results suggest that a) future MR systems that aim to improve viewers’ experience need to consider the integration of a communication channel that does not interfere with viewers’ primary task, that is watching TV, and b) independent of the location of text messages, users prefer to be in full control of them, especially when reading and responding to them. Our findings pave the way for further investigations towards social at-a-distance communication in Mixed Reality.

In 3D environments, objects can be difficult to select when they overlap, as this affects available target area and increases selection ambiguity. We introduce Outline Pursuits which extends a primary pointing modality for gaze-assisted selection of occluded objects. Candidate targets within a pointing cone are presented with an outline that is traversed by a moving stimulus. This affords completion of the selection by gaze attention to the intended target's outline motion, detected by matching the user's smooth pursuit eye movement. We demonstrate two techniques implemented based on the concept, one with a controller as the primary pointer, and one in which Outline Pursuits are combined with head pointing for hands-free selection. Compared with conventional raycasting, the techniques require less movement for selection as users do not need to reposition themselves for a better line of sight, and selection time and accuracy are less affected when targets become highly occluded.

The Cloud Columba CC2 is a web application for the design and simulation of microfluidic systems. I was a member of Columba Team, designed and implemented the Cloud Columba CC2.

The subway is a crucial mode of transportation connecting different areas of a city, but aging infrastructure has resulted in uneven distribution of passengers in different sections of incoming trains. To address this issue, an Intelligent Information Lighting System has been implemented at subway stations to help passengers predict train load levels and improve their experience. This project, carried out by an interdisciplinary team including students from Human-Computer Interaction, Landscape Architecture and Media Informatics, utilized an iterative design process to tackle a real-world urban challenge.

This short film was created as part of a media production course.