Sound of XR
Embodiment in virtual reality: an experiment on how visual and aural First-and Third-person modes affect embodiment and mindfulness

Abstract

Immersion as an umbrella experience and embodiment are regarded as primary advantages of virtual reality and its capability to lead to other reflective experiences. Beyond the basis of stereoscopic head-mounted displays, however, there is paucity of research on altered ways of experiencing virtual reality with regards to the possibilities afforded by this medium. To address this gap, we explore the effects of perspective shifts, both in terms of the visual and aural modalities, on embodiment and embodied mindfulness. Using a virtual reality application that administers a body-scan meditation using different visual- and aural perspectives, we investigate the effects these perspectives have on embodiment and embodied mindfulness. Our results indicate that a first-person visual perspective positively influenced a change in perceived body schema, but not in acceptance of virtual body ownership or control/agency of a virtual body, nor do they suggest an effect of audio-based perspective on these outcomes. Investigation into the moderating effects of mindfulness- and immersive tendencies on these factors suggest that participants with low immersive tendencies experienced a greater change in their body schema in the visual first-person condition compared to the third-person condition. A qualitative content analysis on participants’ experiences indicate that few participants were able to express their experience in terms of audio. Our results contribute to the body of work on altered self-representations for mindfulness and extends on this concept with the notion of listening perspective as well as more generally to the design of sound, perspective, and embodiment in virtual reality.

 

Role: Co-author, Supervisor

Type: Full Paper

Journal: Virtual Reality

Stats: h5-index:59, impact factor: 4.4

Date: 2025

Co-Authors: Mila Bujić, Laura Diana Cosio, Oğuz ‘Oz’ Buruk, Kristine Jørgensen and Juho Hamari


Listener perspective in games and virtual reality

Abstract

The term "perspective" in the context of media practices generally implies a focus on visual perspective and "point of view." However, an investigation into listener perspective reveals a wealth of considerations for and insights into the design and perception of a variety of media experiences. This chapter will provide an introduction into such topics with the focus on games and virtual reality, as these both require an understanding of the implications of medium-specific affordances for embodiment and listening perspective. Focusing on emerging technologies not only provides a glimpse of what is to come, but also a useful lens with which to re-evaluate the status quo concerning audio use in other media.

 

*the image for this paper was created by ChatGPT 4o

 

Role: supervisor

Type: Book Chapter

Conference: The Routledge Handbook of Sound Design

Date: 2024

Co-Authors: Kristine Jørgensen, Oğuz ‘Oz’ Buruk and Juho Hamari


awardIcon
SoundTrek: A Virtual Reality Audio Perspective-Switching Game

Abstract

Virtual reality technologies allow new ways of creating a sense of space not only through sight but also through sound. We present SoundTrek, a game about using listening perspective for navigating through environments in VR. In SoundTrek, the player needs to switch between listening to spatialized sounds from the first- and third-person perspectives to navigate through obstacles. SoundTrek also employs sound design strategies, such as spatialized soundtracks and near-field audio to aid spatial navigation in the absence of visual cues and to create unique aesthetic experiences through sound. We present new ways of utilizing sound in virtual reality game design that highlights design considerations for audio-focused gameplay and diversifies the opportunities for creating rich, interactive soundscapes.

 

Role: Supervisor

Award: Best Innovation Award

Type: Extended Abstract (Student Game Design Competition)

Conference: Companion Proceedings of the 2024 Annual Symposium on Computer-Human Interaction in Play

Stats: Acceptance rate: 53%, h5-index:31

Date: 2024

Co-Authors: Isak de Villiers Bosman, Ross Tordiffe, Tianyou 'Jackie' Huang, Kristine Jørgensen and Juho Hamari


Virtual Reality for Mindfulness: Aspects for Helping or Hindering Focus and Practice

Abstract

Virtual reality is increasingly being explored as a tool for facilitating mindfulness practices, such as through various guided meditation applications currently available. However, it is unclear how the technological affordances of virtual reality align with the goals of mindfulness and how such experiences can be designed to optimally exploit these affordances. We use a guided body-scan meditation in virtual reality and qualitative data from interviews to offer general insight into users’ experiences of virtual reality as a tool for mindfulness and relaxation more broadly. Our results highlight the potential to create audio-visual focus points for helping some participants concentrate, but also the pitfalls of this approach as a source of distraction for others. We point out avenues for further research, specifically into the interactive nature of VR, the benefits and drawbacks of novelty, alternative modes of self-representation, and the potential gap between VR-based practice and everyday mindfulness.

 

Role: Supervisor

Type: Extended Abstract (Poster)

Conference: Proceedings of the 27th International Academic Mindtrek Conference

Stats: h5-index:11

Date: 2024

Co-Authors: Mila Bujić, Laura Diana Cosio, Oğuz Oz Buruk, Kristine Jørgenson and Juho Hamari


The relationship of visual and aural perspective with decentering in virtual reality

Abstract

Several areas of application for virtual reality technologies have seen significant growth recently, including mindfulness practice. This paper investigates the relationship between the mindfulness-related construct of decentering, i.e., detaching oneself from automatic thinking and observing one's thoughts and emotions, and body representation within virtual reality. Using a within-subjects approach, we investigated whether a third-person perspective, both in terms of visual and aural representation, affects decentering in virtual reality. This was done using a virtual reality application that presented a guided body-scan meditation in virtual reality and utilised these different forms of perspective. Our results do not show a significant effect of either form of perspective on decentering in virtual reality.

 

Role: Supervisor

Type: Full Paper

Conference: Proceedings of the 4th African Human Computer Interaction Conference

Date: 2023

Co-Authors: Isak de Villiers Bosman, Mila Bujić, Laura Diana Cosio, Kristine Jørgensen and Juho Hamari


The effect of audio on the experience in virtual reality: a scoping review

Abstract

The use of virtual reality (VR) has seen significant recent growth and presents opportunities for use in many domain areas. The use of head-mounted displays (HMDs) also presents unique opportunities for the implementation of audio feedback congruent with head and body movements, thus matching intuitive expectations. However, the use of audio in VR is still undervalued and there is a lack of consistency within audio-centedd research in VR. To address this shortcoming and present an overview of this area of research, we conducted a scoping review (n = 121) focusing on the use of audio in HMD-based VR and its effects on user/player experience. Results show a lack of standardisation for common measures such as pleasantness and emphasize the context-specific ability of audio to influence a variety of affective, cognitive, and motivational measures, but are mixed for presence and generally lacking for social experiences and descriptive research.

 

Role: Co-Author, Supervisor

Type: Full Paper

Journal: Behaviour & Information Technology

Stats: h5-index:55, impact factor: 3.7

Date: 2023

Co-Authors: Isak de Villiers Bosman, Kristine Jørgensen and Juho Hamari