Sound of XR
Listener perspective in games and virtual reality

Abstract

The term "perspective" in the context of media practices generally implies a focus on visual perspective and ‘point of view.’ However, an investigation into listener perspective reveals a wealth of considerations for and insights into the design and perception of a variety of media experiences. This chapter will provide an introduction into such topics with the focus on games and virtual reality, as these both require an understanding of the implications of medium-specific affordances for embodiment and listening perspective. Focusing on emerging technologies not only provides a glimpse of what is to come, but also a useful lens with which to re-evaluate the status quo concerning audio use in other media.

 

Role: supervisor

Type: Book Chapter

Conference: The Routledge Handbook of Sound Design

Date: 2024

Co-Authors: Kristine Jørgensen, Oğuz 'Oz' Buruk and Juho Hamari


PDF Cite
awardIcon
SoundTrek: A Virtual Reality Audio Perspective-Switching Game

Abstract

Virtual reality technologies allow new ways of creating a sense of space not only through sight but also through sound. We present SoundTrek, a game about using listening perspective for navigating through environments in VR. In SoundTrek, the player needs to switch between listening to spatialized sounds from the first- and third-person perspectives to navigate through obstacles. SoundTrek also employs sound design strategies, such as spatialized soundtracks and near-field audio to aid spatial navigation in the absence of visual cues and to create unique aesthetic experiences through sound. We present new ways of utilizing sound in virtual reality game design that highlights design considerations for audio-focused gameplay and diversifies the opportunities for creating rich, interactive soundscapes.

 

Role: Supervisor

Award: Best Innovation Award

Type: Extended Abstract (Student Game Design Competition)

Conference: Companion Proceedings of the 2024 Annual Symposium on Computer-Human Interaction in Play

Stats: Acceptance rate: 53%

Date: 2024

Co-Authors: Isak de Villiers Bosman, Ross Tordiffe, Tianyou 'Jackie' Huang, Kristine Jørgensen and Juho Hamari


PDF Cite ACM
Shroom Cards: Playful Exploration of Human Positionalities for More-than-Human Design

Abstract

This paper introduces Shroom Cards, a novel tool designed to engage with More-than-Human (MtH) design through playful exploration and reflection. Leveraging design cards and structured activities, Shroom Cards provides concrete starting points to practically engage with MtH approaches in design, such as embodying the roles and purposes of human and non-human entities. In this paper, we detail the design process, which includes Reflective Design Studio exercises to create positionality cards and incorporating playful strategies such as exploration, role-playing, and competition in the structured ideation and reflection activities. Initial testing of the Shroom Cards in an HCI design course students (N=23) shows that the cards and ideation activity effectively stimulate creative processes and diverse perspectives. However, challenges remain in fully adopting non-human viewpoints, suggesting that enhanced role-playing elements are needed for deeper engagement with non-human perspectives.

 

Role: Co-Author, Concept Co-Creator

Type: Extended Abstract (Work in Progress)

Conference: Companion Proceedings of the 2024 Annual Symposium on Computer-Human Interaction in Play

Stats: Acceptance rate: 44%

Date: 2024

Co-Authors: Çağlar Genç, Ferran Altarriba Bertran, Sangwon Jung, Velvet Spors and Juho Hamari


PDF Cite ACM
Virtual Reality for Mindfulness: Aspects for Helping or Hindering Focus and Practice

Abstract

Virtual reality is increasingly being explored as a tool for facilitating mindfulness practices, such as through various guided meditation applications currently available. However, it is unclear how the technological affordances of virtual reality align with the goals of mindfulness and how such experiences can be designed to optimally exploit these affordances. We use a guided body-scan meditation in virtual reality and qualitative data from interviews to offer general insight into users’ experiences of virtual reality as a tool for mindfulness and relaxation more broadly. Our results highlight the potential to create audio-visual focus points for helping some participants concentrate, but also the pitfalls of this approach as a source of distraction for others. We point out avenues for further research, specifically into the interactive nature of VR, the benefits and drawbacks of novelty, alternative modes of self-representation, and the potential gap between VR-based practice and everyday mindfulness.

 

Role: Supervisor

Type: Extended Abstract (Poster)

Conference: Proceedings of the 27th International Academic Mindtrek Conference

Date: 2024

Co-Authors: Mila Bujić, Laura Diana Cosio, Oğuz Oz Buruk, Kristine Jørgenson and Juho Hamari


PDF Cite ACM
The relationship of visual and aural perspective with decentering in virtual reality

Abstract

Several areas of application for virtual reality technologies have seen significant growth recently, including mindfulness practice. This paper investigates the relationship between the mindfulness-related construct of decentering, i.e., detaching oneself from automatic thinking and observing one's thoughts and emotions, and body representation within virtual reality. Using a within-subjects approach, we investigated whether a third-person perspective, both in terms of visual and aural representation, affects decentering in virtual reality. This was done using a virtual reality application that presented a guided body-scan meditation in virtual reality and utilised these different forms of perspective. Our results do not show a significant effect of either form of perspective on decentering in virtual reality.

 

Role: Supervisor

Type: Full Paper

Conference: Proceedings of the 4th African Human Computer Interaction Conference

Date: 2023

Co-Authors: Isak de Villiers Bosman, Mila Bujić, Laura Diana Cosio, Kristine Jørgensen and Juho Hamari


PDF Cite ACM
The effect of audio on the experience in virtual reality: a scoping review

Abstract

The use of virtual reality (VR) has seen significant recent growth and presents opportunities for use in many domain areas. The use of head-mounted displays (HMDs) also presents unique opportunities for the implementation of audio feedback congruent with head and body movements, thus matching intuitive expectations. However, the use of audio in VR is still undervalued and there is a lack of consistency within audio-centedd research in VR. To address this shortcoming and present an overview of this area of research, we conducted a scoping review (n = 121) focusing on the use of audio in HMD-based VR and its effects on user/player experience. Results show a lack of standardisation for common measures such as pleasantness and emphasize the context-specific ability of audio to influence a variety of affective, cognitive, and motivational measures, but are mixed for presence and generally lacking for social experiences and descriptive research.

 

Role: Co-Author, Supervisor

Type: Full Paper

Journal: Behaviour & Information Technology

Stats: h5-index:55, impact factor: 3.7

Date: 2023

Co-Authors: Isak de Villiers Bosman, Kristine Jørgensen and Juho Hamari


PDF Cite Taylor & Francis