Weirding Haptics: In-Situ Prototyping of Vibrotactile Feedback in Virtual Reality through Vocalization

How can we easily design vibrotactile feedback in virtual reality?

When we interact with objects in a virtual environment, vibrations are used to provide you with a sense of tactile feedback. While vibrations are limited, when designed correctly, they can be very convincing. For example, when catching a virtual ball, your controller might vibrate very shortly to persuade you that you are interacting with a real object.

Designing such convincing feedback mechanisms is extremely challenging. There are tools that allow you to design vibrations from low level parameters such as amplitude and frequency, or tools that convert audio into vibrations. However, these tools do not help you to design specifically for virtual object interactions, and moreover they do not assist with correctly aligning the timing of the interaction with the timing of the vibrations.

In this work, we built a framework to design vibrotactile feedback in virtual reality. While interacting with an object, a designer is able to vocalize the intended feedback. Our system converts their vocalizations to vibrations and allows the designer to manipulate this process through simple modfiers. This method makes it easier and quicker to prototype feedback for interactions in virtual environments.

Start designing yourself here: github



Effective haptic feedback in virtual reality (VR) is an essential element for creating convincing immersive experiences. To design such feedback, state-of-the-art VR setups provide APIs for programmatically generating controller vibration patterns. While tools for designing vibrotactile feedback keep evolving, they often require expert knowledge and rarely support direct manipulation methods for mapping feedback to user interactions within the VR environment. To address these challenges, we contribute a novel concept called Weirding Haptics, that supports fast-prototyping by leveraging the user’s voice to design such feedback while manipulating virtual objects in-situ. Through a pilot study (N = 9) focusing on how tactile experiences are vocalized during object manipulation, we identify spatio-temporal mappings and supporting features needed to produce intended vocalizations. To study our concept, we built a VR design tool informed by the results of the pilot study. This tool enables users to design tactile experiences using their voice while manipulating objects, provides a set of modifiers for fine-tuning the created experiences in VR, and allows to rapidly compare various experiences by feeling them. Results from a validation study (N = 8) show that novice hapticians can vocalize experiences and refine their designs with the fine-tuning modifiers to match their intentions. We conclude our work by discussing uncovered design implications for direct manipulation and vocalization of vibrotactile feedback in immersive virtual environments.

Weirding Haptics: In-Situ Prototyping of Vibrotactile Feedback in Virtual Reality through Vocalization
Donald Degraen*, Bruno Fruchard*, Frederik Smolders, Emmanouil Potetsianakis, Seref Güngör, Antonio Krüger, Jürgen Steimle

Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology (UIST '21)
PDF: Pre-print

Contact Person

Dr. Donald Degraen