Depth cues and perceived audiovisual synchrony of biological motion

Due to their different propagation times, visual and auditory signals from external events arrive at the human sensory receptors with a disparate delay. This delay consistently varies with distance, but, despite such variability, most events are perceived as synchronic. There is, however, contradictory data and claims regarding the existence of compensatory mechanisms for distance in simultaneity judgments. Principal Findings: In this paper we have used familiar audiovisual events – a visual walker and footstep sounds – and manipulated the number of depth cues. In a simultaneity judgment task we presented a large range of stimulus onset asynchronies corresponding to distances of up to 35 meters. We found an effect of distance over the simultaneity estimates, with greater distances requiring larger stimulus onset asynchronies, and vision always leading. This effect was stronger when both visual and auditory cues were present but was interestingly not found when depth cues were impoverished. Significance: These findings reveal that there should be an internal mechanism to compensate for audiovisual delays, which critically depends on the depth information available.FEDERFundação para a Ciência e a Tecnologia (FCT

Object-Based Spatial Audio: Concept, Advantages, and Challenges

This book describes recent innovations in 3D media and technologies, with coverage of 3D media capturing, processing, encoding, and adaptation, networking aspects for 3D Media, and quality of user experience (QoE)

The Contribution of Head Movement to the Externalization and Internalization of Sounds

BACKGROUND: When stimuli are presented over headphones, they are typically perceived as internalized; i.e., they appear to emanate from inside the head. Sounds presented in the free-field tend to be externalized, i.e., perceived to be emanating from a source in the world. This phenomenon is frequently attributed to reverberation and to the spectral characteristics of the sounds: those sounds whose spectrum and reverberation matches that of free-field signals arriving at the ear canal tend to be more frequently externalized. Another factor, however, is that the virtual location of signals presented over headphones moves in perfect concert with any movements of the head, whereas the location of free-field signals moves in opposition to head movements. The effects of head movement have not been systematically disentangled from reverberation and/or spectral cues, so we measured the degree to which movements contribute to externalization. METHODOLOGY/PRINCIPAL FINDINGS: We performed two experiments: 1) Using motion tracking and free-field loudspeaker presentation, we presented signals that moved in their spatial location to match listeners’ head movements. 2) Using motion tracking and binaural room impulse responses, we presented filtered signals over headphones that appeared to remain static relative to the world. The results from experiment 1 showed that free-field signals from the front that move with the head are less likely to be externalized (23%) than those that remain fixed (63%). Experiment 2 showed that virtual signals whose position was fixed relative to the world are more likely to be externalized (65%) than those fixed relative to the head (20%), regardless of the fidelity of the individual impulse responses. CONCLUSIONS/SIGNIFICANCE: Head movements play a significant role in the externalization of sound sources. These findings imply tight integration between binaural cues and self motion cues and underscore the importance of self motion for spatial auditory perception

Auditory display design for exploration in mobile audio-augmented reality

In this paper, we compare four different auditory displays in a mobile audio-augmented reality environment (a sound garden). The auditory displays varied in the use of non-speech audio, Earcons, as auditory landmarks and 3D audio spatialization, and the goal was to test the user experience of discovery in a purely exploratory environment that included multiple simultaneous sound sources. We present quantitative and qualitative results from an initial user study conducted in the Municipal Gardens of Funchal, Madeira. Results show that spatial audio together with Earcons allowed users to explore multiple simultaneous sources and had the added benefit of increasing the level of immersion in the experience. In addition, spatial audio encouraged a more exploratory and playful response to the environment. An analysis of the participants’ logged data suggested that the level of immersion can be related to increased instances of stopping and scanning the environment, which can be quantified in terms of walking speed and head movement

Effects of Background Noise and Visual Training on 3D Audio

Spatial audio or 3D audio as an information channel is increasingly used in various domains. Compared to the multitude of synthetic visual systems and 3D representations, audio interfaces are underrepresented in modern aircraft cockpits. Civil commercial aircraft rarely use spatial audio as a supplementary directional information source. Although, different research approaches deal with the benefits of spatial audio. In 3D audio simulator trials, pilots express concern over distractions from background noise and possibly mandatory training requirements. To resolve this, the author developed and tested a 3D audio system to support pilots in future cockpits, called Spatial Pilot Audio Assistance (SPAACE). The experiment took place at the German Aerospace Center’s Apron and Tower Simulator. The developed system creates a three-dimensional audio environment based on normal non-spatial audio. The 27 participants heard the sound through an off-the-shelf aviation-like stereo headset. The main subject of investigation was to evaluate if air traffic control background noise affects spatial perception. The non-normally distributed location error with background noise ( Mdn=6.70∘ ) happened to be lower than the location error without air traffic control background noise ( Mdn=7.48∘ ). The evaluation the effect of visual feedback-based training was the second part of the experiment. In comparing the training session with the no-training session, the location error with training ( Mdn=6.51∘ ) is only moderately lower than the location error without training ( Mdn=7.96∘ ). The results show that humans can perceive the SPAACE audio with high precision, even with distracting background noise as in a busy cockpit environment. The effect of training was not as high as expected, primarily due to the already precise localization baseline without training