Intimate Space; Binaural Panoramas

Visual artist and researcher Charles Stankievech1 delves into the sonic walks created by artists Janet Cardiff and George Bures Miller2. This immersive experience reimagines the concept of sound walks, encouraging listeners to explore the city while a layered soundtrack unfolds in their ears through binaural recordings. In this way, the artists manage to blur the boundaries between the internal space of the listener and the constantly moving external physical environment.

© kham - CC BY-SA 2.0.

Walks Janet Cardiff.

I crafted my latest work titled "20",3 from a voice message on WhatsApp. I focused on developing an intimate perspective for the listener, creating an immersive and abstract atmosphere guided by the voice within a virtual space. Re-recording the voice message with a binaural microphone wasn't feasible, as the essence of the experience lay in the spontaneous nature of the message. In this context, subtle manipulation with binaural panning helped me construct spatiality, generate movement, and instill a proper sense of suspended time. Transcending the listener's physical space to an imagined abstract space within their mind.

"The Encounter," by Simon McBurney4, is a theatrical experience entirely carried out with a binaural microphone, achieving an individual engagement with the audience through headphone listening while simultaneously allowing for a collective sharing of the physical space in which the action unfolds. McBurney's solo performance skillfully blurs the boundaries between the tangible physical space of the theater, where the experience takes place, and the intimate dimension of the narrative.

© Christian Michelides

Simon McBurney in "The Encounter" in "The Encounter", World Premiere at the Edinburgh International Festival, August 2015 by Christian Michelides

In binaural audio, it's often asserted that this technique aims to emulate how humans hear. However, I firmly believe it is a technique designed to establish a more intimate connection with the listener. Perhaps it is the most effective method for constructing a narrative space within the listener's "mind."

When one listens with headphones, the spatial mediation of the immediate physical environment is eliminated, creating confined virtual acoustics within the listener's head. This detachable environment is characterized by a separation between the virtual auditory space and a disconnection from the immediate physical surroundings, giving rise to intermediate listening spaces (between the tangible and the virtual) that challenge traditional perspectives.

You are here, in your mind.

While binaural recordings achieved with specialized microphones are irreplaceable and provide a unique expansion of space that cannot be solely achieved through binaural panning plugins, the use of these plugins becomes an interesting ally in creating three-dimensional spatial perceptions from mono and stereo sounds.

A binaural panner is a vital tool in immersive audio production, strategically placing audio sources within a three-dimensional space to deliver an intimate and realistic auditory experience.

When working with three-dimensional (3D) sound, we choose a focal point as the origin, establishing a coordinate system centered on the listener's head, precisely at the midpoint between their ears. This configuration results in three planes intersecting at the origin, providing a framework for spatial measurements.

However, each plugin utilizes a different coordinate system or set of coordinate systems, depending on the user interface and the plugin's approach.

In the Cartesian coordinate system, coordinates are described in terms of horizontal distances (X-axis), vertical distances (Y-axis), and depth (Z-axis) in relation to the listening point. In the polar coordinate system, coordinates are described in terms of a radial distance and an angle formed by the radial vector with a chosen direction, typically the direction of the positive X-axis.

© Wikipedia

Cartesian and Polar Coordinates: Wikipedia

This underscores the importance of reviewing the specifications of binaural panning plugins to gain more information on how to position sound sources in the three-dimensional space of the chosen binaural panning plugin.5

The choice of a coordinate system significantly affects the workflow. This decision is closely tied to the artistic goal of the project.

Let's take, for example, my work ”Entre Marte y Júpiter,"6 a bilingual experimental radio piece commissioned by the Goethe Institute of Uruguay. Based on an interview with the Uruguayan astronomer Esmeralda Herminia Mallada Invernizzi, co-founder of the Association of Amateur Astronomers of Uruguay.

The challenge in this composition was to blend Mallada's voices in Spanish with the German translation. The choice of the coordinate system provided two different approaches to handling the voices and accompanying sounds for each voice.

For the segment where Esmeralda Mallada explains what an asteroid is and how it moves, I used the polar coordinate system. This allowed me to experiment with more abstract movements, providing a natural sense of how the voice rotated around the sound design. As polar coordinates enable the strategic placement of sound sources at different angles and distances, I managed to evoke a sense of closeness and intimacy based on the listener's perception rather than the strict positioning of the voice in the virtual environment.

I worked on the other segment using Cartesian coordinates. The translator's voice needed to have movement while occupying a specific place in space to contrast with Mallada's voice. In this context, Cartesian coordinates provided precise control over the location of the voice in three-dimensional space, allowing accurate positioning along the X, Y, and Z axes. This resulted in a natural and realistic spatial representation, creating a sense of intimacy from a different spatial perspective.

In both systems, achieving intimacy involves understanding the psychological impact that arises from distinct possibilities of sound positioning in a virtual three-dimensional environment. Factors such as distance, direction, and movement contribute to the emotional connection between the listener and the audio content. The ability to manipulate these elements, whether through the precision of Cartesian coordinates or the artistic flexibility of polar coordinates, allows for a personalized approach to creating auditory experiences that achieve the intimacy and connection we seek as producers.

More focused on space

Most binaural panners offer additional controls that play a crucial role in shaping the intimate qualities of sound, including occlusion control, reflection controls, focus control, amplitude control, damping control, and the option to use custom HRTF7 functions.

© DearVR

DearVR Pro and DearVR Micro audio panning plugins

Next, I will briefly explain some of these parameters and how they affect focus within binaural narratives.

Occlusion control: Occlusion control refers to the ability to simulate the blocking or obstruction of sound by objects in the virtual environment. This control is crucial for creating realistic spatial experiences where sounds can be partially or completely obscured by virtual obstacles, providing a sense of presence within the space.

Reflection controls: Reflection controls enable the manipulation of sound reflections within the virtual space. Manipulating reflection parameters, such as materials and surface angles, enhances the perception of the physicality of space.

Focus control: Focus control allows for the concentration of sound in specific directions or areas within the three-dimensional space. This is crucial for directing the listener's attention and enhancing emotional impact.

Width control: Width control influences the perceived amplitude of the sound field. Adjusting this control helps create a sense of width, proximity, and movement within the space.

Damping control: Damping control refers to the attenuation of sound over distance or through virtual materials. It is crucial for achieving realistic decay and adding depth to the space.

Custom HRTF: The integration of Head-Related Transfer Function (HRTF) capabilities within binaural panning plugins represents a fundamental advancement in improving accuracy in spatial audio reproduction. HRTF, a mechanism that considers individual differences in how sounds interact with the listener's head and ears, significantly refines the immersive audio experience. While many binaural panning plugins come equipped with a default HRTF, some plugins provide the flexibility to incorporate custom measurements. The accessibility of online repositories containing HRTF data from various binaural microphones allows producers to download and integrate these measurements into their chosen plugins.

Final notes

Binaural panning plugins provide the opportunity to creatively and realistically position sounds in space, significantly expanding the available narrative options. Binaural panning tools are straightforward and accessible resources for most independent producers.

From my point of view, they serve as a suitable starting point to grasp spatial functionalities and take the initial steps into the world of immersive audio.

Immersive audio experiences are undergoing a notable transformation, exemplified by the integration of Artificial Intelligence (AI) and biometric features into headphones. These innovations, combined with real-time adjustments in background noise cancellation, provide sound experiences that are personalized according to the user's preferences and needs.

This revolution in listening forms promises to reconfigure completely our perception of time and the various spaces we interact with.

While the study of binaural audio and spatial hearing spans more than a century, the last two or three decades have witnessed significant advances. In this context, experimental binaural narratives not only provide the opportunity to enrich intimate space but also to establish new connections with real physical spaces, becoming essential elements of ever-evolving immersive sonic narratives.

• 1

  Charles Stankievich Website https://www.stankievech.net/

• 2

  Jena Walk (Memory Field) 2006 https://cardiffmiller.com/walks/jena-walk-memory-field/

• 3

  more information about the work "20" https://solrezza.net/en/portfolio/20/

• 4

  The Encounter website https://www.complicite.org/work/the-encounter/

• 5

  It's crucial to refer to the plugin's manual to determine which scale each coordinate system graph utilizes. This ensures accurate interpretation and usage of the binaural panning plugin's spatial features.

• 6

  More information about the work “Entre Marte y Júpiter” https://solrezza.net/en/portfolio/entre-marte-y-jupiter/

• 7

  HRTF (Head Related Transfer Function), is a mathematical representation that describes the filtering process that occurs when sound travels from a sound source to the listener’s ears, taking into account the unique characteristics of the head and ears. Each individual has a different HRTF, as the shape and size of the head, ears, and torso affect how sound is perceived. HRTF captures these individualized characteristics and is used to simulate the spatial perception of sound in immersive audio systems.