Sound Spatialization and Synesthesia

Understanding Sound Spatialization

Sound spatialization, the art of manipulating sound in a three-dimensional space, has gained increasing relevance in contemporary music, art, and technology. Affected creatively by synesthesia with regards to architectural spaces and sound, the process of creating and experiencing spatial sound for me takes on a uniquely immersive dimension. An architect designs works that occupy spaces; I create sound sculptures that fit in the flow of time and perception. For me, the environment architecturally shapes the pieces and how they will be heard. When this synesthetic experience is combined with a strong interest in quantum physics, sound spatialization transforms from a sensory manipulation into a profound exploration of the nature of reality.

Sound spatialization is the technique of positioning and moving sound in a physical or virtual space to give listeners the sensation that audio is coming from different directions and distances. While basic stereo sound allows us to perceive sound from left and right, spatial audio expands this into a 3D experience, using techniques such as ambisonics, binaural audio, and Dolby Atmos. These tools allow sound to be perceived as above, below, in front, behind, or around the listener.

© Wikipedia

In spatial audio, sound can move dynamically through space, giving a sense of immersion that closely mimics how we hear sounds in the natural world. Whether in virtual reality, immersive art installations, or music performances, sound spatialization is used to create environments that transcend traditional auditory experiences, enabling listeners to perceive and engage with sound as though it were a physical object.

SpatGRIS is a spatialization software developed by the Groupe de Recherche en Immersion Spatiale (GRIS) at the University of Montreal, specifically designed for composers, sound designers, and artists working in electroacoustic and immersive sound environments. It offers sophisticated tools for controlling and manipulating the spatial positioning of audio sources in multichannel systems, ranging from basic stereo setups to complex 3D configurations like ambisonics or speaker arrays.

© SpatGRIS

1. Spatialization Control

SpatGRIS allows users to place sound sources in a virtual space, controlling their position and movement. You can assign specific spatial trajectories to sound objects, giving the impression that they move dynamically within a room or 3D space.

2. Multichannel Flexibility

The software supports configurations from stereo to large multichannel systems. This is particularly useful for installations or performances with multiple speakers positioned around an audience, creating an immersive sound environment.

3. Compatibility with DAWs

SpatGRIS is compatible with various digital audio workstations (DAWs), allowing users to integrate spatialization directly into their compositions or sound design projects. This means that you can use it within environments like Reaper or Max/MSP.

4. Ease of Use

The user-friendly interface of SpatGRIS allows for intuitive interaction with spatialized sound objects, offering a visual representation of their positions in space. This makes composing and experimenting with spatial sound easier even if you’re new to spatialization techniques.

5. Advanced Sound Design Capabilities

It can be used for highly intricate sound design work, enabling the design of spatialized soundscapes responsive to static and dynamic movement. The software is often employed in electroacoustic music, sound art, and VR/AR audio applications.

6. Open Source and Free

SpatGRIS is open-source software, making it freely available for use by artists and composers worldwide. This accessibility has fostered a community of users who contribute to the software's development and share knowledge on its use.

SpatGRIS is highly regarded for its flexibility and precision in handling spatial audio, especially within creative and experimental contexts. Whether you're working on multichannel compositions, immersive installations, or VR/AR projects, SpatGRIS provides powerful tools for spatial sound manipulation.

Synesthesia and Sound Architecture

Synesthesia is a condition in which one sensory experience triggers another. In sound architecture synesthesia, sounds are not merely auditory sensations but are perceived as architectural forms or spaces. For instance, someone with this form of synesthesia might hear a low rumbling sound as a solid, weighty structure, while higher-pitched sounds could evoke sharp, delicate beams or arches. In my particular case, when I walk into a performance or installation space, I immediately experience what I can and cannot do in said space, for instance where I can position speakers, what frequencies I can or cannot use and most importantly, where I may have the audience seated in order to provide a quality experience for everyone. The architectural nature of these sound experiences means that every auditory element can trigger a visual-spatial image, often blending multiple sensory modalities into a single, unified perception.

Personally, being affected by sound architecture synesthesia, spatial audio is not just about placing sounds around the listener but about constructing entire sonic landscapes that have both physical presence and emotional resonance. In this sense, sound becomes a building material for creating environments that are as much felt as they are heard.

Key Aspects of Sound Architecture Synesthesia

1. Synesthetic Perception: In this form of synesthesia, sounds (such as music or environmental noise) trigger the perception of architectural structures, spaces, or spatial arrangements. For example, a person may hear a particular sound and "see" a specific building form, room, or geometric shape in their mind's eye, or feel a sense of space expanding or contracting. 
2. Spatial Sound Associations: Sounds can evoke sensations of size, depth, or the spatial orientation of an imagined structure. For instance, certain frequencies may be associated with high ceilings, wide spaces, or long corridors, while other sounds might evoke more enclosed or intimate environments. 
3. Musical and Architectural Interplay: This phenomenon is particularly intriguing in the context of music, sound design, and architecture. Architects and sound designers can create experiences where spatial design is deeply integrated with soundscapes, reinforcing the sensory overlap experienced by individuals with sound architecture synesthesia. 
4. Individual Variability: Like other forms of synesthesia, sound architecture synesthesia can vary widely between individuals. While some may experience highly detailed and vivid architectural forms triggered by sound, others may experience more abstract or subtle spatial sensations.
5. When sound is treated as an architectural element, it creates a heightened sense of physicality in space. Synesthetic individuals may perceive music or soundscapes as physical structures that they can mentally walk through, touch, or inhabit. This unique perspective offers new creative opportunities for spatial audio, allowing for the design of immersive sound environments that feel like entering a new world.

Quantum Physics and Sound Spatialization

Quantum physics describes the nature of the universe as being far different from that which we see. Concepts like wave-particle duality, superposition, and entanglement challenge traditional ideas of space, time, and matter. I find intriguing parallels between sound spatialization and quantum mechanics. For instance, both fields deal with waves – sound waves in audio and probability waves in quantum mechanics. In both cases, waves can interfere, amplify, or cancel each other out, leading to complex behaviours that are difficult to predict.

I have a deep interest in quantum physics and I draw parallels between the behavior of sound in space and the behavior of particles at a quantum level. Just as particles can exist in multiple states or locations simultaneously, sounds can also be designed to shift and morph in a spatial audio field, creating an experience where the listener feels as though they are surrounded by constantly evolving sonic elements.

Synesthetic Perception of Quantum Phenomena through Sound

For someone with sound architecture synesthesia and an interest in quantum physics, sound spatialization can offer a way to experience abstract quantum concepts on a sensory level. Through spatial audio, quantum mechanics can be translated into soundscapes that make the otherwise invisible phenomena of the quantum world more tangible and immersive.

Superposition in Sound: Quantum superposition refers to a particle existing in multiple states at once until it is observed. In sound spatialization, I have represented superposition through layered, overlapping sounds that exist in multiple locations simultaneously. As the listener moves through the space, sounds could shift and collapse into a single, coherent sound, mimicking the wave function collapse of a quantum superposition when observed. In the case of a spatial performance in a dome, sounds move through the space, shifts and collapses while the listener is immobile.

Quantum Entanglement: Quantum entanglement occurs when two particles are linked in such a way that the state of one affects the other, no matter the distance between them. In sound spatialization, I have explored this concept by creating entangled sound sources – when one sound changes, another sound elsewhere in the space reacts. This interaction could create a dynamic relationship between sounds, reinforcing the idea of quantum interconnectedness.

Wave-Particle Duality: In quantum physics, particles can behave both as particles and waves, depending on how they are measured. This duality can be represented in spatial audio by designing sound that oscillates between behaving like a defined object (localized and focused) and behaving like a diffuse wave (spreading out and blending with the environment). For a synesthetic listener, this could evoke architectural structures that appear solid in one moment and dissolve into fluid, wave-like forms in the next.

Artistic & Scientific Implications - Quantum-Inspired Sound Spatialization

By applying quantum principles to sound spatialization, artists and sound designers can explore entirely new ways of creating and experiencing sound. For those with synesthesia, these experiments can take on an even deeper significance, as their unique sensory experiences allow them to intuitively grasp abstract scientific concepts and translate them into artistic works.

At the intersection of sound, architecture, and quantum physics, new modes of expression can emerge. Sound installations, performances, and virtual reality environments could be designed to explore the fundamental nature of reality, using sound to bridge the gap between the tangible and the abstract.

Quantum physics is often described as counterintuitive, defying the logic of our everyday experiences. Sound spatialization, particularly for those with sound architecture synesthesia, can serve as a medium through which these complex ideas become more accessible. By turning quantum phenomena into immersive sound experiences, synesthetic individuals can offer new ways of understanding and interpreting the quantum world.

Conclusion

For someone with sound architecture synesthesia and a deep interest in quantum physics, sound spatialization offers a powerful tool for exploring the intersections of art and science. By translating quantum concepts into spatialized sound, artists can create immersive environments that offer new ways of understanding both the physical and metaphysical world. Through sound, the abstract theories of quantum mechanics become tangible, sensory experiences that can be felt and perceived in entirely new ways.

Sound spatialization is not just a technical achievement, but a medium through which the mysteries of the universe can be explored, understood, and re-imagined. For those with synesthesia, the experience of sound is more than auditory – it is a journey through space, time, and the very fabric of reality itself.