3D audio on the web

3D audio technology can now be considered an audio standard on the web. A website, accessed in an internet browser on a reasonably up-to-date device, is all that is needed to develop and experience 3D audio.

© cables.gl

A special feature of this implementation is the possibility of automatically linking the audio position with the camera position within the cables editor. This means that the perceived audio sources in 3D space automatically change their position relative to the movement and orientation of the camera: the viewing axis in the virtual 3D world corresponds to the orientation of a listener. This principle allows developers and artists to experiment with 3D audio in a way that was previously reserved for complex and specialized audio software, opening up new and accessible creative possibilities for interactive and immersive web applications.

Stereo and 3D Audio

Traditional stereo audio only gives the illusion of the position of a sound source on the horizontal plane between the left and right sides of the listening field. 3D audio allows listeners to perceive sounds from all directions and distances. It can give the impression that they are physically located within a virtual sound sphere, in the midst of all sound sources. This form of audio processing not only contributes to a more intense and realistic user experience, but also opens up new possibilities for developers and creatives to design interactive web content with sound.

PannerNode and AudioListener

A 3D audio setup on the web needs a defined listening position, the so-called AudioListener (a standard object of the WebAudio API). 

As described above, this position in cables is linked to the camera position by default (via the so-called ViewMatrix). Any positioning is of course also possible. Each sound source (e.g. a specific audio file for a sound effect) is assigned its own PannerNode. 

The WebAudio PannerNode is a powerful tool that enables developers to integrate 3D audio into web applications. Put simply, the PannerNode is responsible for determining the virtual position of an audio source in 3D space and how the sound emanating from it is transported to the listener. Sounds appear as if they are coming from certain directions and distances - in front of, behind, next to, above, or below the listener.

The concept of the interplay of a listening position (the AudioListner) and several sound sources positioned relative to it by a respective AudioPannerNode is referred to as object-based 3D audio. Several important concepts come into play here:

Positioning: The position of the audio source in virtual 3D space can be static or moving. The X, Y, and Z coordinates of the virtual 3D space are taken into account, with X representing the horizontal axis (left/right), Y the vertical axis (up/down), and Z the depth (front/back).

Orientation: This defines the orientation of the audio source in the room. This is important to determine the direction in which the sound is emitted. Sound can radiate evenly from one point in all directions or be directed precisely - comparable to a beam of light.

Distance effect: The PannerNode models how the sound changes with increasing distance between the source and the listener. This includes both attenuation (becoming quieter with increasing distance) and changes in timbre (decrease in high frequencies with increasing distance).

Directional effect: This refers to how differently the sound is perceived depending on its direction to the listening position, including the simulation of real-world effects such as the shielding of the sound by the human head (binaural audio).

Listener position: Just like any audio source, the listening position must also be defined in the coordinate system of the virtual 3D space. This is the task of the so-called AudioListener object of the WebAudio API.

3D Audio in cables.gl

cables is a node-based online tool that developers can use to create a variety of web experiences by virtually wiring so-called operators (OPs). No code needs to be written, which makes cables particularly accessible for beginners.

If new functions are required, new OPs can also be programmed in JavaScript, which can be made available to all users. In this way, I have extended the set of audio OPs already available in cables by two OPs so that the established 3D audio functions of the WebAudio API can be used there.

If you want to use 3D audio in cables, first create an account for cables.gl and then join the "Team Spatial Audio": 

https://cables.gl/team/spatialaudio

From now on, the OPs "AudioPanner3DAudioObject" (for positioning sound objects) and "AudioPanner3DListener" (for defining the listening position) can also be used in your own cables patches. Various possible uses are illustrated in the following two sample patches. In both examples, the listening position can be linked to various of the cameras available as standard in the cables editor.

The first minimal example can be used as a copy-paste template and explains the use of the two central 3D audio OPs. The helper sub-patch for visualizing the audio object and its sound properties at its current position is also helpful.

The second patch illustrates moving sound sources with different directions of sound propagation. If you are interested in the JavaScript code, open the patch in the cables editor (https://cables.gl/edit/tr3PMm), select the corresponding OP, and then click on the pencil icon to view the JavaScript code. Each OP can also be cloned so that the code can also be modified and customized. (There are further tutorials on editing OPs on cables.gl.)

In the video linked below from the cables meetup on the subject of 3D audio, I present the Spatial Audio OPs and explain all the parameters and their functions in detail. It is a helpful introduction to the comprehensive possibilities of 3D audio OPs, but also addresses some of the pitfalls that 3D audio on the web can currently still entail.

In the finally linked interactive web audio player of my AI audio project an soot, I used the possibilities of 3D audio to design an interactive audio player user interface: The position of the virtual sound source can be influenced by moving and rotating the animation, which in turn influences the sonic intensity and orientation of the tracks. 

Have fun exploring 3D audio on the web and patching in cables.