Sounds of the Future

The reasons why people deal with sounds and their technical processing are extraordinarily varied. Often, it has to do with the implementation of content – e.g., on the internet or in broadcasting – there is the development of hardware and software for practical applications, and, not least, there are scientific and artistic motivations. These, in turn, can come from very different musical areas and the visual arts. Finally, many people are interested in the audio sector privately or out of passion. 

© Daniel Mayer

Wavefolding, originally implemented by Don Buchla with analog means (Buchla 259). Code example in SuperCollider (class SmoothFoldS, miSCellaneous_lib).

Technological developments combined with the spread of information on the internet bring these – previously often completely separate – groups together, like on this platform, a splendid opportunity to exchange ideas! However, there is also a need to clarify what motivates one's work – more clearly than in the usual working environment where people with similar interests are active. I could, for example, immediately present some unusual methods of sound synthesis – and would also like to do so on this platform in the future – but, on its own, this information would be all too isolated, and the motivations behind it hardly traceable for many readers. Hence: context first.  

I am a composer, love researching and tinkering with sounds, and teach related subjects – Sound Synthesis, Electroacoustic Composition, etc. – at the Institute for Electronic Music and Acoustics at the University of Music and Performing Arts Graz. The field of experimental sound research, preferably based on open-source software, is a humongous cosmos, and it is important to me to share my fascination with it. This focus was not clear from the beginning of my artistic activity. It has emerged via detours. In the following, I would like to go into general and personal motivations before regarding the more technical aspects of the relationship between software and sound. 

1) Transgressions

An essential characteristic of artistic impulses – and this may be a Western-influenced perspective – is the moment of transgression of the culturally familiar and established. That became particularly clear with the emergence of an autonomous art scene in Europe during the Enlightenment, and the artistic avant-garde of the 20th century took this principle to the extreme. Style – characterized by Theodor W. Adorno as social violence – has been established only temporarily throughout history in each case. But even outside the circles of a demarcated or demarcating avant-garde – it is questionable whether the term is even still appropriate given the fragmentation of the arts – the transgression of norms is a central motif. Every kind of experimental pop music or electronica wants to distinguish itself from the mainstream in this way. Joanna Demers describes this accurately in her book Listening Through the Noise – The Aesthetics of Experimental Electronic Music1. Of course, transgression can mean many things, especially in pop and electronica, including differentiation through fashion, appearance, and media staging.

If we look at the narrower field of musical structures, then – again, especially in pop and electronica – the aspect of the sonic is the very place where differentiation can manifest itself. That becomes obvious in countless examples of artists and groups with specific and – at least at the time of their first appearance – novel sounds. In the audio-related areas of contemporary art (new music, electroacoustic music, sound art, etc.), however, the situation is far more complicated: here, it is often not primarily about the sonic, but also or especially about musical form, duration, reference, concept and much more. Innovations and transgressions are possible in all of these fields, such as the experiments with extreme durations – short or long – by Anton Webern, Erik Satie, Morton Feldman, and Eliane Radigue. With the development of computer-based sound processing and synthesis, however, new opportunities for transgression are opening up in the actual medium of sound.

2) Obstacles

What prevents artistic impulses and innovations? Apart from obviously repressive situations, indirect constraints reveal – or rather hide – themselves by prevailing structures in education, the art world, production methods, and market mechanisms. Peter Ablinger: "I realized late in life that music's dependence on institutions: orchestra, ensemble casting, academy, training, instrumental tradition, concert hall, and musicology is not only responsible for the overwhelming historicity of the music business but also becomes a corrupting trap for the latest music, or at least creates a prejudiced climate towards all musical activity that seeks to circumvent these institutions even partially."2 This leaves nothing open in terms of clarity; the "classical" post-war avant-garde of new music, in particular, has – in comparison with that of other genres – questioned its preconditions to a much lesser extent, as Harry Lehmann points out in Die digitale Revolution der Musik3. The powerful instrumental tradition and – cum grano salis – the end of the technical development of acoustic instruments more than 100 years ago have led to a standstill in the field of pure sound following research. After the exploitation of extended playing techniques after the Second World War, it became apparent that further developments and transgressions would primarily come through electronics and computer technology – even if partly connected to classical instruments or other human interaction. 

The diffusion of digital technology is disruptive – looking at the entire history of music – but from our contemporary perspective, it is still relatively slow, as the inertia of all institutions, which is natural and necessary regarding tradition, is a fact of life. There are also indirect constraints in experimental pop music and electronica, e.g., the marketing mechanisms of labels and the predetermined formats of distribution: in earlier years, records and CDs, today the dominant internet platforms. However, as one difference to the traditional music business, digital production is widespread here anyway, paving the way for sonic experimentation with digital means. 

Another half-hidden category of obstacles exists within the digital production techniques themselves: convenient tools with graphical user interfaces (DAWs, plug-ins, online audio processing, etc.), commercially or freely available, make direct work on sound simple and effective in the sense of an end-means relationship: the application of known effects, spatialization, etc., can be implemented quickly and practically. Of course, there is nothing wrong with this in principle. The problem – regarding artistically ambitious handling – begins if one doesn't recognize inevitably existing limitations or underestimates their impact. 

For example, many DAWs one-sidedly lean towards the production of beat-based music. Through this lens, a different temporal organization of the musical material is an exceptional case. Regarding the space of all musical possibilities, it should be contrariwise. As a result, sequencing methods often appear as simple and repetitive patterns with a few extensions; the entire cosmos of structure-generating, which primarily coding can open, is largely excluded. 

The spectrum of available plug-ins orients towards synthesis and processing methods that have already proven themselves, their structure and parameter selection are often predetermined, and the linking of components is standardized (e.g., through LFOs). 

Spatialization by plug-ins is also a temptation, as standard methods – such as movements of point sources – are nearby to use and preclude the need to think about different concepts – e.g., integrating decorrelation into synthesis or processing. 

In the rapidly developing AI applications in the audio sector, which harbor enormous artistic potential, the same patterns can be recognized – perhaps even to a greater extent, as the complexity of the algorithms used suggests the design of interfaces with a black box character. 

In all these examples, the aesthetics of the user interfaces make it difficult, if not impossible, to think about their limitations. The pleasurable play with colorful buttons and controls and the immediate satisfaction of the sonic result are stimulus-response schemes corresponding to the media consumption of our time. Daniel Kahneman's distinction between fast and slow thinking – System 1 and System 2 – can be observed here exemplarily4. Attractive interfaces serve the need for automatic and emotional reactions. It seems clear that artistic work can, and perhaps even must, include these as welcome triggers, but conscious aesthetic decisions must finally go beyond this. 

In no way do I want to reject convenient audio tools. After all, I use them myself, but only for limited tasks (equalizing, editing, etc.). However, from observing and the description of other work processes, e.g., by students and in online forums, I deduce that the limitations of given interfaces – and conversely, the possibilities of individual development with open-source – are grossly underestimated. 

3) Paths

There is a fantastic opportunity to overcome the sociological, psychological, and technical limitations, which I have described above. I am mainly referring to the programming environments SuperCollider, Pure Data, ChucK, Csound, and others, which are currently in use and are at least two decades old. As I prefer text-based programming and particularly appreciate the characteristics of this language, SuperCollider (SC) is my first choice. 

While such environments, like more common tools, enable rapid reaction to sonic results, they also integrate the design of algorithms to a much greater extent, both in synthesis and structure generation. That way of thinking – System 2! – is strategically oriented, laborious, and tedious in parts of the work process but, if successful, enormously rewarding. The tools developed are reusable and can elegantly individualize the artistic work process. It would be beyond the scope of this article to go into the topic in detail. In this essay5,  I have outlined the model of an iterative way of working with algorithms that integrates the artist's reference to history and culture.  

Since thinking in System 1 always exerts a more significant attraction than thinking in System 2, the classical programming of sound and structures will presumably always remain an option for a minority; moreover, it also represents a didactic challenge. Conversely, it is an immense opportunity for those involved because a huge wasteland invites exploration. 

Since the 1970s, an oppositional culture of sound synthesis has developed – apart from research into commercially exploitable synthesis methods such as FM – especially under the catchword non-standard and associated with the names Gottfried Michael Koenig, Herbert Brün, Steven R. Holtzman, and Iannis Xenakis6. The term opposition here means both an orientation against commercial exploitability and against the paradigm of synthesis categories of spectral and physical models strongly oriented towards physics and the analysis of acoustic instruments. In his well-known taxonomy, Julius O. Smith distinguishes between the categories of Processed Recording, Spectral Model, Physical Model, and Abstract Algorithm7. He favors spectral and physical models as the respective core of the two poles of reception and genesis. Non-standard in the historical sense would fall into the Abstract Algorithm category. In recent years, an increasing interest in alternative methods of synthesis can be observed, for example, in individual initiatives (Nick Collins, Luc Döbereiner, Sergio Luque, David Pirrò), symposia and research projects. Let's note here that alternative methods, in a more general sense, transcend the original meaning of non-standard. Physical models (of a non-acoustic nature) can also act as synthesis engines, which illustrates the difficulties of an all-encompassing taxonomy of synthesis methods – although Smith's categorization still appears to be broadly viable. 

Individual work on alternative methods often uses open-source software, and my research in this area closely relates to the development of miSCellaneous_lib, an extension of SC. I have taken the following examples from the tutorials of miSCellaneous_lib and a related artistic research project. The artistic process in my more extensive compositional projects is not fundamentally different.

Granular synthesis, the accumulation of sound particles from any source, was my preferred method for a long time. The range of sonic results is so broad that one could often assume different synthesis methods. Curtis Roads8 provides an excellent overview. The sequencing of individual events by using SC Patterns is a particularly elaborate option of this language9. It also opens up differentiated possibilities for determining the properties of individual grains (sound particles). For this reason, I have integrated several tutorials on granular synthesis into miSCellaneous_lib. The following examples grew from the processing of a five-second recording of a sound recorded in my kitchen.

As a related experiment, I produced a series of electroacoustic miniatures from the material to document the working process. miSCellaneous_lib contains the source code, and everyone can, in principle, reproduce the individual pieces (however, random components in the code de facto exclude a literal reproduction). 

In recent years, I have increasingly explored alternative methods outside of granular techniques and applied them in my latest acousmatic works in the Matters series. Sometimes, also unusual combinations of simple and familiar techniques challenge me compositionally. The Pattern class used in kitchen studies on a granular time level for effect processing10 is used in Matters 9 on a larger time scale.

4) Prospects

The option of researching sounds with open-source software is no longer new. The "gold rush" of computer music in the 1990s, when real-time sound processing became practicable for many musicians outside of large institutions for the first time and extensive open-source projects with global communities developed, also increased interest in alternative ways of synthesis and processing. However, apart from SuperCollider, Pure Data, ChucK, Csound, and a few others, hardly any new major open-source projects have been able to spread in the last 20 years. That may be a surprise when considering the uninterrupted development in the hardware sector. Conversely, however, the permeability between previously unconnected areas is increasing. An example of this is the development of the VSTPlugin-Library by Christof Ressi, which enables the integration of VST instruments and effects in Pure Data and SuperCollider. That connects previously separate universes – a breakthrough that caused quite a stir in the open-source communities.   

The option of using such possibilities artistically is naturally limited to those who spend time and effort learning programming and mastering the specific features of the respective software. I know from my teaching experience that these hurdles can be enormous. However, as with many things in life, strategic thinking – System 2: beyond the moment, with a view to future value – holds the most outstanding opportunities and freedom. 

The effects of the rapid developments in AI are still hard to predict, both in terms of synthesis and processing, as well as regarding the development of (audio) software. It is imaginable that working in programming languages, as was still common for my generation – growing up in the 1970s and 80s – will be replaced by a new paradigm in which processes previously regarded as high-level are delegated to AI as the new low-level. That could also take audio programming to a new tier, opening it up to people without traditional programming skills and requiring further reflection on the impact on artistic processes. Even then, the most fruitful opportunities for individual iteration might lie in the possibility of flexibly adapting the processes. In order not to be helplessly at the mercy of the AI design specifications, at least a basic knowledge of the processes is required. Their genuinely artistic application – individual shaping and transgression of the given – should go beyond merely spontaneous use.

• 1

  Demers J. 2010, Listening Through the Noise, Oxford University Press.

• 2

  Ablinger P. 2009, Due Pratiche

• 3

  Lehmann H. 2012, Die digitale Revolution der Musik. Eine Musikphilosophie, Schott.

• 4

  Kahnemann D. 2011, Thinking, Fast and Slow. Farrar, Straus and Giroux.

• 5

  ALMAT 2020, Symposium on Algorithmic Agency in Artistic Practice.

• 6

  Computer Music Journal, 35(3), 28-39.

• 7

  Proceedings of the 1991 International Computer Music Conference. San Francisco. International Computer Music Association.

• 8

  Roads, C. 2001, Microsound, MIT Press.

• 9

  https://ccrma.stanford.edu/~ruviaro/texts/A_Gentle_Introduction_To_SuperCollider.pdf

• 10

  A Journey in Sound, September 2019, pp. 287-291.