A small homage to a great compositional technique: resynthesis

Resynthesis as a form of imitation

Karlheinz Stockhausen used this compositional technique as early as 1955 in his "Gesang der Jünglinge", and yet it still seems new and fresh in today's premieres. The technique that has not lost its relevance for over 60 years is resynthesis. Unlike (sound) synthesis, i.e. the method of producing artificial sounds, resynthesis attempts to reproduce an existing sound, i.e. to imitate it. Resynthesis is therefore a form of imitation. The principle is simple: you take an original sound that emanates from an original sound source, recreate it with another sound source and then take it further in a way that would not have been possible with the original sound source. This enables a creative further development of the sound that goes beyond the capabilities of the original source. 

Examples from Messiaen to the present day

Even if one immediately thinks of computers when one thinks of resynthesis, it is only imitation and can be practiced by classical instruments. Olivier Messiaen chose birdsong as the original sound, transcribed it and developed his piece further from there.

In Stockhausen's aforementioned piece "Gesang der Jünglinge", a boy's voice is resynthesized by sine wave generators.

In “Monolit B” from Andrés Gutiérrez Martínez bass clarinet and double bass imitate each other and impress with their strong interchangeability.

And Clara Iannotta has taken resynthesis to the extreme with "where the dark earth bends". She chose the repeated audio signal space recording of another sound as the original sound, following the example of Alvin Lucier.

With attention to detail, precise determination of frequency pitches and the invention of new instruments, Clara Iannota created a work that could not be a better tribute to this compositional technique.

What always precedes a successful resynthesis is the analysis.

Analyzing a sound: spectralism as a precursor to resynthesis

The best way to analyze a sound is to look at its frequency spectrum. An example of a frequency spectrum, i.e. the change in sound over time, can be seen in the cover picture. Every sound, every noise can be composed of a sum of sine tones with different amplitudes. The realization that harmonic sounds consist of a fundamental tone and its multiples led to spectralism with Gerard Grisey. To this day, sounds made up of partials with random amplitudes are combined in concert halls in an attempt to hit the exact frequencies.

This is undoubtedly impressive, especially as these frequencies do not correspond to the notes found on the staff. Spectralism has therefore shown that it is possible to synthesize a frequency spectrum. It is a method of sound synthesis and therefore a precursor to resynthesis. Resynthesis goes one step further. The main reason why the a of a violin sounds different from the a of a flute lies in the amplitudes of the overtones.  The amplitudes are not coincidences; they imitate an original sound. While determining these individual amplitudes was still very time-consuming for Stockhausen, Clara Iannotta had a somewhat easier time with the means available today. She used a software tool to visualize the frequency spectrum, searched for the dominant frequencies and carried out a meticulous analysis. While such technical innovations are a help for the analysis part of the composition technique, software tools for resynthesis should be treated with caution.

Resynthesis: the importance of maintaining a degree of freedom

The fact that imitation and resynthesis have remained fresh to this day is certainly also due to the fact that they leave an incredible amount of freedom for creative work. While analysis and resynthesis form a close symbiosis in intuitive imitation in "Monolit B", the steps in "Gesang der Jünglinge" were clearly delineated. From the selection of the vocal sounds to their analysis and the addition of each individual overtone, which was necessary due to the small number of sine generators, each step allowed a great degree of creative freedom. If the orchestration of "where the dark earth bends" had been done by software like “Orchidea” , released in 2022, it would not be the same piece. Software will never write in the score that the strings should paint polystyrene if no one has had the idea before. The close collaboration with the wind players and the joint development of the piece would have been lost. Not to mention the emotionality, the fragility and the tension.

Technological progress has already buried some compositional techniques by making simple reproduction possible and thus leading to boredom, as in the case of granular synthesis. But this is not yet the case with this compositional technique. If you use new technologies in a targeted way, but leave enough freedom to intervene creatively, there is still enormous potential in this composition technique.

The potential of technologies

There are many algorithms in model-based signal processing that are ideal for resynthesis. The term "model-based" is important here. This means that there is a theoretical model, for example the physical model of the violin. The model has many parameters, all of which have a meaning. If you change them, you can expect certain results. For example, you change the length of a string and the pitch changes. For the piece "Lux Aeterna Unlimited", I used an algorithm from speech signal processing. If the algorithm analyzes a singing voice with it, you are then in possession of intuitive parameters such as pitch and amplitudes of the overtones. Under optimal conditions, the resynthesized sounds are indistinguishable from the original. This means that you can recreate the original sound perfectly and now have the parameters in your hands to shape the sound as you wish. Voices continue to develop into sounds that cannot be produced by humans alone.

When using the source-filter model to resynthesize the voice, the source (fundamental frequency and overtone frequencies) and filter (degree of reduction of the frequencies across the spectrum) can be used separately. Now the source can simply be replaced by other sounds and noises, as can be heard in the piece "Dies Irae".

Outlook: Sound shaping and composition

We are only at the beginning of the development of such algorithms. Imagine what would be possible if a loudspeaker could imitate an instrument and merge the advantages of chamber music, contemporary composition with the possibilities of signal processing. Much more can be expected.