The piano player automaton

The piano player automaton allows composers to realize what is "unplayable" for human beings, be it the most precise time structures, enormous speeds, the most diverse tempi at the same time, or simply an incredible number of notes. (Florian Gessler 2004)

Winfried Ritsch

Veröffentlichung
27.09.2024

Reading time
10 Minuten

Listen to article

Go back

The „Automatenklavierspieler“ aka „Autoklavierspieler“ is a playing mechanism, also known as a prefixer, that can be mounted on any conventional upright or grand piano.

The piano player automaton

A massive frame with 88 electromechanical fingers, which are moved by solenoids, is mounted on a keyboard. Controlled by micro-controllers, which are driven over a dedicated computer, the Autoklavierspieler can be controlled over Network, MIDI files, and real-time generated music. The Automat named "Kantor" has been constructed at the Atelier Algorythmics.1

CC BY SA Atelier Algorythmics

Kantor-Prototyp, Erstbetrieb im CUBE am Institut für Elektronische Musik Graz, 2003.

But how did the idea come about to construct and build such a machine, when mechanical automatons for playing the piano had existed for centuries?

Apart from the many music machines that have been built and used over time to create machine music, the aim was to implement an artistic idea together with the composer Peter Ablinger.

Fascination music machines

In the context of the metaphor of materialism in the 18th century, Denis Diderot writes about the fascination of musical machines, especially the piano. "If only it had a selective and dynamic memory, it could play like a human player".2

Not only the technical possibilities, but also the spirit of the age and musical thinking were crucial for the development of music machines; equally crucial is compositional thinking for machine music."3

The perception of the piano as a machine, as described in the 18th century, makes the piano an instrument that is particularly suitable for automation.

The challenge

Peter Ablinger's basic idea was to create a phonorealism4 comparable in effect to photorealistic painting in the visual arts. Beyond the purely technical aspect of the "Quadraturen", further aspects must be considered that were relevant to the development of the automaton. His first experiments with analog whole-tone filters led him to the aesthetic principles of the Quadraturen series and the method of using the analysis data of recorded sounds as source material for his compositions.

This took him to one of the most challenging disciplines, using spoken language to reconstruct the spoken with instruments like the speaking piano.

DEUS CANTANDO - Ablinger / Ritsch - piano speaking

It worked well in the computer simulation and sounded promising, so Peter asked me if a real player piano could do the same, and I naively said "Yes, why not". So we looked for a player piano that could play all keys simultaneously at different velocities.

The first piece in this series, "Zeit im Bild 2", was to be performed in 2003. So we conducted experiments to determine the feasibility of several self-playing pianos.

The Yamaha Disklavier could only play 16 keys in parallel with four different velocities. The Marantz player could not handle different velocities, and the 2006 Bösendorfer Computerklavier crashed due to too many notes, and so on.

However, our efforts were unsuccessful. Peter and I became increasingly nervous as time went on, so I promised him that if we couldn't find one, I would build one.

The insufficient power supply turned out to be a major challenge, as it did not allow for the required number of parallel keys with different velocities. Furthermore, the repetition rate was often limited, mainly by the data bandwidth. A single MIDI5 connection was not fast enough. And so I had to build one, and that was the birth of the player piano.

The build

As a reference for the build, some player pianos have been analyzed and the idea of a robot piano player sitting in front of a piano, has been taken from the idea of Trimpins player piano.6 This fits also the performance purposes, since pianos are widely spread and hard to transport.

The first version of the Autoklavierspieler, a robotic piano player named Kantor, powered by 1,5 kW and a weight of 120kg, was constructed in 2003. As well as artistic research on robotic electromechanical instruments for extreme performances, the main target was the realization of algorithmic compositions for 88 fingers, focused, but not only, on the work of Peter Ablinger. The initial project was interpreting audio recordings on the piano for the series “Quadraturen III”.

Millitron bei seiner ersten Performance in der Helmut Listhalle, Stadtoper Graz, 2008.

With new commissions, the need for dialogs in the compositions, and more performances in Europe, an additional better transportable Autoklavierspieler was needed, especially for the opera production ”Stadtoper” by Peter Ablinger. Millitron, as this one was named, could be optimized from previous experiences. It has half the weight, a dedicated micro-controller board, the "algopic" and "algofet"7, it played more precise, was better usable, especially for quiet pieces and therefore had a better "piano-forte" dynamics. Exchanging the use of a hold circuit, not using PWM Modulation during the hold phase of a key, eliminated the high-frequency noise from the solenoids.

Maschinenhalle #1 - Interview Winfried Ritsch Development, Electronic Music

In 2010 the music theatre production Maschinenhalle #18 required 12 pieces of the automata, and thus the first small series. A dance interface was also developed for this, and the series "Rhea" was created.

Rhea was developed with a focus on even faster repetition, better dynamics for pianissimo, easy transportation, and fast setup. The new electronics developed for this, should enable much finer calibration better adoption of old imprecise pianos, and easier control over Ethernet. Furthermore, Rhea should be the first series for reproduction as open hardware, enabling others to build and handle the robot piano player.

CC BY SA Atelier Algorythmics

Rhea Autoklavierspieler 2010.

Finger Strike

The concept was based on the idea that each key can be played separately and that each mechanical finger can be controlled individually with an envelope. This control can be adapted to the characteristics of any piano, especially older and less precise models. It was shown that a pressure of up to 3 kg per finger is required for full velocity. The repetition speed is limited by the piano's mechanics. Pianos with an efficient repetition mechanism can achieve repetition times of up to 50 ms. In general, the playing speed of grand pianos is perceived as higher and the dynamic range is greater.

First experiments with robotic piano playing 2003.

In the first phase of the attack, the finger is accelerated with maximum force. The acceleration is more relevant for the strength of the attack than the force at the end of the attack, since the speed of the hammer when it hits the strings corresponds to the volume. With a quiet attack, such as a pianissimo, the acceleration must also be adjusted accordingly to avoid a bouncing effect. The new electronics also allow a key to be pressed without the hammer striking the strings first, simply by lifting the damper. This is followed by the sustain phase. The key must not be pressed down to the stop to allow for faster repetition. The release phase is defined by the return spring of the solenoid. This results in a complex system in which the parameters are interdependent. For optimal function, it is necessary to calibrate the parameters for each button using software.

Electronics

The electronics for Rhea are based on the "Escher" microcontroller boards, which have Ethernet and serial interfaces. The FET amplifiers are used to control the customized solenoids. The "Escher" open hardware was developed with a focus on the "Autopianoplayer".

CC BY SA Atelier Algorythmics

Escher Controller.

Escher Board

Escher's open-source hardware is based on Microchip's dsPIC33F708MC DSP motor control microcontroller, a 16-bit controller with 160 MIPS, and a DSP unit that also includes an Ethernet controller. Escher boards provide more than 48 I/O pins, including a 12-channel hardware PWM at 20 kHz and 10-12 bits.

Each player requires 3 Escher boards, with one master board receiving commands over Ethernet, controlling the piano, and forwarding messages to the 2 slave boards via a fast serial connection. This way, each self-playing piano has an IP address and its status can be queried while playing.

EscherFET

To drive the solenoids, each Escher drives two EscherFET boards with 16 channels of FET amplifier each. This allows the individual control of the solenoid coils with a switching frequency of up to 100 MHz a continuous current of 4 A each and a peak current of up to 20 A, provided that the power supply is strong enough. Using high-frequency PWM improved the repetition rate of the piano from 80 ms to 50 ms for fast pianos. Precise control of the amplitude of each key was essential for a better reconstruction of the formant spectra of the speaking piano.

CC BY SA Atelier Algorythmics

EscherFET Amplifier Board.

Solenoids

The solenoids developed especially for Rhea were designed for optimal performance at the required stroke length of 10 mm and an operation at a maximum of 30 V and specially manufactured for this purpose. They also enable a simpler mechanical construction and reduce the overall weight. When activated at 16 times the permissible continuous load for up to 20 ms, the acceleration at the key strike increases and they can be operated at a temperature of up to 70 degrees Celsius.

CC BY SA Atelier Algorythmics

Solenoid constructed for series "Rhea".

To make the keys quieter when struck, cashmere felt was used, which has been used in piano manufacturing for centuries. The mechanical finger made of non-magnetic brass is equipped with felt as a fingertip.

Software

The whole system has been designed for the realization of performances, playing within ensembles, synchronized with video projections, installations with automatic operation and simple interfaces, and stand-alone solo performances. Fast set-up and individual calibration to different pianos and grand pianos have also been implemented.

Three stages of the software can be distinguished: A stage for the composition phase, which is mostly done offline, the performance software that implements the artworks, the integration into other environments, and the microcontroller firmware as standalone software that represents the robot player.

Rheaplayer, as Software Interface written in Puredata.

The player

The programming language Pure Data9 was used for the robot piano player's game to improve rapid prototyping, enable easy adaptation and integration into other projects, and ensure rapid expansion to meet new requirements. The software can also be used as a framework for other projects and runs on most operating systems, but preferably on Linux.

Escher Real-time Operating System

The firmware of the players Kantor and Millitron was written in assembly language to achieve adequate performance on the 16F877 microcontroller. For Escher, C with embedded assembly code was also used for time-critical tasks. The EscherOS firmware was implemented as a simple real-time operating system that distributes commands to parallel interrupt-controlled processes.

Outcome

As mentioned at the beginning, the motivation for the robot pianist was to transcribe recordings in the field of pianos, following a specific aesthetic principle, so that the result could be evaluated on the pieces and their performances at various festivals and installations. The three main works are mentioned here as examples.

Maschinenhalle #1

The concept of "music dancing" is not new. Many interactive implementations have been attempted, including playing instruments with dance. This interface, which allows a dancer to play the piano with reproducible output, was developed for the music-theatre performance "Maschinenhalle #1" for 12 dancers, 12 sound plates, 12 robot robotic pianists with pianinos, and a control station. The dancer sound-plate-piano-player unit has been developed with the choreographer Christine Gaigg the composer Bernhard Lang and stage designer Philipp Harnoncourt as part of an artistic research process. It has been designed as a kind of machine unit, including the dancer as a music worker.

Installation at Ars Electronica Center Museum 2011.

Heptapiano

At the 2011 Ars Electronica Festival, seven automatic piano players from the Rhea series were arranged in a 20-meter diameter circle and played, using Ambisonics Algorithm, as an spaatial piano space. The commissioned work by the Ars Electronica Center Linz has used the 32nd trope of Josef Matthias Hauer, "Und die Wellen umspielen uns," as a base for self-replicating mutation algorithms embedded in noise waves of piano tones.10

Heptapiano, seven robotic piano player, setup and played at Ars electronica 2011 at Lentos Linz

Walking Piano - Automatenklavierspieler on wheels

Resonante Zwischenräume: "Studie 3 - Public Improvisation & Studie 1 – Fassaden" – 360°

Walking Piano is an automated piano player that can be pulled across public spaces. In the series "Resonant Interspaces" as part of the series "Music & Architecture" 2018, new instruments were developed, and autonomous devices were built to acoustically play musical (inter)spaces in architecture. Facades as scores, sounds to fill spaces, and microphones that stroll along the pavement.

Resonante Zwischenräume: "Studie 2 - Mictail" - 360°

Ballet Mecanique - 1925/2022

In the 1920s, George Antheil heralded the peak of the Industrial Revolution with the idea of having his "Ballet Mécanique" played by an orchestra of pure machines. Eight drums, seven bells, a siren, seven aircraft propellers, and up to 16 pianolas, which were already widely used at the time, were to make this happen. However, Antheil failed to implement it technically. It took almost 100 years and the use of five Rhea player pianos and other music machines developed especially for the purpose, as well as precise computer control, to finally realize his vision. However, this and all other aspects of robotic sound generation require a further article.

ensemble mécanique plays ballet mécanique at Kunsthaus Graz

Conclusions and further outlook

After successfully proving with the mentioned artworks, that voices can be understood when played on pianos, the aesthetics of music done with this technique was an important step in this field. What first was foretold never will work, that a piano can talk and was first only a vision, became a successful artistic concept and can be surely enhanced furthermore.

Gray Sound: Peter Ablinger Residency

1
Atelier Algorythmics Graz, http://algo.mur.at/
2
Denis Diderot, Entretien entre d’Alembert et Diderot (1769).
3
Prieberg, F.K., Musica ex machina: über das Verhältnis von Musik und Technik. (1960) Ullstein Verlag.
4
Peter Ablinger, "Phonorealism", online article: http://ablinger.mur.at/phonorealism.html, accessed 30.3.2013.
5
The Music Instrument Device Interface -file standard as serial control interface with about maximal one command per millisecond transmission rate.
6
Wikipedia. Trimpin — Wikipedia, the free encyclopedia. 2011; Online accessed 27-April-2011.
7
Based on PIC16F877 and a two-stage FET solenoid driver this circuit was also used in a lot of other artwork, see "algopic - algofet", http://algo.mur.at/projects/microcontroller/algopic/algopic
8
http://maschinenhalle.at/
9
M. Puckette. "Pure data", in Proceedings, International Computer Music Conference., 224–227. San Francisco, 1996.
10
Heptapiano 2011 URL: http://algo.mur.at/projects/autoklavierspieler/performances/heptapiano

Winfried Ritsch

Winfried Ritsch (born 1964, Tyrol) is an Associate Professor of Computer Music at the Institute for Electronic Music and Acoustics (IEM) at the University of Music and Performing Arts Graz and runs the ‘Klangatelier Algorythmics’. He studied electrical engineering and sound engineering at the Graz University of Technology and the former University of Music and Performing Arts Graz. In addition to his compositions and large media art installations, he has developed innovative artistic concepts and initiated media art laboratories and the artists' initiatives FOND, TONTO, mur.at and has realized art and music theatre productions. He has been honored with art prizes for his work as a composer, including the 1994 City of Graz Prize for Composition, the Max Brandt Prize for Composition in 1997, and the Andrzej Dobrowolski Composition Prize of the Province of Styria in 2020. He tours extensively with his experimental computer music and media art performances and media art installations, in particular with robot ensembles such as the ‘ensemble mècanique’, most recently realizing robotic exhibitions at Kunsthaus Graz 2019 and Museum Fernand Léger in Biot/France. For his artistic activities in the fields of radio art, sound art, and the realization of telematic art projects, he develops robotic musical instruments, and cybernetic models for generative and interactive music and acoustics as open-source projects and has been operating his net culture servers on the Internet since 1998.