Artificial intelligence in music: feared, trivialized, demonized (part 1)

An attempt to approach a complex phenomenon that will occupy all music makers in the coming years. Markus Deisenberger explores these questions from different perspectives in our three-part series - here is part 1 with a cookbook created by AI, improvising with AI, and the almost compulsive urge to find solutions in trained machines.

"If AI is the future of music, we need better drugs," said Damon Alban from Blur in a recent interview, and his quote was just one of many on the subject. The discussion about the use of artificial intelligence in the art and media world has long since reached the center of society, and the situation is somewhat reminiscent of the pandemic. Not only does almost everyone seem to have an opinion on the subject, but the views expressed also oscillate between scaremongering and trivialization, just as they did back then.

Want an example? Historian and bestselling author Yuval Harari, for example, believes that the new technology could destroy the foundations of our civilization. AI will hit us worse than climate change, he says. In contrast, Hannes Eder, the former General Manager of Universal Austria, recently expressed the view in a Standard interview (in connection with the fake song by Drake that was in the news at the time) that there was no need to get hysterical about it (yet). After all, the song had been removed from the platforms and it had not achieved intoxicating streaming figures either. Music companies also hold shares in streaming platforms. In this respect, there is still no need to worry about original music, says Eder. And money will probably continue to rule the world to some extent.

That's right. Money will probably continue to rule the world. But what if there is simply more money to be made with AI-generated music and the audience - or let's say: large parts of the audience - don't care whether they are presented with a real or a fake version? So which reaction to the latest developments is appropriate? Eder's relaxation or Harari's fear? So the first question would be: How far along is the AI? What can it do? And how interchangeable can it make us humans appear because of its capabilities?

Artificial intelligence, transparency, and 99 pasta dishes - to be enjoyed with caution

I talked to Siegfried Handschuh, Professor of Computer Science at the University of St.Gallen in Switzerland, where he is primarily concerned with Natural Language Processing (NLP), i.e. the processing of natural language by computers, about precisely this, about the enormous possibilities of artificial intelligence, but also about its limits. He is very cautious about the advertised ability of ChatGPT (a chatbot that uses artificial intelligence to communicate with users via text-based messages and images) to write sonnets in the style of Shakespeare: "It can imitate style and structure very well, but the system doesn't have the creative genius and human understanding that Shakespeare had. It works according to statistics, providing the most probable answer to a question. But the most probable is also predictable. And the interesting things tend to be unpredictable and unusual."

According to Handschuh, I don't need to worry too much about my job as a journalist, because "something that requires a lot of experience, empathy and perhaps also genius can only be simulated poorly." This means that in the future, companies may be able to have average PR articles done in-house by AI, because generally speaking, Handschuh says, it will be possible to enable people who would otherwise not be able to write texts to write useful articles. But a journalist who writes editorials or talks to someone about artistic intelligence cannot be replaced so quickly. "I assume that assistants will be replaceable. Lawyers themselves will not be replaced so quickly," says Handschuh.
Will these statements stand up to the latest developments? The Cologne tabloid Express has recently employed an AI journalist and the name Klara Indernach (AI for short) is used for texts created with the help of artificial intelligence. "If articles were largely generated with the help of AI, we mark them accordingly," says the editorial team. Before publication, they are of course edited and checked, they add. At least transparent, one inevitably thinks, because we remember a cookbook with 99 pasta recipes published by Burda last May. As the SZ investigated, the entire booklet was produced by AI, including the pictures of the pasta dishes. However, the readers were not informed about any of this.

Of course, AI can prove helpful when evaluating large amounts of data, which can be important in investigative journalism, among other things. In the best-case scenario, AI tools enable more efficient work and free up time for research. But of course, whenever AI content is generated, mistakes can also happen. Klara Indernach's texts should therefore be treated with caution, wrote the SZ. This is probably true to the extent that a human should always check AI-generated content for factual accuracy, but it is also a little unfair because, as far as I know, Klara Indernach and the system have not yet been proven to have made any gross errors. And: The texts of many human journalists should also be "treated with caution".

But Indernach's texts are - if we remember what Siegfried Handschuh said - neither simple PR nor otherwise primitive, which suggests that development is progressing faster than we might all like.

For example, a media company in Kuwait recently unveiled its first virtual news anchor working with artificial intelligence. The AI presenter named "Fedha" made her debut on the Twitter account of Kuwait News. ORF is also working on the use of artificial intelligence in its editorial offices, for example, the use of an "AIDitor" (Artificial Intelligence Editor), which creates social media posts from existing journalistic material.

And in music? What's the situation there? What came out when ChatGPT was asked to compose a song in the style of Nick Cave is well known, as is Cave's reaction to the poor work of art. The result was "bullshit" and "a grotesque mockery of what it means to be human", said the famous singer-songwriter. Siegfried Handschuh can understand Cave's angry reaction. "I would probably react the same way if someone tried to copy my work," says the Swiss researcher. "The system produces texts that have certain stylistic elements of his poetry, but they have nowhere near the potential of the originals." So far so good. So the all-clear? Cave and all other lyricists need not worry too much. Not at all, because Handschuh explicitly points out that AI is a generalist system. "If I were to train the system only on songs by Nick Cave or pop songs per se, it might look different." So if I were to feed a chatbot only with indie and alternative music from the last thirty years, it would very likely come up with a result that Cave wouldn't see as a mockery, that might even amaze him. "What's more, development is progressing exponentially," says Handschuh. This means that while we are still laughing about the fact that the machine is still mediocre in its copy of some artistic genre, a new, improved version of the machine has long since been launched. This may make assessing the quality of an AI today obsolete tomorrow. "And there are genres," says Handschuh, "in which it works better from the outset." "Schlager, for example?" I ask. "Exactly. That would be worth a try."

Emotional machines and their almost compulsive urge to find solutions

But there is also another fact to consider: "In the old science fiction films, there were always characters like Spock or Data who explained everything completely rationally and failed at the emotional, associative aspects," says Handschuh. "Conversely, logical thinking and mathematics tend to be difficult for the average person. Generalizations, recognizing patterns, the associative - that suits us, that's easy for us." What he is getting at is that there has always been this clear, traditional dividing line between the realm of mathematics and logic on the one hand, which is easily accessible to the machine, and that of creativity and intuition on the other, which it struggles with. This has changed fundamentally, says Handschuh, because "these chatbots are very good at making associations. They are good at making things up when they don't have any information. That doesn't mean we're there yet, but we're on the right path to creating a really powerful AI. In ten or fifteen years, we will have about a thousand times the computing power at our disposal." Then he can already imagine "that one day we will have something like a digital consciousness. And the larger the systems become, the more they develop characteristics that we didn't expect. They react to the inputs much more strongly than we expected."

This reaction is known as "few-shot learning". The machine develops an almost compulsive urge to find a solution when a gap opens up, which many people will be familiar with from the media and perhaps also from their experiments with ChatGPT. You should always check a presentation on a certain topic or even literature lists carefully, because sometimes the machine simply invents something; it closes the gap it has identified imaginatively or arbitrarily - depending on how you want to interpret it.

While Siegfried Handschuh and I are talking on the phone, he has ChatGPT search for crime plots for my next book because he has heard that I have written a crime novel. The result that he presents to me towards the end of our conversation astounds me, because while some of the plots are far-fetched or obsessively original, others sound exciting and would be well worth pursuing. An interesting story could actually be developed based on this.

Somehow this contradicts what Handschuh said at the beginning about the potential of current AI systems, because: AI can by far not only be used as a mediocre assistant, but can also be used to generate creative ideas.

Fear of incapacitation

Sounds fascinating and like a promise for the future. So why, according to the results of a recent representative survey, do around 40% of Germans believe that artificial intelligence will have a negative impact on the bottom line over the next ten years? A further 40 percent expect mixed effects. Only 14% believe it will have a "positive impact on the world".

A few years ago, I asked the German AI expert Sepp Hochreiter, who heads the Institute for Machine Learning and the Artificial Intelligence Lab at the Johannes Kepler University Linz, in an interview where he believes the widespread fear of AI comes from. Is it the cinematic dystopias from Hollywood that are fueling the fear that machines could one day overtake us?

"There are several overlapping fears," he said at the time. "First of all, people are afraid of everything new. Then they are afraid of things they don't understand. Then there's the fear of being incapacitated. The computer notices that I make spelling mistakes, and that I am inadequate in some areas. Finally, there is the fear of surveillance and that decisions will no longer be made according to human criteria." The absurd thing, however, is that most people trust politicians, however stupid and clueless they may be, rather than a machine that makes a rational decision, says Hochreiter. "And now there's another facet to it: The fear fueled by movies like 'Matrix' that machines could take over the world and use us, humans, as batteries, which is complete humbug. Why? Because there are already better batteries than humans."

"So do you think natural stupidity is more dangerous than artificial intelligence?" I asked him. His answer: "Definitely. But the most dangerous thing is when stupid people use very intelligent machines. The stupid person, for example, who has equipped his missile with AI and can therefore hit the target accurately ..." Only the combination of AI and human stupidity results in a truly dangerous cocktail.

Western connotations

Back to the music. The Viennese musician Sharmien Zandi has been working intensively on what AI can mean for music production. When she first heard what a data set meant, it was overwhelming, she says. She immediately developed a great thirst for research. "AI must have something to do with music, not just images," was her first thought. A start-up grant from the City of Vienna finally allowed her to explore the topic in greater depth as part of a project. In her project, she researched which platforms are currently available, what is available to musicians, and what results can be generated with them.

"I was interested in gaining access, learning what data sets exist, how they are used, and developing a better awareness of this technology." When she started, there was only a music-to-music generating machine, she says, "but no way to develop text to music yet." That only came later. In the beginning, the parameters were still very simple, but they have since become more specialized as development has progressed, says Zandi. Her impression: "It still has a very Western connotation." There is hardly any Arabic music, for example, but heavy, rock, pop, and jazz are very strongly represented. You quickly realize which data sets are behind it. And the big difference to the visual area: "There's not much surprise." And some things, such as silence, cannot be generated by the machine. (This makes her next project "Silents and AI" all the more exciting). One of the first questions she asked herself was: "Where do they get their data sets from and why do I have to pay for them if it's not even clear where the data sets come from?" An exciting and legitimate question that we will come back to. Her approach to "the real digital piano" was very simple: she sat down at the piano and improvised. "I aimed to understand the process. Then the AI turns the improvisations into something." In a further step, she then tried to work out a score together with the AI. Her question in this regard: "Am I the author or the AI?" is a question we will come back to.

Her research results, including the videos "Love against the Machine" and "Fuck Me Amadeus", can be viewed on her website. Her summary: "The results weren't so cool that I think I've created the latest one-hit wonder." Objectively speaking, that may well be the case, however wooden and not entirely polished some of it may still sound. Particularly in the field of electronic music in techno and dance, there is a feeling that there could be a revolution, perhaps not yet today, but very soon, through more precise prompting and better production, that perhaps in the foreseeable future machines will set the beat, or that the real collaboration between man and machine, as the band Kraftwerk once envisioned in a romanticized form, could become reality. In jazz, there is already a whole series of music projects in which musicians interact with AI in an improvisational way.

This article first appeared on mica - music information center austria on November 27, 2023. Since the topic of music and AI is of particular interest to Sounding Future, we have republished the article here as part of a cooperation between mica and Sounding Future.