Video Editors Are the Next Frontier for AI Music Generators

Quick recap: Legal problems for AI music generators in 2024

During 2024 we saw lawsuits by the RIAA against VC-backed megaliths like Suno and Udio. Global music licensing body Merlin publicly announced that they do not want AI music companies training on their audio. It’s difficult to imagine the future of sound without talking about artificial intelligence. There are dozens of AI music generators on the market today and we’ve explored their impact on the industry at length in a recent article here. 

To understand what’s coming next, we need to look at the dynamics of 2024 along with the unmet needs of a new creator class; the AI filmmaker. 

It’s unclear how long it will take for the RIAA’s lawsuit to conclude. Meanwhile, Suno and Udio are still actively selling their services to an engaged user base.

Major labels may need to work out a new class of music licensing deal for AI startups. Traditional sync fees are prohibitively expensive and don’t scalable for the volume of music that AI models require in order to produce a diverse range of high quality outputs.

Most of today’s conversations about the future of AI music end up focusing on legal issues like model training, content detection and attribution, and artist remunerations. These are important topics that need to be discussed, but yet they tend to overshadow the exciting new capabilities, DAW workflows and creative user experiences that are emerging on the open market.

Additionally, parallel development in AI video generation are about to create new demand for AI film score generators that reflect trends in popular films. Independent content creators will be looking for engaging, dramatic scores they can legally use in their films.

AI film music generators: The next wave of innovation

Major innovations in generative image-to-video are opening up new possibilities for storytellers to create short and full length feature films on a relatively small budget. Yet writing and producing music for a fantasy or adventure title is remarkably difficult, even for an advanced composer.

Danny Elfman, the composer of Tim Burton’s film music, has explained in interviews that his MIDI computer music often sounds less compelling and requires a live orchestra. Very few independent filmmakers have the budget to hire an ensemble of that size.

The majority of AI music generation companies today have focused on beatmakers rather than composers. Products released before 2023, like Boomy and Soundraw, are closer to algorithmic loop remixers than the diffusion-transformer models that define state of the art technology today.

AI filmmakers are unable to create rich orchestral scores with these tools.

Suno and Udio represent the gold standard in AI music models. As this article on AI filmmaking explains in detail, both services are capable of generating scores in any style. Of the two options, Udio appears to create richer and more realistic film cues.

However, as this lawyer explains in a video from November 2024, Suno does not currently grant users the right to commercial use of any music on their platform. This means that you’re free to experiment with creating AI film scores but must not profit financially from it in any way:

There are workflow challenges associated with Suno and Udio as well. Composers are usually tasked with writing and rewriting their scores to fit last minute video edits. The music must adapt constantly to fit the director’s vision. There is no AI music generator today that offers granular controls over timing or arrangement. You can’t dictate energy arcs that align with a film scene.

On a positive note, the sound design DAW Audio Design Desk has improved on many of the industry challenges associated with audio procurement, triggering, and reconforming. They have built in sonic intelligence that hot-swaps new samples based on anchor points.

Audio Design Desk has also built an AI music generation interface, SoundGen, that will soon load directly within their sound design DAW.

© AudioCipher Technologies

The next generation of AI music generators for film will likely involve a system like Audio Design Desk, that approaches composition based on text prompts, energy arcs and trigger points.

The other possibility would be a fully automated solution that requires no human input at all. Video-to-sfx models like MMAudio are already circulating across the internet, but video-to-music has not yet been properly achieved. 

Film companies will likely own the AI music models

Over the next 3-10 years, tech companies will likely train diffusion-transformer models in partnership with library rights holders. Top shelf film scores will not be made available to tech companies unless they expand on the value of their existing audio assets.

Therefore, the most likely outcome would be that rightsholders enter into close partnerships with tech companies and own their models outright, collecting large revenue shares from both generation and music publishing.

It’s unclear how long this process will take, considering there are no dedicated AI film score products on the market today and nobody has stated the intent to release a model of this kind.

We’ll have to wait patiently for the legal details to be worked out first.