AI Has Been Creating Music and the Results Are...Weird

In late May, a minor oversupply at St. Dunstan's church in Due east London'south Stepney district gathered for two hours of traditional Irish music. But this consequence was different; the tunes it featured were composed, in part, by an bogus intelligence (AI) algorithm, dubbed folk-rnn, a stark reminder of how cutting-edge AI is gradually permeating every aspect of human life and civilization—even creativity.

Adult past researchers at Kingston University and Queen Mary Academy of London, folk-rnn is one of numerous projects exploring the intersection of artificial intelligence and creative arts. Folk-rnn's performance was met with a mixture of fascination, awe, and consternation at seeing soulless machines acquisition something widely considered to exist the sectional domain of man intelligence. Simply these expeditions are discovering new ways that man and car can cooperate.

How Does AI Create Fine art?

Like many other AI products, folk-rnn uses machine learning algorithms, a subset of artificial intelligence. Instead of relying on predefined rules, machine learning ingests big data sets and creates mathematical representations of the patterns and correlations it finds, which it and then uses to accomplish tasks.

Folk-rnn was trained with a crowd-sourced repertoire of 23,000 Irish gaelic music transcripts before starting to creepo out its own tunes. Since its inception in 2022, folk-rnn has undergone three iterations and has produced more than 100,000 songs, many of which take been compiled in an fourteen-volume online compendium.

Flow Machines, a five-year project funded past the European Research Council and coordinated by Sony's Computer Science Labs, too applied AI algorithms to music. Its most notable—and bizarre—accomplishment is "Daddy's Motorcar," a song generated past an algorithm that was trained with lead sheets from xl of The Beatles' hit songs.

Welcome Mistakes

Algorithms can mimic the style and feel of a musical genre, but they often make basic mistakes a human composer would not. In fact, most of the pieces played at folk-rnn'due south debut were tweaked by human musicians.

"Art is not a well-defined problem, because you never know exactly what you desire," says Francois Pachet, who served equally the atomic number 82 researcher at Menses Machines and is now director of Spotify's Creator Applied science Research Lab. But, he adds cheerfully, "it's good actually that fine art is non well defined. Otherwise, it would not be fine art."

The generated lead sail for "Daddy's Car" was also edited by a man musician, and some tracks were added by hand. "There was pretty much a lot of AI in there, but not everything," Pachet says, "including voice lyrics and construction, and of course the whole mix and product."

"The real benefit is coming up with sequences that aren't expected, and that lead to musically interesting ideas," says Bob Sturm, a lecturer in digital media at Queen Mary, University of London who worked on folk-rnn. "Nosotros want the arrangement to create mistakes, only the right kind of mistakes."

Daren Banarsë, an Irish musician who examined and played some of the tunes generated by folk-rnn, attested to the benefits of interesting mistakes. "In that location was one reel which intrigued me," he says. "The melody kept oscillating between major and minor, in a somewhat random manner. Stylistically, it was incorrect, but it was quirky, something I wouldn't have thought of myself."

Spotify'south Pachet explains that these unexpected twists can actually aid improve the quality of pop music. "Take the xxx or 50 almost popular songs on YouTube. If you lot look at the melody, the harmony, the rhythm and the structure, they are extremely conventional, which is quite depressing. You accept only iii or four chords, and they're always the same. Creative AI is very interesting, not only because information technology's fun, but also because it brings hope. I hope that we could modify or affect the quality of the most popular songs today."

No Right Answers

"The thing that makes art wonderful for humanity is that there is no right reply—it's entirely subjective," says Drew Silverstein, CEO and co-founder of Amper Music, an AI startup based in New York. "You and I might heed to the verbal same piece of music, and y'all might like it, and I might hate it, and neither of u.s. is right or wrong. It'southward just different.

"The claiming in the modern earth is to build an AI that is capable of reflecting that subjectivity," he adds. "Interestingly, sometimes, neural networks and purely data-driven approaches are non the right respond."

Oded Ben-Tal, senior lecturer in music engineering at Kingston University and a researcher for folk-rnn, points out another challenge AI faces in respect to creating music: Data does not correspond everything.

"In some means, yous can say music is information. We listen to a lot of music, and as a composer, I get inspired past what I hear to make new music," Ben-Tal says. "But the translation into information is a big stumbling cake and a large problem in that analogy. Because no data actually captures all the music."

To put it simply, an AI algorithm's interpretation and agreement of music and arts is very different from that of humans.

"In the instance of our arrangement, it's far too easy to autumn into the trap of saying information technology'south learning the style or it'due south learning aspects of Irish music, when in fact it's non doing that," says Sturm. "It's learning very abstract representations of this kind of music. And these abstract representations have very fiddling to exercise with how you lot experience the music, how a composer puts them together in the context of this music within the tradition.

"Humans are necessary in the pursuit because, at the end of the day, we have to brand decisions on whether to incorporate certain things produced by the estimator that we curate from this output and create new music," Sturm says.

Google's DeepDream

In visual arts, the divide betwixt the perception of humans and machines is even more than accentuated. For case, take DeepDream, an inside-out version of Google'due south super-efficient image-classification algorithm. When you give it a photograph, it looks for familiar patterns and modifies the image to look more like the things information technology has identified. This tin can be useful to turn rough sketches into more enhanced drawings, but it yields unexpected results when left to its own devices. If you provide DeepDream with an epitome of your face and information technology finds a pattern that looks like a canis familiaris, information technology'll turn a part of your face into a dog.

"It'south nigh like the neural cyberspace is hallucinating," an artist who interned at Google'southward DeepMind AI lab said about the software in an interview with Wired concluding yr. "Information technology sees dogs everywhere!"

But AI-generated art often looks stunning and can rake in thousands of dollars at auctions. At a San Francisco art evidence held last year, paintings created with the help of Google's DeepDream sold for up to $8,000.

The Business of Creative AI

While researchers and scientists continue to explore artistic AI, a scattering of startups have already moved into the infinite and are offering products that solve specific business use cases. One is Silverstein'south Amper Music, which he describes as a "composer, producer, performer that creates unique professional music tailored to any content in a matter of seconds."

To create music with Amper, you lot specify the desired mood, length, and genre. The AI produces a basic composition in a few seconds that you lot tin can tweak and adjust. Amper also offers an awarding programming interface (API), so developers can incorporate the platform'due south creative power into their software.

Jukedeck, a London-based startup created by two onetime Cambridge Academy students, provides a like service. Like Amper, users provide Jukedeck with bones parameters, and information technology provides them an original musical track.

The principal customers of both companies are businesses that require "functional music," the type used in ads, video games, presentations, and YouTube videos. Jukedeck has created more than than 500,000 tracks for customers including Coca-Cola, Google, and London's Natural History Museum. Composers are as well learning to utilise the tools to enhance the music they create for their customers.

A third startup, Australia-based Popgun, is building an AI musician that can play music with humans. Named Alice, the AI listens to what you play and then responds instantly with a unique creation that fits with what you played.

In the visual arts industry, business use cases are gradually emerging. Last year, Adobe introduced Sensei, an AI platform aimed at improving human creativity. Sensei assists artists in a number of ways, such as automatically removing the background of photos or finding stock images based on the context of a affiche or sketch.

Collaboration Between AI and Human Artists

Perhaps not surprisingly, these startups are founded and managed by people who have strong backgrounds every bit artists. Amper'south Silverstein studied music composition and theory at Vanderbilt University and equanimous music for TV, films, and video games. Ed Newton-Rex, founder and CEO of Jukedeck, is also a good music composer.

But non everyone is convinced of the positive office of artificial intelligence in arts. Some of the attendees at folk_rnn's issue described the AI-generated pieces as lacking in "spirit, emotion and passion." Others expressed concerned for the "cultural impact and the loss of the human beauty and understanding of music."

"I haven't met ane musician that I've told virtually this who hasn't reacted with something close to the negative side of things," said Úna Monaghan, a composer and researcher involved in folk-rnn who spoke to Inverse. "Their reaction has been from slightly negative, to outright 'why are you doing this?'"

The developers of creative AI algorithms do non generally share these concerns. "I don't think humans will become redundant in music-making," says Newton-King. "For a outset, nosotros as listeners intendance about much more than just the music we're listening to; we intendance nearly the artist, and about their story. That will always be the case."

Automation and IoT Predictions

"Nosotros think of functional music equally music that is valued for its use case and not for the inventiveness or collaboration that went into making it," Silverstein says. But artistic music, Silverstein explains, "is much more about the process than the apply case. Steven Spielberg and John Williams writing the score of Star Wars, that's about a human collaboration."

"The primal use-cases we see lie in collaboration with musicians," says Jack Nolan, co-founder of Popgun. "Artists can use Alice as a source of creative inspiration or to help them come up with melodies and chord progressions in their music. We don't think people will e'er end wanting to create their own sounds. We recollect AI will help them exercise this, rather than replace them."

Daren Banarsë agrees on the benefits of collaboration. "I e'er detect it daunting when I have to start a large-scale composition. Possibly I could give the computer a few parameters: the number of players, the mood, fifty-fifty the names of some of my favorite composers, and information technology could generate a basic structure for me," he says. "I wouldn't expect it to work out of the box, just it would exist a starting point. Or information technology could output a choice of melodic ideas or chord progressions for me to look through. And somewhere in there, there's going to be a reckoner glitch or random quirk, which could take me in a completely unexpected management."

Ben-Tal admits that some jobs might be affected. "Working musicians volition have to adapt," he says. "I bear witness this to my students and say, 'You need to upwardly your game.' This will mean some of the entry-level jobs into the music industry will non exist at that place in five or 10 years, or yous'll need to exercise things differently or have a different ready of skills."

'Democratizing Creativity'

AI creativity can also help people without inherent talent or hard-earned skills express themselves artistically. Take Vincent'due south AI drawing platform, which helps transform rough sketches into professional-looking paintings, and the AI music platforms that create decent music with minimal input.

Jukedeck'southward Newton-Rex describes this as "democratizing" creativity. "People with less formal musical education tin get to grips with the basics of music and use AI to assist them make music," he says.

Pachet concurs. He draws an analogy between recent AI developments and the arrival of the first digital synthesizers in the 80s, followed past digital samplers. At the time, there was a similar fright that musicians would lose their jobs to computers. "But what happened was the exact opposite, in a sense that everyone took these new machines and hardware with them and learned how to utilize them productively," he says. "The music industry exploded in some sense."

"There will exist more people doing music, and hopefully more interesting music," he adds, reflecting back on AI creativity. "I cannot predict the future, merely I'm non worried about AI replacing artists. I'grand worried about all the other things, the well-defined problems, like automated healthcare and autonomous vehicles. These things are really going to destroy jobs. But for the creative domains, I don't call up it's going to happen."