What if you could sing in your favorite musician's voice? | Holly Herndon

Hey, I'm Elise Hugh, you're listening to Ted Talks Daily.

And yall, you're in for a treat.

It's surprising and it's delightful

from the multidisciplinary artist Holly Herndon.

Her Ted 2020 talk introduces us to her digital twin.

That is how she's married her own voice with AI

to write songs using machine learning of her voice.

By the end, she performs and so does her voice

in a separate musician altogether.

If this doesn't make sense, you'll get it once you listen.

Hey, I'm Stephen Johnson, host of the Ted interview podcast.

On this show, we talked to some of the world's most interesting people.

On the latest episode, Researcher and Activist Zarru, J.I.R.R.M.

talks about her nationwide campaign for a fair wage in the service industry.

We're in a very historic moment where after 160 years of this wage structure

lasting since emancipation, it is finally being rejected by millions of workers.

Check out the Ted interview on Apple Podcasts, Spotify or wherever you listen.

Hi, my name's Holly, and I'm an artist.

That was my voice, but I didn't sing that clip.

I trained an AI on my own voice, and now she can sing anything in multiple languages.

Her name is Holly Plus, and you just heard her perform

El Conte de la Fabila, which is a traditional song

arranged by Maria Arnaul and Catalan, not a language that I speak,

and not a vocal tradition that I've trained in.

Those melodramatic runs are really difficult to hit.

To show you Holly Plus's full range, I'll also play an example in German.

This is Mack the Knife by Brecht.

Of course, Holly Plus can also perform my own music, so here's an excerpt from my own song

Frontier.

Holly Plus uses a process called Tamber Transfer, where the tamber or sound quality of one sound

may be mapped onto the performance of another.

Tamber Transfer is done by creating a machine learning model of a sound,

in this case it's my voice, so I recorded a wide variety of phrases in my entire vocal range.

To be clear, this version of Holly Plus is reading notes from a score.

So I'll also play you an example of her singing in Spanish,

best in a Mujo by the lascas.

Thank you.

I can't speak Spanish, by the way, so.

Teaching an AI of the sonic properties of one sound in order to generate an entirely new sound

is what I like to call Spawning. Spawning is what allows Holly Plus to create a wide range of

vocals that I didn't sing from a set of recorded phrases that I did sing.

I like to think of Spawning as a kind of 21st century corollary to the musical tradition of sampling,

which had a really big impact on both music and intellectual property, but I think Spawning

is far more exciting and potentially really weird.

So for example, with sampling, usually you copy and remix a recording by someone else to

create something new, but with Spawning, you can perform as someone else based on trained

information about them. And as an artist, this is making me rethink my own past work, as not only

my archive, but potentially also I myself could become reanimated with AI. This also opens up

the question of how we deal with the collective human archive, if we can reanimate old media.

It opens up really big ethical and intellectual property questions that require entirely new

conceptual and legal frameworks. One way that I like to think about intellectual property,

I call identity play. So rather than limiting the use of my voice, I'm creating instruments

to allow as many people as possible to create music with me and even as me. That's why I've made

versions of Holly Plus freely available for anyone to use online. If I can allow people to play with

my IP, my digital identity, my intellectual property, what might they come up with? Could someone

else go on tour as me with my permission? Could I be in a thousand different bands in multiple

languages and what would that even sound like? To be clear, musicians have been taught to be

really protective over our IP and that's for really good reason. But I'm trying to think of ways

to make this new capability mutually beneficial. So to allow people to use my voice,

but still to maintain the ability to approve certain derivative works. So I invite you to

consider, if given the opportunity, who would you like to perform through? And can you imagine

someone else performing you? With that in mind, I'd like to invite the incredible musician

fur to the state. So today, fur will be performing his own song, murky. And with one microphone,

you'll hear her first beautiful natural voice. And with this microphone, you'll hear a live

version of Holly Plus developed with Valkyrie Labs.

Take it away for him.

Yes, the truth is I show you every day.

And it's so murky that I've been wise to know, but I'll keep on swimming

further up the road. And I've been calling out calling out your name deep down in my sleeve.

And I even memorize your face. So I ain't no longer even need. And I just handle my best

and keep the train moving on. And you know what it is. It's just you that I'm choosing

because it's so murky that I've been wise to know. And I'll keep on swimming further up the road.

And it gets so murky that what you know. And I'll keep on swimming further up the road.

Swimming, I'm swimming further up the road. And I'll keep on swimming

further up the road.

PRX