Vocal pitchshifting by large amounts, octaves etc. AI?

[chagzuki]

I'm assuming octave shifts via traditional methods will always sounds artificial, Smurf-like etc., but it seems to me that AI ought to be able to cope with this sort of task.

I had a quick look at vocal AI tools and they seem to be largely novelty-driven on the basis that people can input their voice and transform it into a replica of a famous singer.

Are there AI tools for subtle manipulation of pitch and expression of voice?

[vurt]

you looking to sing like minnie ripperton?

[chagzuki]

you looking to sing like minnie ripperton?

Yep, that's the one. And change my name too. Got to go all in.

[zerocrossing]

you looking to sing like minnie ripperton?

Or Holly Herndon?

[jules99]

I'm assuming octave shifts via traditional methods will always sounds artificial, Smurf-like etc., but it seems to me that AI ought to be able to cope with this sort of task.

I had a quick look at vocal AI tools and they seem to be largely novelty-driven on the basis that people can input their voice and transform it into a replica of a famous singer.

Are there AI tools for subtle manipulation of pitch and expression of voice?

The most powerful and most complex tool in this regard I've come across is Ircam Trax: https://www.flux.audio/project/ircam-trax/

It's not cheap, but its tools change vocals so naturally, I haven't heard anything remotely close.

[CinningBao]

Octave shifting a voice will always be tricky if you don't want artifacts. A lot of physical stuff changes in the vocal path (of the human) when you sing in different pitches, for example switching to a 'head voice' from a 'chest voice' and that kind of stuff just isn't doable even in the best vocal manipulation software, IRCAM Trax included (which I have and love). Voices are just too complex for the currently developed DSP tools we use. And our ears are too sensitive to the artifacts.

There _may_ be a way someone is developing which would give us decent large-range vocal shifts (by creating a neural network which understands the differences between, for example, head and chest voice, and understands how to adjust the signal so it sounds natural at extreme shifts) but it isn't anywhere near public consumption yet, and is probably being developed by NSA or the US military.

Since we're in an Effects forum, I'm going to have to say, as well, that this kind of stuff will not be what we would call real-time. It will likely (first) be offline things like Melodyne using ARA to load audio in and to (pre-)effect it as the project plays. We can barely do decent FFT manipulation without a buffer to analyse the material (which is essential for the algorithms to work effectively).. I've done a little audio GAN work in the neural networking world, and we did achieve near-real-time extraction but the waveforms were very small and the brain was only running at 44.1kHz and 16bit.

Of course I may have to eat my hat as someone will probably pop up and say "THIS IS THE NEW AI TOOL YOU ALL NEED TO DO EXTREME PITCH SHIFTING OF VOCALS" and I'll have to line up a humble-pie pudding to consume after munching down my fibrous hat.

[chagzuki]

There _may_ be a way someone is developing which would give us decent large-range vocal shifts (by creating a neural network which understands the differences between, for example, head and chest voice, and understands how to adjust the signal so it sounds natural at extreme shifts) but it isn't anywhere near public consumption yet, and is probably being developed by NSA or the US military.

Yeah, I'd imagine AI could do a good job of figuring that stuff out... I'm sure it will in the near future, the field is developing so fast.