High quality audio morphing effect plugin, why it doesn't exist yet?

[mandolarian]

but I wouldn't say it's a well defined (or definable) process at all.

At least what i described is how for example Dr Boulanger uses the term in his audio programming book and Dr Eric Lyon uses it in his PD externals. I understand that you don't like how it sounds, but that's a different question.

Uh...I thought it was 'The Question'. High Quality audio morphing seemed to be the key point. Maybe the thread has morphed to include all the low-fi variants.

[highkoo]

Sound morph can be compared to picture morph, it's not voodoo.
Read this easy to read 2 pages of basic stuff:
http://music.columbia.edu/cmc/musicandc ... /05_06.php

Thats a good link.
The last couple paragraphs are I think where this needs to go;
"Morphing the Centroid"
and
"Feature Morphing"

Feature Morphing Example: Morphing the Centroid

Music cognition researchers and computer musicians commonly use a measure of sounds called the spectral centroid. The spectral centroid is a measure of the "brightness" of a sound, and it turns out to be extremely important in the way we compare different sounds. If two sounds have a radically different centroid, they are generally perceived to be timbrally distant (sometimes this is called a spectral metric).

Basically, the centroid can be considered the average frequency component (taking into consideration the amplitude of all the frequency components). The formula for the spectral centroid of one FFT frame of a sound is:
...
The (individual) centroid of a spectral frame is defined as the average frequency weighted by amplitudes, divided by the sum of the amplitudes, as follows:

We add up all the frequencies multiplied by their amplitudes (the numerator) and add up all the amplitudes (the denominator), and then divide. The "strongest" frequency wins! In other words, it's the average frequency weighted by amplitude: where the frequency concentration of a sound is.


Now let's take things one step further, and try to morph the centroid of one sound onto that of another. Our goal is to take the time-variant centroid from one sound and graft that onto a second sound, preserving as much of the second sound's amplitude/spectra relationship as possible. In other words, we're trying to morph one feature while leaving others constant.

To do this, we can think of the centroid in an unusual way: as the frequency that divides the total sound file energy into two parts (above and below). That's what an average is. For some time-variant centroid (ci) extracted from one sound and some total amplitude from another (ampsum), we simply "plop" the new centroid onto the sound and scale the amplitude of the frequency bins above and below the new centroid frequency to (0.5 * ampsum). This will produce a sort of "brightness morph." Notice that on either side of the centroid in the new sound, the spectral amplitude relationships remain the same. We've just forced a new centroid.

I think that multiple optional Feature Morphs are what we are all going to want for audio. (Assuming it cant be automatic.) Like, we want to be able to make various start and end 'assignments' on features. And I think that smooth Centroid morphing is what has to happen for the in-between audio to even make sense, let alone sound "good".
(I mean, when I think about doing it 'simple', like with a crossfader, the chances of the in-between just being close to noise is more likely than anything else.)

How to control the theoretical plugin that can do the 'routing' is a whoooole other story though.

[monas]

@ Harry_HH, it's what i wrote. there's hardly a way around kyma if you want to use morphing with really good quality.

@ mandolarian, i never said that I think it sounds bad. it was a response to cron saying he doesn't like a specific technique.

[cron]

I think morphing is more a skill than an effect.

actually morphing is a well defined term in computer music as a kind of cross synthesis where spectral features of two sources are interpolated in the spectral domain. so it's most definitely an effect.

Exately, no matter if we are talking about audio or video.
According to "cron"'s philosophy, e.g. John Landis had to use the same technology in the "Black or White" morphing scene than one of the most famous early example of morhing trick when Maria was transformed to Robot in Friz Lang's Metropolis 65 years earlier. Althoug Metropolis has unbeatable charm, you have to admit that morphing has developed since. Of course you need "a skill" although you use technology no matter how advanced, but the same concerns all the effects (an other story is that today when we have all kind of audio effects available everywhere, the relative ability/skill to really use and understand tech has decreased) , eq, compression, reverb etc. You can achive quite a nice reverb by shouting in corridor, or you can animate the whole thing manually in stead of digital technology and get a very good result as e.g. Stephen R. Johnson did in the Peter Gabriel "Sledgehammer" (all these examples represent the state of art techology of its time), but the modern technology opens new possibilites every day. AND WE WAN'T THAT F**** MORPHING PLUGIN, you deaf developers! H.

Hey, I'd love a morphing plug-in too! I just don't think it's possible to do it on two audio streams in real time with no pre-processing. All your plug-in can do is work with 'the now', the current FFT frames or whatever. It can't see into the future or look at the sounds it's working with as a whole, which seriously limits its capability to morph between two sounds which are themelves in spectral motion. Most real-time morph applications live in that kind of 'grainy crossfade' territory, which I assume is the best one can do when you're having to work with data on a frame-by-frame basis.

Having said all this, I'm surprised nobody has yet ported one of the CSound opcodes (or similar) to VST. Something where you can just pick two soundfiles inside the plug and morph between them. There are so many fragments of computer music languages which have been ported to VST over the years I'm surprised that 'true' morphing hasn't had a look in given how alluring it is. We need to find a bored computer science student!

[monas]

@ cron

the opcodes don't really need to be ported. as i said above there's a program called Cabbage that lets you use any Csound code as vst, and soon even as AU. then you could use for example

http://www.csounds.com/manual/html/pvsmorph.html

here's the page

http://code.google.com/p/cabbage/

[cron]

@ cron

the opcodes don't really need to be ported. as i said above there's a program called Cabbage that lets you use any Csound code as vst, and soon even as AU. then you could use for example

http://www.csounds.com/manual/html/pvsmorph.html

here's the page

http://code.google.com/p/cabbage/

Oh, nice! I missed that. Thanks. Been threatening to learn CSound for almost a decade here. Maybe now is the time!

[Harry_HH]

@ Harry_HH, it's what i wrote. there's hardly a way around kyma if you want to use morphing with really good quality.

OK - very few know this kyma, and what is written, it's both expensive and rather complex to use, we don't wan't plugins like this. What we will have is the latest high tech morphing application with a great sound, versatility and innovative easy-to-use interface. For me the main function of this thread is to send a signal to all developers to work with this, all arguments which support this message are wellcome. Theory is (to me) interesting "an sich" but not very useful from this point of view (I leave it to the developers). H.

[Neon Breath]

For those who find that Kyma's morphs sound bad or uninteresting in the earlier example I posted, I suggest you to listen to these commercials created by the sound designer Pete Johnston, all featuring audio morphs with Kyma:

http://www.bantusound.com/SoundMorphing ... gPage.html

I haven't found any effects (or even soft synths known for performing some morphing like Alchemy) sounding as smooth and effective as these examples.

[whyterabbyt]

@ Harry_HH, it's what i wrote. there's hardly a way around kyma if you want to use morphing with really good quality.

OK - very few know this kyma, and what is written, it's both expensive and rather complex to use, we don't wan't plugins like this. What we will have is the latest high tech morphing application with a great sound, versatility and innovative easy-to-use interface. For me the main function of this thread is to send a signal to all developers to work with this, all arguments which support this message are wellcome. Theory is (to me) interesting "an sich" but not very useful from this point of view (I leave it to the developers). H.

actually i dont support this 'message'. "oi! you developers! I demand you make this thing I want" isnt really a productive direction for the thread, nor is it its main function.

[metalifuxx]

It's almost the same as trying to do that with photo or video. The sources need to already share some fairly similar traits or else it just does not achieve the effect your aiming for, or wont be convincing. Like trying to put a jagged square/rectangle through a similar sized round smooth circle/hole. Then morph them and trying say it is a a "squa-circle", with perfectly matched "offspring" traits. It's either going to look like one or the other, or just two separate identities. Only Conan O'brien's show can do that perfectly

I think the tools listed in this thread could probably do what your trying to do with some effort, and choosing the right two sound sources. But picking two randomly different sound sources and morph/blend them in the way you want, I don't think will work like you want it to or think it can be done with some magic plugin. The two sources being "morphed" in the Kyma examples had fairly similar traits, so the desired effect worked better. You might have to do some heavy pre-editing and pitch/time stretch matching between the two samples first, especially if they don't share any similar characteristics.

[Arglebargle]

It's weird, I have an old VST demo version of Prosoniq Morph that has never stopped working. Don't think I've ever used it for anything tho. I would definitely like a morphing plugin.

[Harry_HH]

Hey, I'd love a morphing plug-in too! I just don't think it's possible to do it on two audio streams in real time with no pre-processing. All your plug-in can do is work with 'the now', the current FFT frames or whatever. It can't see into the future or look at the sounds it's working with as a whole, which seriously limits its capability to morph between two sounds which are themelves in spectral motion. Most real-time morph applications live in that kind of 'grainy crossfade' territory, which I assume is the best one can do when you're having to work with data on a frame-by-frame basis.

Having said all this, I'm surprised nobody has yet ported one of the CSound opcodes (or similar) to VST. Something where you can just pick two soundfiles inside the plug and morph between them. There are so many fragments of computer music languages which have been ported to VST over the years I'm surprised that 'true' morphing hasn't had a look in given how alluring it is. We need to find a bored computer science student!

Well, to keep the thread alive (and increase possibility that some developer picks it up), two comments:

1) "It can't see into the future or look at the sounds..." Why not, this wouldn't be the first plugin which looks forward while operating (e.g. some limiters). And if we need more "forward looking" tool here, I trust that the propellerheads will find out new things to implement it, if they just worked with it. What they (and the finance/resource decision makers) seem to need now is the motivation.

2) I would be happy, as a first step, if we just got a plugin such a Prosoniq Morph for Windows updated. But at the moment we don't even have this one.

H.

[mandolarian]

For those who find that Kyma's morphs sound bad in the earlier example I posted, I suggest you to listen to these commercials created by the sound designer Pete Johnston, all featuring audio morphs with Kyma:

http://www.bantusound.com/SoundMorphing ... gPage.html

I haven't found any effects (or even soft synths known for performing some morphing like Alchemy) sounding as smooth and effective as these examples.

Maybe it's just shitty mp3 files, but not finding these samples particularly high quality. Radio quality, sure. High quality, not so much. Clever sound design, though.

[cron]

It's almost the same as trying to do that with photo or video. The sources need to already share some fairly similar traits or else it just does not achieve the effect your aiming for, or wont be convincing. Like trying to put a jagged square/rectangle through a similar sized round smooth circle/hole. Then morph them and trying say it is a a "squa-circle", with perfectly matched "offspring" traits. It's either going to look like one or the other, or just two separate identities. Only Conan O'brien's show can do that perfectly

I think the tools listed in this thread could probably do what your trying to do with some effort, and choosing the right two sound sources. But picking two randomly different sound sources and morph/blend them in the way you want, I don't think will work like you want it to or think it can be done with some magic plugin. The two sources being "morphed" in the Kyma examples had fairly similar traits, so the desired effect worked better. You might have to do some heavy pre-editing and pitch/time stretch matching between the two samples first, especially if they don't share any similar characteristics.

This.

One thing that's interesting about sound morphing is that it shouldn't be seamless if it's to work as a gesture. I mentioned earlier about morphing my voice into a gong. The main reason it sounded dull was simply that you couldn't hear the morph! It was too smooth. It's not really the morph that we're interested in so much as the sounds between. The morph can't be so seamless that it completely escapes the listeners attention. There has to be some conflict there to announce 'this is an effect' to the listener. Kyma's 'between morph' sound when dealing with sounds of differing pitches is that distinctive kind of 'frequency bending' granular scratch which doesn't really sound like either source. But, it seems this sound is what makes Kyma's morph so popular! It's far from seamless, and it's all the better for it.

Just a thought.

[Neon Breath]

This.

One thing that's interesting about sound morphing is that it shouldn't be seamless if it's to work as a gesture. I mentioned earlier about morphing my voice into a gong. The main reason it sounded dull was simply that you couldn't hear the morph! It was too smooth. It's not really the morph that we're interested in so much as the sounds between. The morph can't be so seamless that it completely escapes the listeners attention, but then it can't be so dissonant that it sticks out. Kyma's 'between morph' sound when dealing with sounds of differing pitches is that distinctive kind of 'frequency bending' granular scratch which doesn't really sound like either source. But, it seems this sound is what makes Kyma's morph so popular. It's far from seamless, and it's all the better for it.

Just a thought.

Pretty interesting thought. I think you are right, the sweet spot where things get sonically interesting is actually right in the middle point of the morph.