Integrating Spleeter into Melda crossovers

[vectorwarrior]

Spleeter is an audio separation system that allows you to separate the sound into various components based on an AI model.

It allows you to separate audio in various ways:
Vocals (singing voice) / accompaniment separation (2 stems)
Vocals / drums / bass / other separation (4 stems)
Vocals / drums / bass / piano / other separation (5 stems)

Having this built into the melda crossovers as a crossover type could be very interesting. For example, adding reverb to only the vocals of a track, adding transients to drums, compressing bass... all this could be done on a mixed track.

Acon Acoustica has just incorporated it (I imagine they use Spleeter) and Melda's crossovers would be an even better place to utilize this technology.

More information here:
https://github.com/deezer/spleeter

Thought it would be an interesting conversation topic, so wanted to post here.

[jmg8]

So is it like open source code that anyone can use for free?
Would be cool to have it, definitely interesting!
How do you think it would work in a crossover though? In RX and such it get separated into the elements but there is no blend between them. For example you can't apply an effect to half the drums and half bass. How would the crossover distribute the elements? Drums on the left etc? If you placed a crossover in the middle of the drums area, what would get separated?

[timbit2006]

I've done this exact thing you describe with adding effects to already mixed vocals before using RX7 and seperating out the tracks. I'm going to try Spleeter to see how it works in comparison to RX7's Music Rebalance. To get it fully clean I had to do a lot more with noise reduction and dereverb though.

[vectorwarrior]

How do you think it would work in a crossover though? In RX and such it get separated into the elements but there is no blend between them. For example you can't apply an effect to half the drums and half bass. How would the crossover distribute the elements? Drums on the left etc? If you placed a crossover in the middle of the drums area, what would get separated?

There are lots of ways this could be implemented, but yeah, splitting up into 'areas' of the crossover range and letting the band points determine where they take their sound from. The blending slope would work the same way the Spectrum crossover type works, with it's Level Slope setting.

In many ways this would work similarly to the RatioMulti module in MXXX. Where multiple inputs are put into what is effectively a crossover and the ratio control controls where in the range of inputs the output is sourced from.

[MeldaProduction]

I'd actually really like to get into more neural networking stuff, it's just there's no time and I'm simply too swamped with all the requests and, hopefully, the mixing revolution , which btw. won't use NNs probably

Btw. I also checked what it does, and I must say it's much better than I thought, but it still sucks... The resulting audio quality is horrible.