some thoughts about unmix and stems technologies

[Krakatau]

From the time I bought Spectralayers Pro, I was amazed of the progress made in the work of unmaking track

Not very accurate at many aspects (yet) the voice isolation though sounds like a real gem in much circonstances... and the additions of all stems always matches the initial audiophile accurately

my thought was about the potential techniques of resynthesise the lacking parts of the signals of many stems , even if their additions do not match the primary file anymore

the question would be how and at what degree would it be possible to recreate an instrument or another from a stem supposedly focused on a specific instrument/signal ?

[Lotuz2019]

the question would be how and at what degree would it be possible to recreate an instrument or another from a stem supposedly focused on a specific instrument/signal ?

Reminds me of a similar post.

Demucs in UVR5 in most cases is very usable to me to remove/isolate drums and bass, but it’s the ‘other instruments’ stem that bugs me a little. Is there a stem separation tool that allows you to input the name of a certain instrument? If in a track I can identify for example a Rhodes electric piano and I would like a stem of this single instrument, then I’d like to request the stem separation tool to focus on this instrument, compare it with a database of various Rhodes electric piano samples, filter out other instruments, and if this filtering degrades the Rhodes stem too much, use the database to repair the Rhodes stem.

[BertKoor]

If you invest some millions, I'm sure it can be done. A simple matter of training the model.
I mean, it's just a variation on the deepfake technology.
Problem is probably recouping the investment.

[Krakatau]

Demucs in UVR5 in most cases is very usable to me to remove/isolate drums and bass, but it’s the ‘other instruments’ stem that bugs me a little. Is there a stem separation tool that allows you to input the name of a certain instrument? If in a track I can identify for example a Rhodes electric piano and I would like a stem of this single instrument, then I’d like to request the stem separation tool to focus on this instrument, compare it with a database of various Rhodes electric piano samples, filter out other instruments, and if this filtering degrades the Rhodes stem too much, use the database to repair the Rhodes stem.

indeed...interesting !

[Krakatau]

If you invest some millions, I'm sure it can be done. A simple matter of training the model.
I mean, it's just a variation on the deepfake technology.
Problem is probably recouping the investment.

very intelligent IMO, ...I didn't though one minute !!!

[Shabdahbriah]

the question would be how and at what degree would it be possible to recreate an instrument or another from a stem supposedly focused on a specific instrument/signal ?

IF you know the "specific instrument", and have access to it, OR don't mind/care if it is that instrument. Convert the 'stem' to a MIDI file, and play that (or other) instrument with it. [2c]

[Krakatau]

the question would be how and at what degree would it be possible to recreate an instrument or another from a stem supposedly focused on a specific instrument/signal ?

IF you know the "specific instrument", and have access to it, OR don't mind/care if it is that instrument. Convert the 'stem' to a MIDI file, and play that (or other) instrument with it. [2c]

not a bad idea, but often the extracted instruments are lacunar for that task, unfortunately...

[kidslow]

I believe there is enough public information out there to figure out how to train your own UVR5 models, without needing millions. Likely thousands ... maybe of dollars and hours.