No. That's not it. I'm not talking about complex stuff like that. I'm simply talking about basic audio mixing.yellowmix wrote: ↑Sat May 14, 2022 11:09 pmThe technology you're discussing is called audio source separation. It's nowhere near the level of being able to do this in real time, nevermind modelling multiple analog filters on top. You can see the current best in Celemony Melodyne's ability to separate individual notes, as well as Zynaptiq Pitchmap and Hit n Mix RipX (or whatever they're calling it these days). If you want to give it a spin you should look into those, but the results probably won't be what you desire.bmanic wrote: ↑Sat May 14, 2022 10:46 pm Mind you, this wouldn't be easy to pull off. Splitting audio smoothly into separate streams is not simple at all. Also, how would it be done with chords where MIDI notes are slightly out of sync? Probably the plugin needs to have built in input quantize and that way have a better chance of splitting the stream in a useful manner.
If somebody would figure this out, it'd allow us users to put this filter effect on any of our favorite synthesizers and still have it work sort of polyphonically, meaning each midi note would have it's own filter envelope being triggered and released upon note-off messages.
Example:
1) Have a basic VSTi that provides a saw wave. Both VSTi and the VST FX filter plugin receives the same MIDI information.
2) Press MIDI note 1 -> saw goes into filter VST FX plugin
3) Press MIDI note 2 -> filter VST FX plugin does a super quick crossfade while sending first audio that came through to "audio stream 1" and the newly generated saw wave from the VSTi to "audio stream 2"
4) Let go of the first note that you pressed -> saw stops playing and decays out with the VSTi release envelope -> VST FX filter also now goes into it's release stage on any envelope or other modulation sources.
5) .. the 2nd note is still in it's earlier phases of modulation as the note is still held in.
.. etc. This does require a tiny bit of look-a-head for the FX plugin to do the splitting and cross-fading into new audio streams possible without clicks but it should be possible to create.
But like I said earlier, it can't of course deal with chords. However that may still end up like a really interesting effect. Especially if the VST FX filter has a built in input quantize, that helps you separate each chord newly played into a new audio stream. This way you could have polyphonic everything even with chords (where each chord is their own "voice"). Naturally you can't separate the actual notes in each chord but at least you could separate the chords themselves.
That would be an offline process. What I'm talking about is almost real-time. It would also open up the door to other much more creative effects in a polyphonic manner. Imagine being able to add different delays, phasers, flangers, granular, bit or samplerate reduction stuff, per incoming MIDI note, for each of your synth's voices.
It's territory that hasn't ever been explored before. Heck, if I get any spare time this summer I may take a crack at this. Should be possible to setup in something like Reaktor.
