Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

DSP, Plug-in and Host development discussion.
mikejm
KVRer
26 posts since 5 Apr, 2017

Post Mon Jun 25, 2018 9:04 pm

I have been building my own synths in Reaktor, and I have reached a point where the synths I wanted to make are basically done.

However, I do a lot of modal synthesis. This is where every partial of a sound is created additively on an individual basis, generally each with a sine wave or resonant bandpass filter. This gets very CPU intensive. This wouldn't be a problem with modern processors, but Reaktor doesn't support multicore. I am currently maxing out a 4.4 Ghz i7 core on my ensembles, and wanting more. So it appears I have outgrown Reaktor for what I want.

I am trying to figure out the next best environment for me.

From what I can see, the best options might be Flowstone or Synthedit. As I understand, Flowstone and Synthedit can both make VSTs directly. Both also support multicore. Max might work too but does not make VSTs as easily.

My basic needs would be the following:

- Library of basic tools: simple sine wave oscillator, LFO, filters (BP/LP/HP), linear smoother, simple non-interpolating delay (rounded to nearest sample), AHDSR envelope.
- A way to build and link "macros" so I can keep my projects organized (eg. I generally build one polyphonic partial, then copy and paste it and link them up to as many partials as needed, with only "P#" differing between them).
- A way to perform simple math easily. Most of my projects work via multiplication/division/exponents/logarithms from the partial #, Note Pitch/Vel, and a knob input to set each parameter.

Flowstone or Synthedit?

Any thoughts or suggestions?

Thanks

mikejm
KVRer
26 posts since 5 Apr, 2017

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Mon Jun 25, 2018 11:22 pm

Actually, you know what. I downloaded Flowstone and Synthedit demos and it looks like they'd both be a huge pain in the ass to do what I want.

Seems like since I already have the synth design down and it's working, it would make the most sense to learn Juce and then just re-write it in that.

Should be a pain in the ass too, but at least I'd have total control and could probably better optimize things.

eg. It would probably be easier in Juce to just write one partial and then just specify within in a partial # "n" and set it to run "n" copies of that partial (n=1,2,3,4 ...) to get the full 100 partials or so I might want per note.

User avatar
Delta Sign
KVRian
519 posts since 22 Jun, 2018

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Mon Jun 25, 2018 11:47 pm

Pure Data would be another option, it is crazy CPU efficient. I can run pretty complex things even on this dying old machine. However, It's basically a self contained environment. I think there are ways to make plugins out of PD patches, but I never looked into it.

But if you want to take the next step and use Juce, then do it! Lots of people prototype their stuff in a visual real-time environment first (mostly in PD), and then port it to "actual" software/hardware.

Good luck!

Miles1981
KVRian
1361 posts since 26 Apr, 2004 from UK

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 6:19 am

You will still have an issue if you put these instruments in a host, as the host assumes that plugins are mono threaded (when they are not, it may work up to a point).

Xenakios
KVRian
1156 posts since 9 Sep, 2005 from Oulu, Finland

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 6:32 am

Miles1981 wrote:You will still have an issue if you put these instruments in a host, as the host assumes that plugins are mono threaded (when they are not, it may work up to a point).
The host has no means to enforce that, though. Using multithreading in a plugin may or may not work, it just needs to be tested if it's a good approach.

User avatar
Aleksey Vaneev
KVRAF
3529 posts since 7 Sep, 2002

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 7:00 am

Xenakios wrote:
Miles1981 wrote:You will still have an issue if you put these instruments in a host, as the host assumes that plugins are mono threaded (when they are not, it may work up to a point).
The host has no means to enforce that, though. Using multithreading in a plugin may or may not work, it just needs to be tested if it's a good approach.
Most probably it will work, we use threads inside main audio thread to build FIR filter kernels in CurveEQ, no crash reports so far, it works on both Windows and Mac.
Image

User avatar
Delta Sign
KVRian
519 posts since 22 Jun, 2018

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 7:11 am

Yeah, most U-He synths can distribute voices between cores, and I think there are a bunch of other synths as well. It's definitely possible.

mystran
KVRAF
5001 posts since 12 Feb, 2006 from Helsinki, Finland

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 7:30 am

Aleksey Vaneev wrote:
Xenakios wrote:
Miles1981 wrote:You will still have an issue if you put these instruments in a host, as the host assumes that plugins are mono threaded (when they are not, it may work up to a point).
The host has no means to enforce that, though. Using multithreading in a plugin may or may not work, it just needs to be tested if it's a good approach.
Most probably it will work, we use threads inside main audio thread to build FIR filter kernels in CurveEQ, no crash reports so far, it works on both Windows and Mac.
Why would it crash?

The potential problem is that if we have N cores and host is trying to run N threads in parallel, each processing a different plugin and each plugin decides to then run another N threads of it's own, you could end up with N^2 threads trying to compete for only N cores. This will still "work" just fine and if we're running short blocks anyway the threads probably don't even eat their time quanta, so probably won't get much excess pre-emption either, but that's a possibility.

The real question though is whether it slows down or speeds up the processing in the real-time sense, because audio being partially a serial process and often close to a tree rooted at the final master output, you'd prefer to get down the dependency chain as fast as possible to leave more time for the late serial processing. If a threading plugin manages to free up dependencies faster then that's a win. If one plugin threading causes another plugin to process longer so it blocks it's dependencies for longer then that's a potential loss.

My gut instinct says that it's almost always a win for CPU intensive plugins to multi-thread, but if you expect multiple parallel instances of the plugin, then it makes sense to share a thread-pool and commit all the jobs from a particular instances to be completed as an atomic block, so that at least one of them completes as fast as possible. I suppose one could try to analyse this more exactly too, but in order to do so one would need to know the exact logic that the host uses for threading.. which is usually not available. :)

What you do not want to do (like for real) in a plugin is try to be clever and pin your threads to specific CPUs, because this will significantly increase the probability that two multi-threading plugins (or a plugin and the host or whatever) will end up slowing each other down mutually (in the real-time sense!), because the specific CPU core they want happens to be in use... but as long as every thread is eligible to run on every CPU one would usually hope for the OS scheduler to be able to sort it out.
If you'd like Signaldust to return, please ask Katinka Tuisku to resign.

mystran
KVRAF
5001 posts since 12 Feb, 2006 from Helsinki, Finland

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 8:02 am

The fun of multi-threading scheduling with regards to "late dependencies" looks like this:

Suppose we have 2 cores and 4 jobs (A,B,C,D). Now job A depends on job D, while A, B, C feed to master.

If we schedule B+C on the 2 cores first, we then have to process D and A in serial, taking "3 units of time" total. If we schedule C+D first, then we can run A+B at the same time and only need "2 units of time" total.

Obviously in real-life we don't know how long each of the jobs takes so doing "optimal" scheduling is impossible, but the point is, in a multi-threaded scenario the job completion order can make or break a schedule.

edit: oh ... and plugin's multi-threading internally can either help or hurt this, more or less at random.. but if a plugin is CPU intensive enough that it's processing time is significant portion of the processing time of the whole audio graph, then multi-threading is likely to always be a win :)
If you'd like Signaldust to return, please ask Katinka Tuisku to resign.

Miles1981
KVRian
1361 posts since 26 Apr, 2004 from UK

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 8:56 am

Xenakios wrote:
Miles1981 wrote:You will still have an issue if you put these instruments in a host, as the host assumes that plugins are mono threaded (when they are not, it may work up to a point).
The host has no means to enforce that, though. Using multithreading in a plugin may or may not work, it just needs to be tested if it's a good approach.
IT works, but it's a bad pratice because of resource contention and some hosts forcing affinity for subsequent threads to be on the original processing thread affinity, which could be different when doing processing.

Miles1981
KVRian
1361 posts since 26 Apr, 2004 from UK

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 8:59 am

Delta Sign wrote:Yeah, most U-He synths can distribute voices between cores, and I think there are a bunch of other synths as well. It's definitely possible.
It's possible, but it's still a bad practice.
Unfortunately, the issue is that you only get one thread for processing instead of sharing a thread pool. Plugin API deficiency.

Miles1981
KVRian
1361 posts since 26 Apr, 2004 from UK

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 9:01 am

Miles1981 wrote:
Xenakios wrote:
Miles1981 wrote:You will still have an issue if you put these instruments in a host, as the host assumes that plugins are mono threaded (when they are not, it may work up to a point).
The host has no means to enforce that, though. Using multithreading in a plugin may or may not work, it just needs to be tested if it's a good approach.
IT works, but it's a bad pratice because of resource contention and some hosts forcing affinity for subsequent threads to be on the original processing thread affinity, which could be different when doing processing.
Oh, BTW that host is the one with a fruit as its publisher.

mikejm
KVRer
26 posts since 5 Apr, 2017

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 11:50 am

Delta Sign wrote:Yeah, most U-He synths can distribute voices between cores, and I think there are a bunch of other synths as well. It's definitely possible.
This is probably the most realistic way I can run my synths that would help the current CPU issue, unless perhaps just by nature of coding it in Juce it becomes automatically 40% more CPU efficient.

Currently I can get up to around 50 partials of 6 Polyphony before my 4.4 GHz core maxes out. I would like upwards of 100 partials.

Letting it split the polyphony among the different cores would be the simplest way to ideally resolve this.

In principle, what I would need is:

- General stage to process the NotePitch/Gate data with the knobs and calculate general "upstream" coefficients/parameters from that (must be done in one process)
- Individual downstream audio streams for each of the polyphonic voices based on the upstream data (can be split among up to 6 cores for 6 voices).
- Recombine audio streams (mix back together) and apply some monophonic processing (eg. distortion/delay/etc)

This should completely solve my CPU woes irrespective of whether Juce is intrinsically more efficient, and I think it is doable.

I've been looking at this Vanilla Juce project which I think I can learn the basic principles from:
https://forum.juce.com/t/new-simple-yet ... izer/23984

Should be painful at the start but hopefully worth it long term. If I already have all the design/signal flow/math done in Reaktor, all I need to do is replicate it here. So I presume that shouldn't be too bad, right?

Thanks guys.

mystran
KVRAF
5001 posts since 12 Feb, 2006 from Helsinki, Finland

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 12:13 pm

Miles1981 wrote:
Miles1981 wrote:
Xenakios wrote:
Miles1981 wrote:You will still have an issue if you put these instruments in a host, as the host assumes that plugins are mono threaded (when they are not, it may work up to a point).
The host has no means to enforce that, though. Using multithreading in a plugin may or may not work, it just needs to be tested if it's a good approach.
IT works, but it's a bad pratice because of resource contention and some hosts forcing affinity for subsequent threads to be on the original processing thread affinity, which could be different when doing processing.
Oh, BTW that host is the one with a fruit as its publisher.
Urgh.. can you maybe explain something about this "host forcing" thingie?

I mean... if I create a thread pool (ie. a bunch of threads and a queue) in the plugin (eg. either when the dynamic library is loaded or first time a plugin is instantiated.. obviously one shouldn't create threads in processing methods) and put some jobs in the queue for them to process, how is the host even supposed to know which threads are processing which plugin when the actual host thread just sits on a generic semaphore?
If you'd like Signaldust to return, please ask Katinka Tuisku to resign.

User avatar
Delta Sign
KVRian
519 posts since 22 Jun, 2018

Re: Reaktor can't handle my big projects anymore (no multicore). Good next option? Flowstone? Synthedit?

Post Tue Jun 26, 2018 12:24 pm

mikejm wrote:
Delta Sign wrote:Yeah, most U-He synths can distribute voices between cores, and I think there are a bunch of other synths as well. It's definitely possible.
This is probably the most realistic way I can run my synths that would help the current CPU issue, unless perhaps just by nature of coding it in Juce it becomes automatically 40% more CPU efficient.
That's most likely going to happen. Reaktor is terribly CPU inefficient in my experience.

You'll still have to think about optimizing things, of course.

Return to “DSP and Plug-in Development”