On variable buffer sizes in vst 2 and 3 and other host idiosynchrasies

[Gregorius]

I have not experienced any performance issues in FL. Same business as Studio One or Reaper. I could not say my Tone2 Gladiator runs different or I can load less instances before CPU is too hot. I did experience though that automation (if used) was more accurate (especially noticable with some tapestop effects) and I think FL only uses that 1 sample stuff, if they see a reason for it like automation, live MIDI input or such. I've seen plugins crashing which could just not handle variable buffersizes, but no spiking (it was a waves plugin and I think an early version of nexus). So whatever the fuzz is about, as a user I see no difference and because almost all of my used plugins work fine and performant with FL, I'd rather sort out the ones that introduce problems - no matter if it's subjectivly FL's fault or the devs.

And I have to say for some guys with soooooo much experience in the biz, there's often much crying from only the same few ones over and over again.

[Markus Krause]

@evildragon:
Please note that all our plugins can handle a variable block size. FL Studio (and only this host) still causes crackles on some of our plugins. The reason is the CPU load of the context-switch and not that we do not support a variable block-size.

Please note that your suggestion would introduce some additional latency for *all* users in all other hosts just to fix something that should be fixed just within FL Studio.
The only workaround would be to read out the Host ID. Then check if it is FL Studio. If FL Studio is used an additional latency is used to workaround the CPU-load-jitter/crackle-problem of FL Studio.

[BlueprintInc]

@evildragon:
No. I will not do this because I think it is a bad idea . This would introduce additional latency for *all* users just to fix something that should be fixed within FL Studio

...speechless. We're talking about the most basic principles in audio industry...

[matt42]

Please read my previous post properly. I have given a detailed technical explanation why FL Studio should be fixed. I am coding audio software since 1995 and really know how computers work.

Keep in mind that nearly every VST plugin runs on more than 2 threads. Audio and a second idle thread for GUI.

I did read your post correctly, but I'm not sure you read mine with the same care. When I say multi thread processing in a dsp forum I think it's clear we are talking about the audio thread. I have to wonder if you are being deliberately condescending.

I don't think we will have much progress discussing this here, but I coded a plugin that required more than 4 audio threads (multiple channels of zero delay convolution, each channel requiring multiple threads) running glitch free, so it is possible.

[matt42]

No thread needs to wait for the other.

So what happens when you get to the end of the main process function and the other threads haven't completed yet?

[EvilDragon]

Please note that your suggestion would introduce some additional latency for *all* users in all other hosts just to fix something that should be fixed just within FL Studio.

u-he sorts this out with a global option per plugin - enable base latency of 16 samples or not. PDC is adequately reported or not.

Not exactly rocket science!

[camsr]

4) It can occasionally happen that the plugin with internal block-processing needs to processes a block (like 64 samples) while FL Studio just needs 1 samples. Things get even worse when several instances are running. I want to point out here that internal block-processing for CPU optimization is a common practice. Hardware does it, we do it, U-he does it, all FFT plugins do it, many synthesizers do it.

This isn't so much of a problem rather an inconveinence, as in this case the effect plugin would just buffer the sample it receives, and defer the work until the input buffer is full. A generator plugin with only an output buffer, may or may not process the amount of samples needed at that time to remain current. The issue is still speed, as any deffered audio processing must complete in less time (assuming this is done to avoid any block-related overheads). This is going to sound crazy, because it goes against our ideal of latency, but having larger host or driver buffers would help smooth out these spurious workloads and work to minimize peak CPU load. Even if the host is tossing random buffer sizes at the plugins.

[matt42]

Multicore CPUs exists since two decades now. And you're saying that plugin-developers should not support it or rewrite their software , just because FL Studio isn't working properly?

Please don't put words in my mouth.

In general the host has a bigger overview of what is going on. A host will often have multiple parallel channels requiring processing, so in general, it makes sense for the host to handle multi threading.

I say in general, because the plugin may make use of a technique that is basically only implementable with multi threading, or, perhaps, the plugin is so CPU intensive that a single core struggles to keep up. In most other cases it doesn't make sense to multi thread the audio engine on the plugins side.

[mike_the_ranger]

From a user perspective, I don't see where the problem is. I can set my FL buffer size incredibly low knowing the CPU hits high, or set it big knowing the CPU has more time. I select whatever fits my needs. The same thing is true for FL's fixed/variable buffer size. As a user I can set it to whatever I want. Why is that a problem for you? *If* I think there's anything wrong I can just disable it. If I want perfect automation, I leave it enabled and enjoy its advantages. I btw never had any issues in that regard with any plugin in FL...

[mystran]

I say in general, because the plugin may make use of a technique that is basically only implementable with multi threading, or, perhaps, the plugin is so CPU intensive that a single core struggles to keep up. In most other cases it doesn't make sense to multi thread the audio engine on the plugins side.

It would be really really nice if plugin APIs had some standard way to submit a set of jobs in bulk to be processed by the host's threadpool. This way the host could still process all the jobs from a given plugin together and rather than having multiple plugins and the host itself compete for the cores. I don't think context-switching is a huge deal as such (the time-slices involved are short enough that nothing is likely to get pre-empted), but this would allow some plugins to complete faster and release dependencies further down the audio graph, which would potentially improve the overall parallelism (same reason multi-threading a CPU hungry synth makes sense even when a single-core could theoretically keep up with it).

[camsr]

Multicore CPUs exists since two decades now. And you're saying that plugin-developers should not support it or rewrite their software , just because FL Studio isn't working properly?

Please don't put words in my mouth.

In general the host has a bigger overview of what is going on. A host will often have multiple parallel channels requiring processing, so in general, it makes sense for the host to handle multi threading.

I say in general, because the plugin may make use of a technique that is basically only implementable with multi threading, or, perhaps, the plugin is so CPU intensive that a single core struggles to keep up. In most other cases it doesn't make sense to multi thread the audio engine on the plugins side.

The host could dispatch/signal threads, but it's the OS task to execute them in a timely manner, and not the host. This is a huge issue on a modern OS, that is trying to simulate a RTOS.

[matt42]

Edit: Never mind

[mystran]

The host could dispatch/signal threads, but it's the OS task to execute them in a timely manner, and not the host. This is a huge issue on a modern OS, that is trying to simulate a RTOS.

No. It's not a huge issue on a modern OS. There are conceptual problems with priority scheduling, but it generally works as intended to good enough approximation that you don't need to worry about it.

The big problem with both host and plugins multi-threading at the same time is not a thread-scheduling problem, but rather a work-scheduling problem. The difference is important.

When you are trying to process a complete audio-graph (which we assume is a DAG) where multiple nodes can be processed in parallel, but no node can be started before all of it's dependencies have been processed, it makes sense for the host to try to give priority to longer dependency chains (this is typically the bottleneck for meeting a real-time deadline) and possibly nodes that are input dependencies to a lot of other nodes (to try and improve parallelism down the line).

Suppose for example that we have 2 cores and two plugins A and B. Plugin A can use multi-threading internally. The output of plugin A is sent to plugin C. The output of plugins B and C are mixed to master.

Now, if the host starts processing plugin A and plugin A fires up it's two threads for the two cores, everything is great. When it completes, the host can put plugin B on one thread and plugin C on another thread. Everything is parallel, the joy. However what often happens in reality is that the host will try to process both plugin A and B. Now we might end up with B taking one of the cores, so that A can only process with one core instead of two, which means that A takes roughly twice as long (in wall-clock time) to complete. This in turn means that it also takes twice as long (in wall-clock time) before we can start processing plugin C. We'll have the result of B earlier, but we can't do anything with it before C is also done. This isn't as bad as not multi-threading at all (and that's also important: we can't in general do better by threading less), but it's still far from ideal.

The problem here is not thread scheduling. The problem is that a plugin (A in this case) has no way to tell the host that it has two sub-tasks that can be done in parallel, but must both be completed before the plugin can finish. In the above example, if the host had that information, it could avoid trying to process plugin B until plugin A's sub-tasks are done. But I want to repeat for emphasis: you can't fix this by multi-threading less; the only way to fix is it multi-thread more intelligently, which requires support into the plugin APIs.

[camsr]

It seems vague because a sub-task could impart anything on behalf of the plugin. I think a more strict definition is required.

In comparison to the current methods in use, which calls a subroutine and executes in the context of the calling thread, seems to give the plugin the decision.

[camsr]

Within the Directed Acyclic Graph (had to look that up), there are differences in block sizes being fed into plugins. But the dependency node should always be the largest and latest minimum buffer?? Like, if one plugin down the chain required 256 samples fixed input buffer, it's dependencies could be given multiple smaller blocks to process, for whatever reasons.