Well this is a kick in the nuts: VST2 plug-ins

[mr.ardour]

although strictly speaking knowing the audio latency is not enough, because you also need to account for things like host render-ahead, hence the wall-clock delta.

I probably shouldn't say this, because the hosts I work(ed) on never ever do "render-ahead", but from a conceptual point of view, I can't see any difference at all between "other latency" and that caused by render-ahead. That is, if the host provides information about output latency, it ought to include any latency caused by render-ahead.

[Blue Cat Audio]

Exactly, if you don't separate your domains you will have this issue you describe. The host can solve this conflicts because it knows if a parameter is automated or if a parameter is currently controlled via a remote device.

Nope, my point is that it cannot solve it (properly). It only works for "read only playback". As soon as the audio engineer is interacting with the DAW or plug-ins in any way (which is 99% of the time), roundtrip latency is here, and conflicts are even more obvious if displays are not in sync with user interactions. So IMHO it's not a solution and does not justify GUI/DSP separation in any way.

[arne]

Exactly, if you don't separate your domains you will have this issue you describe. The host can solve this conflicts because it knows if a parameter is automated or if a parameter is currently controlled via a remote device.

Nope, my point is that it cannot solve it (properly). It only works for "read only playback". As soon as the audio engineer is interacting with the DAW or plug-ins in any way (which is 99% of the time), roundtrip latency is here, and conflicts are even more obvious if displays are not in sync with user interactions. So IMHO it's not a solution and does not justify GUI/DSP separation in any way.

I know what you wanted to point out, but it is solved in a way an engineer can work without being irritated by this issue. You just need a proper host and plug-ins that support it.

Just imagine a volume slider where automation was written in the host. Now the mix engineer listens to the mix and hears something he don't like and looks at the screen and sees the volume automation and now wants to interact with it. At the moment the slider is touched by the engineer the host will stop sending the automation data to the plug-in and will use the movement of the engineer. You're right that this is now not in sync anymore, but the engineer knows this and use the ear and not the display to alter the sound and compensate for the latency itself.

[Urs]

Let me ask. The example of 200ms latency. Does that mean something like 8192 samples per render call? Does that mean 5 FPS for the UI update?

[arne]

Let me ask. The example of 200ms latency. Does that mean something like 8192 samples per render call? Does that mean 5 FPS for the UI update?

No. The number of samples per render call has nothing to do here. It can be as low as possible with your audio hardware.

[Urs]

Ok. So it's because of PDC because lookahead or something.

We've never been confronted by any user with this issue, but if there were any, I'd prefer to implement a delay for visual feedback using the following information:

https://developer.steinberg.help/displa ... cy+Support

... and not be bothered by an API where the UI can't access the DSP data in a timely manner. I mean, how does the host distinguish between requests for timing critical data and completely unrelated kinds of visual feedback? If every request for data is delayed due to PDC... then sample/wavetable editing, programmatic tooltips, meta info etc. are just really, really difficult to visualise smoothly. You know, you do something with the mouse and stuff moves a quarter second behind?

[arne]

I'm not specifically talking VST3 here and I won't. All plug-in formats have this information available. I'm just arguing that separation of DSP and UI controller is a good and professional thing for one reason I outlined above (there are other good reasons outlined in many programming books about model view controller). There's a reason why there is support for this stuff in many plug-in formats like LV2, AAX, VST3, RackExtension (there you are forced to do it, I love it). GMPI had it too if I remember correctly. Because handling this is not easy and the host can at least provide help for handling ordinary parameters. Sure to handle wavetable or other bigger memory blobs it would be nice if the API would help more.

[Urs]

I have corrected myself earlier in this thread in that I do not mind the separation at all as design choice and a good pattern. My criticism is all about enforcing this with an asynchronous protocol for complex data exchange, which creates more problems IMHO than it solves.

[S0lo]

It didn't come up here, so I will give you an idea why separation of DSP and UI controller is something natural. You all heard about plug-in delay compensation. If you think about how it is done for a moment you should see that this is done by running the audio "earlier" thru the DSP than you hear it. Just place a plug-in with a latency of 200ms into the last position of the chain. All plug-ins before it will process audio more than 200ms before it goes thru the speakers. Now if you don't separate your logic your UI will show changes from automation or DSP analysis data 200ms too early. You can think of it as separate time domains you should treat differently.

If you put a VU meter/analyzer before that 200ms plug-in you'll see the display roughly 200ms before it reaches the speaker. And if you put the VU meter/analyzer after the 200ms plug-in you'll see the display roughly at the time it reaches the speaker. Why is this a problem? It's merely a side effect of plugin placement.

Audio engineers of today are aware of plug-in latency, They can calculate it. If the engineer deliberately puts a VU/ana at a certain place it could really be his intention to see it at that time. One might even say that its wrong to show it at another time.

[arne]

I have corrected myself earlier in this thread in that I do not mind the separation at all as design choice and a good pattern. My criticism is all about enforcing this with an asynchronous protocol for complex data exchange, which creates more problems IMHO than it solves.

OK, fair point. But most people in this thread are against this separation, because in VST2 they used the same float storage for their DSP and their UI. What is really, really bad design. VST2 also does not enforce to code this way, but people did it anyway.

[Dewdman42]

I can definitely see some advantages of separating the UI/DSP, but I also think there are many plugin cases where it should not have to be forced to that more complicated method. There are lots of plugins that don't even use any fancy meters or anything that will care about that kind of GUI timing issue, the desire will be for the knobs to move correctly when the user moves the knobs...and there is nothing that can be done about the fact that the sound will respond later due to latency. Those simple plugins...and there are many of them out there...don't need a fancy Data/View architecture....and I would argue that if that had always been the requirement of VST2, many of those plugins would never have even come into being because many plugins are produced by small time developers which will be easily discouraged by such imposed data/view requirements.. That is not even getting into the question of how to synchronize the data between DSP and UI...which complicates it even further. And now that you mention the timing issue between UI and DSP in terms of meters need to be handled different timing then knob turning, etc...that further complicates everything for anyone wanting to make a simple plugin..that is a lot of stuff to digest.

But... I can also see how certain plugins could unquestionably benefit, particularly if the timing disparities were somehow handled automatically by the framework. If they Aren't handled automatically by the framework though, then it just needless introduces complexity that a lot of people will be ignoring anyway in their simple plugins.

[FigBug]

OK, fair point. But most people in this thread are against this separation, because in VST2 they used the same float storage for their DSP and their UI. What is really, really bad design. VST2 also does not enforce to code this way, but people did it anyway.

I'm still not convinced this is really, really bad. Storing the same value in two locations adds additional complexity. Adds the possibility of bugs in updating the parameter in the wrong direction, or not at all. Requires a communication protocol between the UI and DSP. And for what benefit? What is being gained from this additional complexity?

[Blue Cat Audio]

OK, fair point. But most people in this thread are against this separation, because in VST2 they used the same float storage for their DSP and their UI. What is really, really bad design. VST2 also does not enforce to code this way, but people did it anyway.

No, I don't think most people are against this separation. It's just that enforcing this with the API does not make sense, and there are indeed cases where this separation IS a problem, no matter how the plugin is designed internally.

Most audio plugins crashes are NOT due to GUI/DSP coupling or "bad design choices". In most cases, it is caused by a different understanding of the protocol (whatever it is) by the plugin and the host, or simply concurrency issues when the threading model of the API is not clear.

I personally don't care how the plugins that we host are designed internally, as long as they are stable and efficient. Same for apps that host our plugins. But I do care that the protocol to communicate between them is well designed: clear and simple yet flexible enough so that anyone can quickly write code that works and does not feel limited by what can be done with it.

[mystran]

OK, fair point. But most people in this thread are against this separation, because in VST2 they used the same float storage for their DSP and their UI. What is really, really bad design. VST2 also does not enforce to code this way, but people did it anyway.

I'm still not convinced this is really, really bad. Storing the same value in two locations adds additional complexity. Adds the possibility of bugs in updating the parameter in the wrong direction, or not at all. Requires a communication protocol between the UI and DSP. And for what benefit? What is being gained from this additional complexity?

It's really really bad. It's one of the more reliable ways to end up with Heisenbugs.

The bare minimum you should always do is make a copy of every parameter value at the beginning of your process() call into DSP private storage. Either use C++ atomics or insert a fence so the compiler is actually forced to do the copies properly and then only use those local variables for processing. On the GUI side you need to do something similar, so essentially you want at least three copies.

Otherwise you risk all kinds of weird bugs like a filter blowing up or an array access segfaulting, because you accidentally wrote something like an interpolation function that implicitly assumes the GUI doesn't change one of the values in the middle of a loop. If you're really unlucky, your compiler might initially optimize the thread-unsafe stuff into safe code. Then at some point you update the compiler and the new version does slightly different optimizations and now you have a bug that happens so rarely that by the time people start complaining you don't even remember the compiler updated.

That said, generally the better design is to use some sort of thread-safe message queues. It's much harder the shoot yourself in the foot with a known good message queue and these can be made wait-free/lock-free so there's no issue with real-time either. I would recommend against using any host-provided messaging mechanism though, because plenty of hosts have a long track record of not being able to do anything correctly.

[camsr]

I agree, the host should not provide that functionality. The host should also be a client of a messaging protocol that sits between host and plugin. This decouples both changes in any future protocol updates. However the execution context of this protocol would probably be inside the host process.