Our Community Plugin Format

[camsr]

The host would know about the plugin via the functions the plugin calls to request (and remember, shared) system resources

This description is too vague.

A simle example could be, if the plugin made a large allocation. Instead of just doing it, and letting the system figure it out, the host could confirm that it's readily available to make first.

What do you mean by "available"? Allocated by another plugin or host? It is probably used. If it is not used by others, why is it still allocated???

Too vague, okay. What if a plugin requested to allocate 24GB for all of it's sample data? Would you want to implement all the code in your plugin to make sure this doesnt crash/slowdown your DAW? Or let the host figure it out for the plugin?

It would be beneficial to make something to avoid "duplication of concerns". The plugins should be able to get some kind of data without having to have the same boilerplate code in every one of them. Another example is running the CPUID instruction, to check for supported instructions. Why should every plugin be concerned about runing this instruction, when the host can run it once, and relay the relevant information to the plugin? The results of CPUID do not change much, afaik.

[shabby]

I personally would love to see:
proper shader support - GL/DX or whatever for funky UI stuff.

The editor interface should be just that, an interface for controlling the plugin, whether to use HW accelerated drawing or support for certain technologies is not something that the plugin format itself should be concerned with.

Though on that note, it would be interesting to try do something about the window behaviour. In both vst2 and 3 the host passes a native system window handle (HWND, X11 Window, etc) to the editor. To integrate that with existing gui frameworks usually requires a lot of custom, platform specific and sometimes outright ugly code for embedding or re-parenting the window given by the gui framework. As most gui frameworks don't support that particular niche use case.

I don't have any good suggestions how to fix it though, particularly not without forcing a specific gui framework, which is not a what the plugin format is supposed to do.


(*) Full disclosure: I work for ELK and has build a lot of the plugin hosting code in ELK.

You make some interesting points there!

But I think it's absolutely essential to have hardware acceleration built-in. Modern Plugin's esp commercial ones basically have to have a super nice and fast UI - nobody wants windows 3.1 slider bars - it's part of the marketing strategy now.
Wasting cpu time rasterizing is unforgivable when the vast majority of systems have hardware acceleration. Cross platform hardware fragmentation is easily handled directly with GL or whatever other wrappers are out there.

I say all this because with my very short time using IPlug2 (one whole day so far ) - I think that it's a major step in the direction - using Nanosvg and whatever backend is super easy to whip up a modern UI very quickly. And having shader support would bring a hell of a lot to the table visually.

Being stuck with a cpu hogging, crap looking rasterizer - feels like being punished for unknown DSP crimes

[Youlean]

@shabby

I think you would be surprised to know how many plugins and hosts are using software rasterisation, and that doesn't mean that graphics performance is bad or it looks like win 3.1

Also, gpu rasterisation will use CPU too.

[nix808]

• GUI window should allow shapes other than rectangle/square

This is a joke, right?

It wasn't a joke. I think it would be neat
To apply a host menu bar would become impossible though
If the consensus is it's silly, then don't worry 'bout it

here's the software that gave me the idea->

playcenter2.gif

[syntonica]

• GUI window should allow shapes other than rectangle/square

This is a joke, right?

It wasn't a joke. I think it would be neat
To apply a host menu bar would become impossible though
If the consensus is it's silly, then don't worry 'bout it

here's the software that gave me the idea->
playcenter2.gif

It's still a square window without decoration or frame. The surroundy bits are just transparent. Usually, it's just a .png for the background.

[nix808]

The surroundy bits are just transparent. Usually, it's just a .png for the background.

do you know if VST2 or 3 could allow such? I would use it sometimes
Cheers!

[mystran]

The surroundy bits are just transparent. Usually, it's just a .png for the background.

do you know if VST2 or 3 could allow such? I would use it sometimes
Cheers!

You can certainly do it in any plugin (at least on Windows, never tried that kind of stuff on macOS), but you'll need to open a top-level window that isn't a regular plugin editor inside a host-window (because otherwise it's just the host window background that shows through). FL's Wave Candy works kinda like this, with just the settings in the regular plugin window and then the actual visualization in a separate popup with rounded corners and optional transparency with or without blurred background.

[shabby]

@shabby

I think you would be surprised to know how many plugins and hosts are using software rasterisation, and that doesn't mean that graphics performance is bad or it looks like win 3.1

Also, gpu rasterisation will use CPU too.

Your never going to get large fancy alpha blended 60fps UI's with software rasterization - without burning up some CPU . Think something along the lines of FabFilter - multiple alpha blended graphs and AA lines/edges - that would be an almost criminal waste of CPU time.

Unless a UI system and it's backend have API bit rot, GPU set up is minimal CPU time, all the various stages: constants,textures,combiners,shaders e.t.c. can be done very quickly without chewing up our DSP power

[mystran]

@shabby

I think you would be surprised to know how many plugins and hosts are using software rasterisation, and that doesn't mean that graphics performance is bad or it looks like win 3.1

Also, gpu rasterisation will use CPU too.

Your never going to get large fancy alpha blended 60fps UI's with software rasterization - without burning up some CPU . Think something along the lines of FabFilter - multiple alpha blended graphs and AA lines/edges - that would be an almost criminal waste of CPU time.

Actually it's not quite so simple. I'm not sure what they use right now, but I believe Skia at least used to (and unless they've done something new recently, it's probably what they still do) basically rasterize complex polygon paths on the CPU, then upload the resulting coverage mask as texture to the GPU for painting and blending, because as soon as you want better AA than what you get out of hardware MSAA (ie. you can't just stencil-and-cover), rasterizing on the CPU can actually end up the fastest approach, especially for the complicated cases.

The actual painting/blending is pretty cheap on either GPU or CPU though, so it's mostly a matter of optimizing texture bandwidth (ie. one channel coverage vs. 4-channel RGBA) if you're going to do the final composition on the GPU anyway. If you have a bunch of layers all rendered on the CPU, then it might well be faster to just blend them there as well and only upload the final image.

The thing to keep in mind though is that naive software rendering is quite slow. Even if your core rasterizer is fast, it still makes sense to keep track of things like dirty regions, tight clipping bounds and intelligent caching. It is quite easy to end up spending 90% of the rendering time just blending fully transparent regions and repainting stuff that didn't change from the last frame and this is obviously not competitive with anything.

ps. I should probably explicitly point out that I'm not trying to claim CPU is always faster. Rather I'm trying to point out that "it depends" and the answer is not always obvious, because some 2D rendering tasks are actually really easy on the CPU and really hard on the GPU and the CPU can end up being the bottleneck even in a seemingly GPU based solution.

[nix808]

thanks mystran!
I have asked for this capability in Flowstone. We do have child windows in a recent addition, so maybe it can come to be

I have noted that GPU is not always the solution for drawing
-perhaps for Windows, no barriers to GDI would be good

[shabby]

@shabby

I think you would be surprised to know how many plugins and hosts are using software rasterisation, and that doesn't mean that graphics performance is bad or it looks like win 3.1

Also, gpu rasterisation will use CPU too.

Your never going to get large fancy alpha blended 60fps UI's with software rasterization - without burning up some CPU . Think something along the lines of FabFilter - multiple alpha blended graphs and AA lines/edges - that would be an almost criminal waste of CPU time.

Actually it's not quite so simple. I'm not sure what they use right now, but I believe Skia at least used to (and unless they've done something new recently, it's probably what they still do) basically rasterize complex polygon paths on the CPU, then upload the resulting coverage mask as texture to the GPU for painting and blending, because as soon as you want better AA than what you get out of hardware MSAA (ie. you can't just stencil-and-cover), rasterizing on the CPU can actually end up the fastest approach, especially for the complicated cases.

The actual painting/blending is pretty cheap on either GPU or CPU though, so it's mostly a matter of optimizing texture bandwidth (ie. one channel coverage vs. 4-channel RGBA) if you're going to do the final composition on the GPU anyway. If you have a bunch of layers all rendered on the CPU, then it might well be faster to just blend them there as well and only upload the final image.

The thing to keep in mind though is that naive software rendering is quite slow. Even if your core rasterizer is fast, it still makes sense to keep track of things like dirty regions, tight clipping bounds and intelligent caching. It is quite easy to end up spending 90% of the rendering time just blending fully transparent regions and repainting stuff that didn't change from the last frame and this is obviously not competitive with anything.

ps. I should probably explicitly point out that I'm not trying to claim CPU is always faster. Rather I'm trying to point out that "it depends" and the answer is not always obvious, because some 2D rendering tasks are actually really easy on the CPU and really hard on the GPU and the CPU can end up being the bottleneck even in a seemingly GPU based solution.

Mystran that is super interesting info!!

For sure - piss poor implementations are always going to be slow on the GPU. And if that is still true on the current SKIA - that seems like a bonkers crazy way to do it

But you know - a high performance well thought out GPU system will always trounce a CPU solution. Just sounds like they couldn't be bothered to think things through and adopted a simple naive system


P.S. I just fixed my reaper vst3 init problem by switching IPlug2 to using SKIA - it's pretty damn cool.

[mystran]

For sure - piss poor implementations are always going to be slow on the GPU. And if that is still true on the current SKIA - that seems like a bonkers crazy way to do it

But you know - a high performance well thought out GPU system will always trounce a CPU solution. Just sounds like they couldn't be bothered to think things through and adopted a simple naive system

That is very much not the case. They are (or at least were) doing this because it literally has better performance (especially on low-end hardware) than any known GPU algorithm with comparable AA quality. There was also GPU paths for NV_path_rendering where available, but that's NV only (and while you can use manual stencil-and-cover without NV as well, it looks like garbage on hardware with low or no MSAA available).

[shabby]

For sure - piss poor implementations are always going to be slow on the GPU. And if that is still true on the current SKIA - that seems like a bonkers crazy way to do it

But you know - a high performance well thought out GPU system will always trounce a CPU solution. Just sounds like they couldn't be bothered to think things through and adopted a simple naive system

That is very much not the case. They are (or at least were) doing this because it literally has better performance (especially on low-end hardware) than any known GPU algorithm with comparable AA quality. There was also GPU paths for NV_path_rendering where available, but that's NV only (and while you can use manual stencil-and-cover without NV as well, it looks like garbage on hardware with low or no MSAA available).

Mystran - I have personally implemented a super cool path system a while back - it uses the outside point stencil trick - combined with a SDF edge system - it's pretty damn awesome!! I wondering why they didn't do that - it's a fairly obvious way to do it.

[Youlean]

Even with CPU rasterisation you are not wasting dsp performance, since gui thread is running in lower priority.

It would be nice if GPU rendering is always faster, but this is not the case in the real world, at least if you use popular 2D graphics libs.

For example on my Windows system, I have tested in my plugin software rasterisation using Cairo and GPU rasterisation using nanoVG.

Cairo uses 5% of one CPU core
nanoVG uses 15% of one core and 10% of GPU

The graphics is the same BTW in both cases.
Of course, if you scale up the GUI a lot, nanoVG will become faster since most of the stuff will be solved by GPU.

[syntonica]

That was one reason I gave up on the NanoVG/OpenGL version of my GUI. It still pretty much used the same CPU. I was able to make optimizations that cut the GUI CPU time to less than half of what it was using native Cocoa drawing. The other was that Apple announced they were deprecating OpenGL.

Unfortunately, I've discovered that in Catalina, Apple changed their clipping and now I can't take the optimizations anymore without using their crappy list of dirty rects... I've tried just using a flag, and while it works, it still uses the same CPU as when I redraw the whole GUI 60x a second. Thanks Apple. Why don't you go do something useful, like put a Metal back end on your drawing and let me worry about if I'm coloring outside the lines or not!