I want to start adding a custom functionality to my plugin ideas, but I am not sure where to invest my time in development of the GUI.

I would ultimately prefer to use vector based graphics for dynamic objects like meters and controls, but I am not sure how much overhead this would have. Also, I cannot stomach the look of the Windows dialogs, but I know I need something here that is crossplatform.

So, here is a simple list of the way things should work:
Vector graphics for dynamic objects, that can draw curves.
Programmable input handling for keyboard and mouse.
Can support crossplatform CUSTOM File dialogs with settings for default folder and favorites.


So far, Direct2D seems to be the candidate for windows. But I don't want to leave Mac users hanging, as many people have a need for plugins on the Mac. I know Mac uses OpenGL for hardware acceleration, but this is exactly the opposite of Direct2D, and would require twice the code, too much for myself.

Without hardware acceleration, things would be fine if it could be efficient. I do not like JUCE because of the license. I prefer to stay closed source at this time. Also, I don't like the way it handles mouse input.

So what options are there?

Well, you could buy a license for JUCE so that you can do closed source

Direct2D is ridiculously slow (actually even MORE so with "acceleration") and Vista+ only. On top of that Direct3D (which Direct2D uses for acceleration) causes all kinds of graphics glitches in some hosts; it's not really designed for plugins. Even if you don't mind the crazy API (of D2D; native D3D API is fine) it's more trouble than it's worth.

Unlike Direct3D, OpenGL actually DOES work perfectly fine on Windows with pretty much any host as long as you're careful about your OpenGL contexts.

I would have thought Direct2D would have less glitches. But I am not interested so much in the 3D aspect. I was just trying AriesVerb and this is OpenGL. The performance was very good, with no sign of graphical errors except for the 3D filter plot, where polygons would jump around at a certain view angle. I did notice a fixed editor refresh rate. As most GDI+ based plugins refresh the editor at the host buffer rate, this stood out. Is this a pitfall of OpenGL inside a host, or is it implementation specific?

I tried looking up for myself a long time ago, how to handle UI input when using OpenGL, but I didn't find much. What are some recomendations for mouse and keyboard handling? I need to be able to have text input boxes and buttons and knobs/sliders.

Yeah, I also thought D3D would work better until I actually tried. It seems that D3D is only really designed for the situation where it has complete control over drawing.

Refresh rate is really up to you. Doing host-buffer refresh is not necessarily a good idea though (eg for 8ms buffers you'd do 125Hz which is about 5 times too much already). I suggest doing refresh with WM_TIMER interval timer set at 20-25Hz or so; if CPU is very busy you'll miss some updates but that's really what you want anyway. Obviously if you don't need real-time visualization then you can also just update whenever contents change.

If you want good, stable vector images, you probably should not try to use OpenGL triangle rasterization directly. Even with anti-aliasing the quality can vary quite a bit. On the bright-side you get bilinear texture filtering for free so depending on what you want to do that might not actually be a problem. Or you could just draw shapes into a texture on CPU, then use OpenGL for blending (and possibly shading). Obviously depends on what you want to do.

That really depends. Intel (integrated) OGL support is generally lousy but you can safely target OGL 2.0 features without too many problems (YMMV).

However, if you wish to dive into hardware acceleration and not isolate your user base then it would be wise to implement multiple render paths to support a wide variety of hardware and select an appropriate one based on the user's hardware. Abstracting your low-level rendering code from high-level GUI draw calls will make this a breeze, as well as opening up many optimization strategies with regards to batching and combining geometry calls etcetera.

Direct3D support on Windows completely trumps OGL support but of course this means no cross-platform rendering code "straight out of the box". Again, supporting multiple rendering paths could incorporate multiple APIs but for VST GUI rendering (very lightweight in the grand scheme of things) this is probably using a sledgehammer to crack a nut. However, my background is in real-time 3D rendering and I have little experience with VST development so I will take you at your word with regards to D3D being problematic for VST GUI development.

OpenGL is a low level graphics API. It has no concept of user input, and quite rightly so. You should not be mixing rendering and UI code together as neither system should have any knowledge of each other so there should be no coupling between them. I do something like this for GUI development:

1) Grab UI
2) Let GUI manager decide which widget the UI should be applicable to
3) Perform any logic in response to UI
4) Set widget to "dirty" and thus needs to be redrawn
5) GUI manager iterates over widgets periodically and adds dirty widget's API-agnostic drawing information to scene manager
...
n) Scene manager builds drawing list from API-agnostic drawing information, optimizing where applicable
n+1) Graphics system operates on said list and compiles the necessary data to be sent to the GPU using API-specific calls and data structures, optimizing this data where applicable

Notice that the GUI system doesn't directly make any draw calls, especially widgets not making any draw calls themselves. By abstracting the low-level API draw calls from the GUI you give the scene manager (which handles the drawable items that actually get rendered) an opportunity to batch draw calls and optimize where possible. Likewise, the graphics system actually takes the list of API-agnostic draw calls from the scene manager and compiles the necessary data structures in whatever format the graphics API (OpenGL /DirectX/GDI and so on) requires, optimizing where possible. This allows you to swap and change your graphics API to suite the end user's capabilities, e.g. you could support oldschool video cards with low versions of OGL such as fixed function rendering (or even bypass OpenGL completely and use the OS's graphics API such as GDI/GDI+ on windows) or cards with shader support and specific (modern) OGL features and so on.

This is just some food for thought. If you're only doing GUI development then you can simplify things by integrating the GUI manager and scene manager together. Even this might seem heavy handed for simple GUI rendering (especially for largely static "scenes" such as plugin GUIs) as you won't really be doing any heavy work in terms of taxing the GPU or require high frame rates. However, IMO abstracting the graphics API from your GUI at least gives you flexibility if you ever need to change APIs or use specific API features that may or may not be available on all hardware. Having your GUI framework code littered with API-specific drawing calls is not only highly questionable practice but can also lead to headaches in terms of state m-a-n-a-g-e-m-e-n-t (<- this word is "spam" apparently?) and tightly coupling your generic GUI code with a specific API.

I'm not talking about features. I'm targeting pretty much OGL 1.1 with a few selected extensions (eg texture_env_combine). What I'm talking about is actual redraw problems: D3D content not clipping properly, or not drawing properly or not invalidating windows below properly or whatever else non-sense, when the containing window or one it's siblings doesn't quite happen to do what D3D expects. Slightly different issues depending on whether you use D3D9, D3D9Ex or D3D10+ and whether you have Aero running or not, but none of them quite work. And that's on ONE system (this isn't anything esoteric; normal Win7 + NV GTX570). You don't even need to draw anything: just replace your WM_PAINT with Clear+Present (and Begin/EndScene and ValidateRect and all that other crap that you need to render an empty frame) and you already run into weirdness. Happens with D2D too.

Compare my experience with OpenGL: works.

You can safely support at least OGL 2.0 in this day and age, doing away with fixed function pipeline and the awful texture combiners. All that stuff is emulated by the driver anyway.

rw, Opengl works as a renderer only, and the VST requires a UI. How would you implement it on top of the VST, so it is able to isolate keystrokes and mouse clicks within regions? I have seen this is many video games in my life, so I know it must not be too complicated. I believe the most popular solution was directinput, but if this could be done in something crossplatform, it would be better.

In the end of this idea, I would like to see my GUI work something like this...
A round knob with a text readout in the center.
The ability to update that text readout while turning the knob, and while the host automates it's parameter.
And the ability to right click/shift click on the knob to input a text value directly, with the text updated.

My idea is to refresh the editor at buffer rate. For users with onboard audio only, this could mean refresh times of 190ms. I don't know if it would be too much trouble to use v-sync or not, knowing how that is driver dependent. It would be cool to see the editor refresh linked to the monitor refresh rate if the buffer is faster than.

Is there any chance the editor could fall out of sync with the host?

Actually turns out some people are still using cards (eg G550) that don't even support texture_env_combine. I'm sure OGL2.0 gives you a fairly wide user base, but I like to keep as much backwards compatibility as reasonably possible. It's not like it matters much for GUI drawing anyway.

You can sync to v-sync, sure, but good luck getting that to work in two plugins at the same time. So in practice it's a BAD idea. Also for purposes of visualization 60Hz (or whatever) is total overkill anyway. Even video really only needs about 25Hz. Just because it's hardware accelerated doesn't mean it's free so I'd just use something reasonable and save the rest of the CPU for more important stuff (like audio).

Just to put things in perspective, lots of console games these days run at 30Hz (eg half vsync). Those that run at refresh (eg 60Hz) usually do so in mostly because higher refresh rate means less input lag from user to screen. If it takes 3 frames (note that this is pretty much the best case and doesn't include whatever latency the monitor itself has) from the button press to gun shooting, then at 60Hz it's ~50ms and at 30Hz it's roughly 100ms; this difference is noticeable where as 30Hz vs 60Hz refresh really isn't.

As for staying in sync, you should really deal with the situation that GUI refresh can't keep up with the audio. Sooner or later you'll run out of CPU in GUI thread anyway, whether that's because a random background task or just too much audio code. While it's not necessarily something that happens in "normal operation" you should still be prepared so you don't crash or cause a system lockup or anything else stupid like that. Normal thing to do is to drop video frames; most of the time anybody won't even notice unless you drop several frames in a row or your framerate drops for a longer period of time.

Okay that is a good point. Is this why no one uses it?

Don't use DirectInput, it's a wrapper for the usual window messages and even Microsoft reccomend against using it.

If you are rolling your own editor using VST 2.x then for Windows you'd create your window as a child of the host-supplied window handle (void* parent cast to HWND) and then your message pump would pass the usual OS messages to your newly-created window via the callback WNDPROC function. You'd handle your mouse/keyboard messages however you please, i.e. passing the pertinent information to your GUI whereupon it can handle the mouse/keyboard input by processing it and testing widgets to see whether the UI is pertinent to them.



Your actual GUI rendering will be independent of the GUI logic. In a simplified way, you'd take the UI and do some bounds checking for each widget to see which widget is the focus of the UI. You'd then react to that input by informing the plugin that a parameter has changed and then the plugin would do whatever function in response to the input.



Your rendering can be refreshed however you choose, it certainly need not be refreshed each time your audio block is processed. As GUIs change infrequently, you need only update the GUI in response to GUI parameter changes (and only the section of the GUI that actually needs to be changed).