Developing a fast and responsive GUI - is JUCE too bloated?

[dt2m]

Hello everyone - been lurking this fantastic forum for a while, have a question:

I'm starting to get into DSP plugin development, and my interest is in creating plugins with very responsive GUIs (60fps, GPU rendered) with as little performance hit to the CPU as possible.

The gold standard to me is the FabFilter suite of plugins that seem to be running real-time FFT spectrum analyzers as well as buttery smooth UI elements at 60fps, even in the midst of a busy mix. They seem to be rolling their own framework, which I'd expect from that level of performance.

I've tried to run strings on the executables of some of my favorite plugins to determine which library the developers have used, and it seems like there's a high correlation between the ones that use JUCE and the ones that have UIs that break a sweat running at anything higher than 30fps. To me it seems like JUCE has included way too much in the framework (a web browser? really?) which in turn only results in more CPU cycles being wasted on a subpar UX.

To me, it seems like the Steinberg VST3 framework might offer better UI performance when using VSTGUI, but is this a wrong assumption? I'm just starting out and would like to choose the best framework beforehand for the many years to come.

I have three specific noob questions that I wonder about:

1. What framework/which techniques should I use for lowest-overhead GUI development? Does the Steinberg VST3 framework perform better than JUCE for UIs?
2. What is the distinction between the VST3 framework and the VST3 API?
3. If I were to decide to attempt to roll my own GUI library instead of using either VSTGUI or JUCE, does anybody know of any examples/google keywords that can point me in the right direction, if they're there?

I am well aware that I might have bitten off more than I can chew in this regard, but every journey starts with a single step... Thanks for looking at my noob questions - and if they're too broad please excuse me as this is still a bit new and exciting to me!

[FigBug]

The issue with JUCE isn't bloat (i.e. the web browser is just a wrapper around the system webview, if you don't want it don't include that module, it will have 0 affect on your plugins performance). The issue with JUCE is that most of the rendering happens on the CPU.

VSTGUI (to my understanding) is mostly bitmap based. This is fairly fast, but fairly inflexible and doesn't play well with HiDPI screens. JUCE, everything is vector based. This means everything can scale perfectly and not get blurry, but it's a lot more math, which is slow.

If you want high performance graphics, you need to do it on the GPU. This means using something like DirectX on Windows and Metal on macOS. As macOS is deprecating OpenGL, it could mean writing your GUI twice.

Or you may want to look at something like Vulkan. There is no off the shelf GUI framework with great GPU performance that I've found. You still may want to use something like JUCE, since it does the VST, VST2, AAX, AU wrappers for you which saves a lot of time. Let JUCE create the plugin window for you, but then not use their drawing routines.

You may want to look to web technologies to do your UI. Desktop UI frameworks aren't getting much love these days. All the R&D seems to be gong into the web. Maybe using something like WebGL gets you a high performance UI that you only need to write once and isolates you from the technological differences on macOS & Windows.

You may also want to look into React Native.

[EvilDragon]

VSTGUI has vector primitives and polygon drawing as well (same as JUCE really), just the same as JUCE has bitmap support. They're both pretty much the same in this regard, JUCE is not vector only.

JUCE does support OpenGL rendering, tho (VSTGUI D2D, OGL and Metal). AFAIK VSTGUI doesn't have a SVG parser but JUCE does.

[hibrasil]

In iPlug2 we have 60FPS GPU backends https://iplug2.github.io

[hibrasil]

The answer to your main question is "no" though.

[Guillaume Piolat]

In Dplug we have a fast software rasterizer that always write 4 pixel at once (a fork of `https://github.com/cerjones/dg2d`). The bottleneck is rather uploading parts of the buffer that were updated on screen, on macOS doing it fast is quite the challenge.
(EDIT: and I don't think JUCE is too bloated for this)

[dt2m]

Thanks so much to everyone - things make a lot more sense to me now. iPlug2 looks promising as well. For now i'm still getting started with DSP programming so I'll return to this after having prototyped some more actual code.

Quite shocked at how many different approaches there are to doing this- really makes you think about how much unnecessary RAM & CPU that gets wasted on UI in complex DAW sessions. I guess I finally have the answer as to why the built-in devices in Ableton/FL use so little CPU compared to VSTs.

[syntonica]

I'm not sure what the smart guys use on Windows these days, but absolutely go with something Metal-based on the Mac. Apple has deprecated OpenGL and unless a third-party decides to step in, it's only going to decay and crumble with time.

On a side note, I used OpenGL with nanoVG to do drawing. It was the same speed as optimized CPU drawing, but it scaled very well. If the GUI got bigger, the CPU usage was the same for OpenGL, but grew somewhat proportionately with system calls. (Actually, with OpenGL, scaling was a simple as passing a single new size ratio!)

[FigBug]

really makes you think about how much unnecessary RAM & CPU that gets wasted on UI in complex DAW sessions. I guess I finally have the answer as to why the built-in devices in Ableton/FL use so little CPU compared to VSTs.

The amount of CPU used by the GUI should be fairly irrelevant. Most plugins spend the majority of their time with their UI closed, and even if open, the UI thread should be at a lower priority than the audio processing thread. The CPU meter in most DAWs only show the CPU usage of the audio thread, not the UI thread. If a system is overloaded, the UI should just slow down and not affect the audio processing.

[mystran]

If you want high performance graphics, you need to do it on the GPU. This means using something like DirectX on Windows and Metal on macOS. As macOS is deprecating OpenGL, it could mean writing your GUI twice.

It's actually not quite so black&white. Vector path-rendering in particular is a problem that is not necessarily all that trivial to solve on the GPU either. In fact, depending on how it's implemented, you could still be spending quite a bit of CPU just generating the paths to be rendered (eg. stroke-generation is not always trivial) and sending the data to the GPU.

Now, I want to make it clear that I'm not trying to discourage anyone from rendering on the GPU. Rather I'm just trying to point out that it's really not a silver bullet. In order to get a high-performance GUI you need to write high-performance GUI code. More importantly, you need to understand what is and isn't expensive in your graphics pipeline. No matter what your graphics pipeline looks like, it's almost always possible to make it perform poorly if you don't understand it's limitations.

This is why I would personally like to discourage anyone just starting out from "rolling their own" framework, unless they already have experience writing high-performance GUI code. Writing your own framework is a lot of work and performance never comes free. Rather I would suggest taking an existing framework, sorting out all the other things first, because even if you decide to "roll your own" one day, the work required to move your code from one framework to another is frankly quite trivial compared to the work of actually writing a framework.

ps. My own current frameworks can do both CPU and GPU rendering, even at the same time. For most purposes I use the CPU rendering though as it works quite fine.

[syntonica]

really makes you think about how much unnecessary RAM & CPU that gets wasted on UI in complex DAW sessions. I guess I finally have the answer as to why the built-in devices in Ableton/FL use so little CPU compared to VSTs.

The amount of CPU used by the GUI should be fairly irrelevant. Most plugins spend the majority of their time with their UI closed, and even if open, the UI thread should be at a lower priority than the audio processing thread. The CPU meter in most DAWs only show the CPU usage of the audio thread, not the UI thread. If a system is overloaded, the UI should just slow down and not affect the audio processing.

Until you run into a GUI that takes up a whole thread on my CPU, pinned at 25% and everything screeches to a halt.

That was definitely not a good programming choice. Having the GUI open made my system unusable.

[innervisions]

JUCE does have a few GUI performance issues, which are related to rendering on the CPU as FigBug mentioned.

But you can certainly write 60fps graphics/FFT renderers in it, either using the JUCE built in OpenGL renderer, or (better) just using smart high level design in your GUI. No magical framework would solve your GUI performance problems if you're recalculating your FFT-based analyzers on the message thread and block everything else while that happens.

As for rolling your own framework - this is only a good idea if you're into writing frameworks, and realize that would probably mean it would take a long time (maybe even years) before you get a product out, because you're gonna spend a long time on low level 'framework' code.

Much easier IMO to just find the performance bottlenecks in an existing framework and work around them.

[Youlean]

It is insanely difficult to achieve good 2D graphics performance actually. There are a ton of problems associated with ether CPU and GPU rendering:

CPU:
Best compatibility, works on every system
Easy to implement, quite a few great fast drawing libs like Cairo for example
Slow to paint to screen on macOS, especially if you use P3 screen
Uses CPU to render everything.

GPU:
Really bad compatibility. No real way to support older systems. Problems on both macOS and Windows in terms of what is supported.
Hard to implement, there are a few good libs too like NanoVG
Faster to draw, but still using CPU
If just blitting images, it should be fast
If using a lot of stroking, it will be slower than CPU rendering since all GPU libs are constructing paths with CPU. Skia strokes all paths on separate textures and then it uploads it to the GPU on OpenGL for example.
Using GPU to draw graphics could cause more wattage drawn from the CPU with integrated graphics which will make it throttle making the audio performance worse which is not possible with just CPU drawing where the main thread will just wait, and GUI will stutter.

In the end, using the CPU for GUI should be much better if the performance is good enough on HiDPI screens.

[kingozrecords]

I think Kuiml is well tested and is something that has been bench marked. In My opinion Bluecat's plug n' script offers the most viable solution. But I obviously question whether or not the include file; like in the case of a browser. I wonder if it is dynamically truncated. Like in the case of Jquery for instance, it's minimized, but you could also rebuild it with a less obtrusive library. I doubt Blue cat's has taken that into account but they should have. Perhaps they'll read that and find a way to achieve it.

But the point I make is that KUIML was designed to run fast online many years ago; for connection speeds which mere fractions of what is available offline. That is a point worth considering.

[syntonica]

I think the ultimate answer is, it doesn't matter what your backend is, just limiting redraws to dirty widgets is the best optimization you're going to get.