Could we finally ditch C++ for VSTs?

DSP, Plug-in and Host development discussion.
otristan
KVRAF
1971 posts since 28 Mar, 2005

Post Mon Sep 16, 2019 2:42 am

Python won't work well for audio plugin
https://wiki.python.org/moin/GlobalInterpreterLock
Olivier Tristan
Developer - UVI Team
http://www.uvi.net

Benutzername
KVRist
350 posts since 23 Jan, 2008 from Hamburg, Germany

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 3:20 am

DJ Warmonger wrote:
Mon Sep 16, 2019 1:48 am
The possible alternative to C++ is Rust, which is however not conceptually different being, it's just streamlined for modern patterns and hardware.
Rust is great as it guarantees memory and thread safety while it still is as fast as C++. It would be a perfect choice for plugins and hosts. Having no more memory leaks, dangling pointers, null references, ignored return values or thread sync problems are very good arguments. The integrated build system with automatic dependency handling and builtin unit testing is very nice too.

However, the learning curve is quite steep, especially if you come from a C++ or even Java/C# background. Rust requires a very different mind set and it is very strict. You'll have to fight the compiler a lot when you start out but the effort will be rewarded at the end. Rust gave me a hard time but now I really love the language for my low level stuff.

External libraries with C interfaces are usually easy to use with Rust. There are even VST2 bindings out there that work very well at a first glance.

That being said, platforms like VST3 are based around C++ ABIs and C++ concepts and this is a big problem. Doing VST3 COM with Rust is pure hell as you have to emulate C++ vtables with structs, figure out inheritance, convert function signatures, take care about platform dependent types before even getting to do the things that COM does (IUnknown, QueryInterface etc).

It is doable but definitely no fun.

User avatar
vortico
KVRist
302 posts since 19 Jul, 2008

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 3:47 am

>Upon doing uni courses around DSP

A fact of life is that universities (by design) do not follow industry standards. Their goals are different, in a similar way to how the government is different than the commercial sector. C++ is a good money-making language for retail software. MATLAB, Python, Julia, and Quorum (whatever that is) aren't. They're useful for prototyping but are rarely distributed in the product itself, for good reason.

>I think having more elegant syntax makes DSP algorithms noticeably easier to understand and share.

I disagree. Problem-solving is naturally hairy, so a language that tries to solve a problem in a non-hairy way is like trying to fit a square block in a round hole. "Elegant syntax" means that it looks perfect on the surface, but good programmers don't care about the surface. They care that the language lets them handle deep problems without much friction, so they can solve a nasty problem in a somewhat decent way. For example, if a programmer feels that the most straightforward way to solve a problem with a performance requirement is to throw around a bunch of pointers, then the "best" language for the problem is one that allows you to easily throw around a bunch of pointers.

Anyway, to answer your original question, you can absolutely write a VST2 or VST3 plugin with any language you want. If the language lets you compile to a shared library with C-compatible symbols (e.g. Rust, Go, Crystal), you can reimplement Steinberg's VST headers if you really want. Or if the language interfaces with C in some way, you can write a C or C++ wrapper, either by embedding an interpreter or writing a "native extension". ...Except for Python, I wouldn't waste time trying to make it work with multiple threads because the interpreter is not fully thread-safe (requires GIL).

With all that said, writing a VST plugin is usually 25% DSP and 75% graphics, interaction, and cross-platform madness. You will quickly want to use C/C++ for the latter category, since those areas tightly integrate with the OS, which are C or Objective C.

If your goal is to start a career writing VST plugins and other retail/commercial software, it's nonsense to avoid using C++ at this point. If you're in school or recently graduated, you need as much practice as you can get. Nobody's going to hire you because you "choose to solve problems in ways that make your life easier". They'll hire you to do the job normally and professionally. On the other hand, if your goal is just to write a single VST plugin, I'd recommend using C++ anyway so you can just be done with it without messing with unusual environments.
VCV Rack open-source virtual modular synthesizer

User avatar
wrl
KVRist
36 posts since 12 Aug, 2011

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 4:02 am

Rust is pretty cool and the learning curve isn't too brutal. Just sucks for a few weeks while you get used to how strict the compiler is.

The issue with Rust is that you're completely on your own at this point in the framework and library regard. You'll basically end up writing your own plugin framework from scratch. Enough people/companies have done it in C/C++/Delphi/D, so it's not an insurmountable hurdle, but it's important to realise what you're getting yourself into.
owner/operator LHI Audio

untimelyman
KVRist
50 posts since 28 Mar, 2014

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 7:44 am

Almost all signal processing textbooks written in the last 25 years have MATLAB codes. That does not mean MATLAB is suitable for production-ready software.

mystran
KVRAF
5384 posts since 12 Feb, 2006 from Helsinki, Finland

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 9:35 am

wrl wrote:
Mon Sep 16, 2019 4:02 am
Rust is pretty cool and the learning curve isn't too brutal. Just sucks for a few weeks while you get used to how strict the compiler is.
I'll play devil's advocate: have they figured out how to handle intrusive (yes, because cache matters) cyclic references without resorting to "unsafe" already?
If you'd like Signaldust to return, please ask Katinka Tuisku to resign.

User avatar
wrl
KVRist
36 posts since 12 Aug, 2011

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 10:05 am

mystran wrote:
Mon Sep 16, 2019 9:35 am
I'll play devil's advocate: have they figured out how to handle intrusive (yes, because cache matters) cyclic references without resorting to "unsafe" already?
Truthfully, I don't know. I'm still in C (not C++ even) for all my audio stuff and I'm using intrusive containers all over the place.

I do Rust for a fair amount of my non-audio work, and I feel that especially for medium or high-risk applications (file format decoders, network protocols/services/clients), the additional development overhead of doing everything in Rust is worthwhile. However, for audio (and things like UI), folks are still figuring out what the best form to write in is. The best Rust I've seen to date generally looks a bit functional rather than imperative or OO.

I have a friend who does Rust all day every day and when I ask him about this stuff (and I do bring up intrusive containers) he says that sometimes `unsafe` is necessary and that it isn't always a bad thing. Certainly, plenty of libraries have `unsafe` blocks in them, but they wrap all of that up into a safe API, and that's the real win. The footguns are still there but you can corral them off.
owner/operator LHI Audio

Benutzername
KVRist
350 posts since 23 Jan, 2008 from Hamburg, Germany

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 10:07 am

mystran wrote:
Mon Sep 16, 2019 9:35 am
wrl wrote:
Mon Sep 16, 2019 4:02 am
Rust is pretty cool and the learning curve isn't too brutal. Just sucks for a few weeks while you get used to how strict the compiler is.
I'll play devil's advocate: have they figured out how to handle intrusive (yes, because cache matters) cyclic references without resorting to "unsafe" already?
Well, "unsafe" is not evil. It basically just means "I know exactly what I'm doing here".

The best way to handle cycling references is to structure your code and data in a way that you don't need them. I know that this is easier said than done, especially if you have to work with existing code. If you start a new project then it's much easier.

User avatar
wrl
KVRist
36 posts since 12 Aug, 2011

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 10:29 am

Benutzername wrote:
Mon Sep 16, 2019 10:07 am
The best way to handle cycling references is to structure your code and data in a way that you don't need them. I know that this is easier said than done, especially if you have to work with existing code. If you start a new project then it's much easier.
The problem is that they're incredibly useful and they're also all over the place. Tree with parent pointers? Cyclic references. Doubly-linked list? Cyclic references. Even in greenfield projects, sometimes (oftentimes, honestly) cyclic data structures are the right trade-off of complexity and performance.

I don't mean to pick on you specifically here, but I don't particularly find "just don't use those, then" to be a good response, especially not to seasoned developers who have (in many cases) already analysed the tradeoffs, settled on their preferred solutions, and used them in production.
owner/operator LHI Audio

mystran
KVRAF
5384 posts since 12 Feb, 2006 from Helsinki, Finland

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 10:41 am

Benutzername wrote:
Mon Sep 16, 2019 10:07 am
The best way to handle cycling references is to structure your code and data in a way that you don't need them. I know that this is easier said than done, especially if you have to work with existing code. If you start a new project then it's much easier.
This is not really an answer though, even though it's the canonical "answer" that Rust advocates like to give. There are plenty of algorithms which literally lose their complexity bounds if you give up cyclic data-structures. At that point, the only realistic alternative is to use indirection tables which (1) add pointless overhead and (2) essentially bring all the safety issues back.

The reason I keep bringing this up every time I hear about Rust is because just about 99% of all the bad-pointer problems I've had in the past 10 years had to do with cyclic structures one way or another. If your data naturally forms a DAG then it is actually NOT that hard to manage it safely in C++ with direct membership, stack allocs, RAII and/or smart pointers.

Don't get me wrong, I'm not against the compiler enforcing certain invariants unless you specifically tell it that you want to break them, but claiming that it solves memory safety while it still relies on "unsafe" in many practical situations is just false advertising. In comparison, managed garbage collected languages essentially DO solve the problem (at the cost of having to periodically trace through the memory at run-time) and apparently some variations of Haskell could even detect semantic garbage (ie. technically reachable objects that the application is never going to access; these are how you leak memory in a garbage collected system) by leveraging their type system.
If you'd like Signaldust to return, please ask Katinka Tuisku to resign.

Benutzername
KVRist
350 posts since 23 Jan, 2008 from Hamburg, Germany

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 3:46 pm

mystran wrote:
Mon Sep 16, 2019 10:41 am
The reason I keep bringing this up every time I hear about Rust is because just about 99% of all the bad-pointer problems I've had in the past 10 years had to do with cyclic structures one way or another.
That's exactly why I brought this up. The problem is obviously not the programming language here. If structures and algorithms are built around something that is responsible for 99% of all bad-pointer problems of a system then there is something fundamentally wrong with the system. No programming language will ever fix this.

There are always possibilities beyond just replacing pointers with index tables. Convert recursive functions into iterative ones, move algorithms out of the data classes into a larger scope, create bus systems instead of tree structures, add authorities that exclusively manage all object lifetimes and connections and the list goes on. Let the algorithms do all the dependency work, not the data cells. A leaf should not be able to call the roots of a tree to order more water. It's the job of the tree's water management system to serve water to all the leafs that are draining.

Of course there still may be situations where simple trees with pointers are the only viable solution. If that is the case then adding a few "unsafe" blocks is nothing to be ashamed of.
wrl wrote:
Mon Sep 16, 2019 10:29 am
I don't mean to pick on you specifically here, but I don't particularly find "just don't use those, then" to be a good response, especially not to seasoned developers who have (in many cases) already analysed the tradeoffs, settled on their preferred solutions, and used them in production.
I'm programming for about 33 years now and I have been working as a professional developer for the past 20 years or so. Around the time when I got my first job I also created my first VST plugin (around 1998/1999). I started with Basic and 6502 assembler as a child and then switched over to Pascal, C and finally to C++ later on. I've worked on more platforms and operating systems than I can count over all these years. However C++ is still the language of choice in my day job (CGI solutions for CAD systems) next to assembler, C#, Python and several shader dialects. From time to time I also have to do some Java and Kotlin coding for another project but I absolutely prefer to use C++ if I can. Rust and audio coding is just for fun these days.

I think that I'm seasoned enough to know about the benefits of tree structures and also about their dependency problems. That's why I'm always looking for alternatives and new ways to improve the tools in my toolbox. Learning a new programming language is not a threat to my view of the world. I see it as an opportunity to question my work flow and to question the things that I believe to know.

I also experimented a lot with strictly functional languages a while ago and while they are not my cup of tea they have some really nice properties to them. They really taught me that OOP is absolutely not the end of all means. Sometimes the algorithm is way more important than data wrapping and encapsulation at any cost. Rust sits nicely in the middle here and that's why I like it.

Will I use Rust for my main job? Not anytime soon. The main problem for me is not in the "unsafe" algorithms. The problem is that there are still no good platform independent GUI systems available that work the Rust way. Of course I could use Qt or Gtk but then it would look like I'm just creating C/C++ code with Rust keywords and that's pointless.

I hope that WebAssembly will come to the rescue here. But this has still a long way to go.

BerryLaCroix
KVRist
37 posts since 9 Apr, 2018

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 3:56 pm

VST development relies a lot on external SDKs. for plugins these are almost exclusively in C++. Plus many companies have invested heavily in their own C++ libraries for plugin development which will take man years to replace by something else.

Hence C++ is, and will be, in the years to come, the language of choice.

My 5 cents.

User avatar
Zaphod (giancarlo)
KVRAF
2475 posts since 23 Jun, 2006

Re: Could we finally ditch C++ for VSTs?

Post Mon Sep 16, 2019 7:27 pm

mystran wrote:
Sun Sep 15, 2019 5:55 pm
Forgotten wrote:
Sun Sep 15, 2019 5:13 pm
Also, it's pretty easy to embed inline assembly code into C++ which I would imagine would be very useful when trying to optimize audio processing.
Inline assembly with C++ is rarely a good idea these days (at least on mainstream hardware), since compilers these days are quite decent at low-level optimization and intrinsics let you write SIMD code directly from C++ while still leaving register allocation and instruction scheduling/selection to the compiler. Last I checked, MSVC didn't even support inline assembly for 64-bit targets, although you could certainly still link separately compiled assembler files if you really wanted... but honestly it's rarely worth it.
Assembly, assembler or intrinsics are more or less the same technique. They work very well and straightforward using c/c++
About compilers optimisations: we tried recently to compile very simple code using visual studio 2017 and xcode 10, in all possible compiler settings (and instruction setting) but our assembler version (based on the same CPU instructions) was always faster. A lot faster.
My 2 cents

User avatar
Guillaume Piolat
KVRist
199 posts since 21 Sep, 2015 from Grenoble

Re: Could we finally ditch C++ for VSTs?

Post Tue Sep 17, 2019 1:01 am

Benutzername wrote:
Mon Sep 16, 2019 3:20 am
That being said, platforms like VST3 are based around C++ ABIs and C++ concepts and this is a big problem. Doing VST3 COM with Rust is pure hell as you have to emulate C++ vtables with structs, figure out inheritance, convert function signatures, take care about platform dependent types before even getting to do the things that COM does (IUnknown, QueryInterface etc).

It is doable but definitely no fun.
We sell VST2 / VST3 / AAX / AU plug-ins and use the D language:
- you can disable the GC, and we do. For tools the GC is useful.
- it is native, as fast as C++. All native languages are practically equally fast, it's all about the backend.
- automated dependency linking+downloading
- you get to use LLVM + same frontend on macOS and Windows
- supports COM ABI (useful for VST3 and AAX) and C++ ABI (useful for AAX)
- it builds faster

But reading this forum it seems there is no alternative to C++ :dog: the problem is rather investment in a C++ codebase, only new codebase can afford a language change.
VST/AU/AAX: Couture | Panagement | Graillon

soundmodel
KVRian
686 posts since 28 May, 2010 from Finland

Re: Could we finally ditch C++ for VSTs?

Post Sat Sep 21, 2019 8:00 am

whyterabbyt wrote:
Mon Sep 16, 2019 1:28 am
soundmodel wrote:
Sun Sep 15, 2019 9:08 am
I tend to find that the developments in programming languages ought to be reflected to application domains
You 'tend to find' that something ought to happen?

Erm, that sounds more like opinion than a solid technical reason. What 'developments' that cant be leveraged in C++ impact on software plugins?
and then it seems like keeping using old technology just disallows one from taking advantage of new developments.
You really need to provide evidence of that. Lisp and Prolog still have a very significant toehold in AI development, for example, and Python, which seems to be one of the most common languages is nearly 30 years old.

C++ is perfectly suitable for VST development. Argument smacks a little bit of 'the world should adjust to me.'
My uni courses have taught me otherwise. We have explored "why C++ is a bad language" just like some else have.

I believe that the future of programming is in DSLs (Domain-Specific Languages). I.e. using and developing languages that are "closely" tailored for the problem domain. The host language can be of several kind. But even for these there exists evidence that some are "better preferred by users" than some others. But I believe this "evidence-based programming language development" is a newish thing. I've been taught that for long there has existed a philosophical problem related "how to measure programming language use".

Return to “DSP and Plug-in Development”