Do you ever think of developing in other languages than C++?

DSP, Plug-in and Host development discussion.
User avatar
KVRian
924 posts since 25 Sep, 2014 from Specific Northwest

Post Mon May 17, 2021 4:00 pm

Q9000 wrote:
Sun May 16, 2021 5:37 pm
C++ is unfixable.
C++ is easily fixed. Don't use any libraries except the C standard lib. No std lib, no Boost, etc., etc.

"Safe" computing comes at a cost, usually far greater than what you want to pay for DSP. Save safe coding for OSes, Internet and running your nuclear reactor.

User avatar
KVRist
121 posts since 24 Jan, 2021

Post Mon May 17, 2021 4:29 pm

mystran wrote:
Mon May 17, 2021 12:31 pm
So like.. what exactly is Rust supposed to do for me?
Speaking as a Rust user (and addressing everyone in the thread, not just you): If you have any doubts of this kind, don't use Rust.

The Rust community, like the C++ and Java communities before it, certainly contains many enthusiasts. By this I mean people who have both a shallow understanding of something and a sentimental attachment to it. They don't grasp the engineering tradeoffs involved, and they confuse their sentimental feelings for technical superiority. It's a phase a lot of people go through. We're on KVR, so we're surrounded by relevant examples, as well as counterexamples. They're vocal, but they're hardly the entirety of any community.

The designers of Rust are all veterans. They have been burned repeatedly by the drawbacks and limitations of other technologies, particularly C++ (but also OCaml, Haskell, Java, Cyclone...). They're conscious of the costs and benefits of the tradeoffs they choose to make. I'd listen to them and ignore the enthusiasts as much as possible, because the enthusiasts didn't make any of those decisions in the first place. And as annoying as the enthusiasts can be when they yell about using Rust for everything, the designers are pretty clear about some reasons why you shouldn't.

If you're primarily modifying existing code, and it's not written in Rust, don't use Rust. Switching to Rust means writing new code.

If your team is a small number of veterans in constant communication, and you religiously review and test each other's code, don't use Rust. Many parts of the design aim to catch mistakes that slip through code review, especially in larger teams where some programmers are less experienced.

If your code is strictly and reliably single-threaded, don't use Rust. Many of the safety features are specifically designed around the hazards of concurrent code.

If you deploy code built in debug mode, don't use Rust. Many of the "zero-cost" abstractions rely on the optimizer removing redundant operations, such as bounds checking inside a loop. (I can't imagine this biting an audio plugin, but weirder things have happened.)

If your code is specific to a particular platform, there may be no point using Rust. In fact, if your code runs on embedded processors that LLVM can't target, you cannot use Rust. All of the C++ and assembly I've written in the past seven years has been for such processors. The future might be different, though, considering ARM's inexorable march towards world conquest at every level of the supply chain.

The community that actually matters to your product consists of your team and your customers. If your teammates (or customers) want to do something that Rust would make easier, that's when you should consider using Rust.

All that said, though, the GUI situation is getting really good. I prototyped something in egui the other day (for editing a custom file format) and it was worlds easier than the Flutter version I wrote a few months ago.
I hate signatures too.

KVRAF
6368 posts since 12 Feb, 2006 from Helsinki, Finland

Post Mon May 17, 2021 7:26 pm

Super Piano Hater 64 wrote:
Mon May 17, 2021 4:29 pm
If you deploy code built in debug mode, don't use Rust. Many of the "zero-cost" abstractions rely on the optimizer removing redundant operations, such as bounds checking inside a loop. (I can't imagine this biting an audio plugin, but weirder things have happened.)
This is nonsense. I often write stuff where I don't even get the priviledge of developing in debug mode, because the thing just doesn't work (as in "does not meet real-time goals" which is the same as not working) without optimizations.

As for "optimizer removing bounds checking" that's a big lie we've been told for decades now. The "sufficiently smart compiler" does not exist and one can prove that it's never going to happen: proving general array index bounds statically requires dependent types that are undecidable in the general case. Note that there are languages (with dependent types) where you can prove a useful subset of cases statically, but Rust is not one of them and I'm going to insist that a runtime check is not useful for typical real-time DSP code, because by the time it fires on your users's system it's already way too late. What you need is a build-time guarantee and you either need a type system that can prove it, or you need to prove it yourself the hard-way.
All that said, though, the GUI situation is getting really good. I prototyped something in egui the other day (for editing a custom file format) and it was worlds easier than the Flutter version I wrote a few months ago.
Non-sense. Immediate mode GUIs (such as egui) are not workable for anything complex.
Preferred pronouns would be "it/it" because according to this country, I'm a piece of human trash.

User avatar
KVRist
121 posts since 24 Jan, 2021

Post Mon May 17, 2021 8:40 pm

mystran wrote:
Mon May 17, 2021 7:26 pm
This is nonsense. I often write stuff where I don't even get the priviledge of developing in debug mode, because the thing just doesn't work (as in "does not meet real-time goals" which is the same as not working) without optimizations.
I think I miscommunicated something here. I'm saying that the same is true of most Rust programs: Optimizations have to be turned on for the damn things to be of any use. The scenario I described, of "deploying code built in debug mode," is one I have encountered in the wild, and I picked on it as a case where Rust would have been inappropriate.
mystran wrote:
Mon May 17, 2021 7:26 pm
As for "optimizer removing bounds checking" that's a big lie we've been told for decades now. The "sufficiently smart compiler" does not exist and one can prove that it's never going to happen: proving general array index bounds statically requires dependent types that are undecidable in the general case. Note that there are languages (with dependent types) where you can prove a useful subset of cases statically, but Rust is not one of them and I'm going to insist that a runtime check is not useful for typical real-time DSP code, because by the time it fires on your users's system it's already way too late. What you need is a build-time guarantee and you either need a type system that can prove it, or you need to prove it yourself the hard-way.
You've explained before that you wouldn't write DSP code in Rust, for precisely these reasons. I honestly wasn't trying to change your mind on that.

If I wrote DSP code in Rust, I'm sure I'd use one of the dozen or so libraries that map overloaded operators to SIMD intrinsics. I'd store data in fixed-length arrays so the optimizer would have the greatest number of opportunities to remove redundant operations. In cases where it failed to do so, I would go ahead and use unsafe operations, and comment them with explanations of what I was trying to do, why I invoked the scary magic word, and what future alternatives someone else might use if they want to make it go away. At all stages, I'd be inspecting the compiler output and benchmarking whatever I could.

If none of that worked, I'd rewrite it in Fortran.
mystran wrote:
Mon May 17, 2021 7:26 pm
Non-sense. Immediate mode GUIs (such as egui) are not workable for anything complex.
I tend to agree, but it depends on your complexity threshold. MS Word? That's a fool's errand. A "hello world" demo with one button? Piece of cake. I think most plugin UIs are somewhere in the middle. Maybe it falls on me to provide a counterexample.
I hate signatures too.

User avatar
KVRAF
4449 posts since 21 Jan, 2008 from oO

Post Tue May 18, 2021 11:35 am

How about coding in Julia? It has a JIT compiler ok, but no garbage collector and it abstracts away a lot of complexity. Has multiple dispatch and is dynamically typed with parametric polymorphism, so expressive but type stable. It's also fast like C.
lucid.network | former user Suloo

KVRian
549 posts since 29 Jan, 2017

Post Tue May 18, 2021 12:08 pm

mystran wrote:
Mon May 17, 2021 12:31 pm
Safe Rust is also not workable for performance sensitive numerical code, because the overhead of runtime bounds checking on arrays is ridiculous and you really want to relax strict IEEE semantics so that your compiler can actually do some floating point optimization.
I think that's not true. The float overflow checking is turned only in debug mode. In release mode it's the same behaviour as in c++.
So how about writing the GUI side in Rust? Has somebody actually figured out how to write GUI code in Rust yet? Widget demos are all great, but a realistic GUI is a large pile of event-driven state and a complex set of cyclic dependencies and that's something that affine typing truly struggles with. It also happens to be the problem domain where classic OOP inheritance and encapsulation truly shine.
I recently saw a 3D game written in Rust so I assume plugin GUI should be doable as well.

I don't want to start C++ vs Rust fight, because it's pointless and I'm not a Rust advocate :D, so I'd approach from a pragmatic point of view as somebody exploring and learning Rust. The problems/risks I see with this language:
- lack of fast-math - it's possible via LLVM instrinsics, works as performant as with the compilation switch but is more clunky
- limitations on data structures which might be a big deal because some of them might be impossible to write without unsafe
- lack of compilers - currently there's only LLVM, but GCC frontend is being worked on. So if you have some exotic toolchain and/or device you're out of luck
- bounds check - I saw some instrinsics that could help mitigate this problem. Apparently the compiler is getting smarter but one would have to constantly verify is bounds check is happening or not (haven't verified this yet)
- lack of SIMD instrinsics in stable rust - you'd have to use nighly version
- possibility of Rust moving more towards "safe way" instead of "performant way". So in theory the language designers could sacrafice performance for correctness
- existing gazilions of c++ code that will never get rewritten :D

Having said all above working with Rust feels way more fun than with C++. I can actually add dependecy which compiles and works without spending hours adding it to CMake and hoping it will compile and link correctly.

Regarding unsafe - you seem to be worried about using it, but using unsafe is really just going back to C/C++ world where you're on your own?

User avatar
KVRian
924 posts since 25 Sep, 2014 from Specific Northwest

Post Tue May 18, 2021 12:25 pm

One language I haven't seen mentioned here yet is Pony:
https://www.ponylang.io/
It has a very interesting concurrency model that would solve scads of DSP vs editor issues with inter-communication and it would be interesting to see if could be adapted to real-time DSP processing.

Along with Nim and Zig, it's another language for me to keep an eye on.

As an aside, I gave Free Pascal a whack a while back and took a week to recode a plugin from C++. I discovered why Pascal lost the language wars. :lol: Basically, I had to abandon it because the optimization just wasn't there. Unoptimized C++ was still pretty much as fast or faster than optimized Pascal.

KVRist
409 posts since 4 Jan, 2007

Post Tue May 18, 2021 12:50 pm

ulzee wrote:
Tue May 18, 2021 11:35 am
How about coding in Julia? It has a JIT compiler ok, but no garbage collector and it abstracts away a lot of complexity. Has multiple dispatch and is dynamically typed with parametric polymorphism, so expressive but type stable. It's also fast like C.
Being a number crunching language with lots of math libs at a minimum it could make a pretty cool prototyping environment. It seems embeddable on a C binary.

Knowing almost nothing on the language, I guess it's designed for throughput and maybe it could cause clicks on the first runs.

KVRAF
6368 posts since 12 Feb, 2006 from Helsinki, Finland

Post Tue May 18, 2021 1:47 pm

ulzee wrote:
Tue May 18, 2021 11:35 am
How about coding in Julia? It has a JIT compiler ok, but no garbage collector and it abstracts away a lot of complexity. Has multiple dispatch and is dynamically typed with parametric polymorphism, so expressive but type stable. It's also fast like C.
What are you talking about? This looks a whole lot like a GC: https://github.com/JuliaLang/julia/blob/master/src/gc.c
Preferred pronouns would be "it/it" because according to this country, I'm a piece of human trash.

KVRian
549 posts since 29 Jan, 2017

Post Tue May 18, 2021 2:00 pm

syntonica wrote:
Tue May 18, 2021 12:25 pm
One language I haven't seen mentioned here yet is Pony:
https://www.ponylang.io/
It has a very interesting concurrency model that would solve scads of DSP vs editor issues with inter-communication and it would be interesting to see if could be adapted to real-time DSP processing.

Along with Nim and Zig, it's another language for me to keep an eye on.

As an aside, I gave Free Pascal a whack a while back and took a week to recode a plugin from C++. I discovered why Pascal lost the language wars. :lol: Basically, I had to abandon it because the optimization just wasn't there. Unoptimized C++ was still pretty much as fast or faster than optimized Pascal.
Zig is definitely interesting and IMO looks even better than Rust because it's ... simpler and has better C interop. Not sure if it will gain momentum tough

User avatar
KVRian
924 posts since 25 Sep, 2014 from Specific Northwest

Post Tue May 18, 2021 2:39 pm

0degree wrote:
Tue May 18, 2021 2:00 pm
Zig is definitely interesting and IMO looks even better than Rust because it's ... simpler and has better C interop. Not sure if it will gain momentum tough
Zig is getting a bit Rococo for my taste, but I'm waiting for the big 1.0 to drop before I dive in. The async functions (i.e. coroutines) look pretty basic, however, and the fact that Zig uses inference to find them sort of goes against what Zig is supposed to be about. I've never been a big fan of coroutines, which I've used in Lua.

KVRian
549 posts since 29 Jan, 2017

Post Tue May 18, 2021 2:48 pm

syntonica wrote:
Tue May 18, 2021 2:39 pm
0degree wrote:
Tue May 18, 2021 2:00 pm
Zig is definitely interesting and IMO looks even better than Rust because it's ... simpler and has better C interop. Not sure if it will gain momentum tough
Zig is getting a bit Rococo for my taste, but I'm waiting for the big 1.0 to drop before I dive in. The async functions (i.e. coroutines) look pretty basic, however, and the fact that Zig uses inference to find them sort of goes against what Zig is supposed to be about. I've never been a big fan of coroutines, which I've used in Lua.
Thanks for info. I guess we'll have to wait and see. So far it's not super popular but who knows

User avatar
KVRAF
4449 posts since 21 Jan, 2008 from oO

Post Tue May 18, 2021 7:55 pm

mystran wrote:
Tue May 18, 2021 1:47 pm
ulzee wrote:
Tue May 18, 2021 11:35 am
How about coding in Julia? It has a JIT compiler ok, but no garbage collector and it abstracts away a lot of complexity. Has multiple dispatch and is dynamically typed with parametric polymorphism, so expressive but type stable. It's also fast like C.
What are you talking about? This looks a whole lot like a GC: https://github.com/JuliaLang/julia/blob/master/src/gc.c
yes, I stand corrected. I'm not an expert, just getting into programming with Julia and wondering if it could be used.

From what I found out is that it could be avoided almost completely with optimized Julia code in a sense that one doesn't allocate or just once and uses a warm-up function for precompilation and running the process isolated on one core.
https://unix.stackexchange.com/question ... ng-process

Although, these discussions end up making the difference between hard and soft real time development. And they seem to agree on that Julia couldn't guarantee hard real time requirements, because even if you run everything isolated, there would still be shared memory requests that could interrrupt the process.
https://discourse.julialang.org/t/julia ... r/34654/41

Not sure if hard real time is required here.

Here is a talk about RTC processing in Julia: https://www.youtube.com/watch?app=desktop&v=dmWQtI3DFFo

Maybe with a StaticCompiler it could be improved. https://github.com/tshort/StaticCompiler.jl
Approach explained: https://docs.google.com/document/d/1rLp ... aXGKQ/edit
lucid.network | former user Suloo

Return to “DSP and Plug-in Development”