Once I tried to start a fair discussion... :)

[mahaya]

First of all, thanks to Jules for replying to questions on this thread.

Maybe I missed something in the documentation but I'm not sure I fully understand the purpose of SOUL. It sounds a bit like Reason't Rack Extensions (e.g. a DSP representation that can be run on different hardware), Reaper's Jesusonic or maybe Ableton's Max for Live. Of course, with the difference that the processing hardware for SOUL is flexible.

I can also see some browser applications for example in the 3D audio field, but I'm a bit at a loss how the implementation of classic audio production software would be done. Could you (Jules) elaborate on what you expect as SOUL products for DAWs? For example if it should be a "new plug-in format", how would a plugin developer approach a plug-in using SOUL and especially how would the GUI be done?

[mystran]

That's why we're keeping the almost-impossibly-difficult-to-write bit as closed-source, so we can keep that option open.

Why not just be honest and tell us exactly which parts are open-source and which parts are not?

It's not like people on this forum are open-source fanatics or anything. There are concerns in this thread that suggest the exact opposite, when people are expressing their concerns on whether SOUL-code might disclose their proprietary trade secrets.

What people don't like is bait-and-switch schemes and pigs in a poke. Either it's free or it isn't (or some parts are free and some parts are not; we just want to know which parts are and aren't), but when you're trying to tell people to adopt your "open standard" which apparently relies on some proprietary magic faerie dust which is so secret that you're not even willing to tell us what technical purpose does it serve, that just smells rotten.

Knowing JUCE, it's probably going to be a working product, but we'll have to wait and see about the "open" part.

[jules]

Maybe I missed something in the documentation but I'm not sure I fully understand the purpose of SOUL. It sounds a bit like Reason't Rack Extensions (e.g. a DSP representation that can be run on different hardware), Reaper's Jesusonic or maybe Ableton's Max for Live. Of course, with the difference that the processing hardware for SOUL is flexible.

I can also see some browser applications for example in the 3D audio field, but I'm a bit at a loss how the implementation of classic audio production software would be done. Could you (Jules) elaborate on what you expect as SOUL products for DAWs? For example if it should be a "new plug-in format", how would a plugin developer approach a plug-in using SOUL and especially how would the GUI be done?

I could elaborate here, but TBH I'd just be repeating all the stuff from my original announcement talk last year, and the docs we've released in the github repo now, so if it's OK with you I'll use my limited KVR time to try to address some of the (sigh..) FUD..

I appreciate the honesty, but unfortunately that shifts the context from "we create a standard open to all for the better good" up to, "once we have everyone's using it, we may make big profit from it".

Not at all.. Linux is open, but Redhat makes multi-billion dollar profits by providing a good version of it to people who need that. openGL is open, but billions are made from revenue from hardware, software and middleware products that use it. TCP/IP is open, but Cisco make billions by selling good implementations of it.

What we're doing is no different to those or many other examples of open standards. Being open and free for developers and end-users doesn't mean that the same things can't also be commercialised by getting companies who profit from it to pay for useful services to help them do that.

We want to do both: create an open standard, and create a services business to actually make money from it.

Why not just be honest and tell us exactly which parts are open-source and which parts are not?

We've been really honest and open about this project right from the start! And you can look at the repo now to see exactly what's there! All the front-end compiler code is open. The JIT engine is still closed (and is still a work in progress anyway, we're not ready to let that go public). That'll probably remain the situation in 2020, after which I'm not going to predict where we'll end up - it'll depend on many factors like our resources, partners etc, and we're keeping our options open to see how things unfold. Likewise as far as licensing goes, we're keeping our options open to see what partnerships come along. But that's literally the whole story. There aren't any nefarious secret plans we're holding back!

What people don't like is bait-and-switch schemes and pigs in a poke. Either it's free or it isn't (or some parts are free and some parts are not; we just want to know which parts are and aren't), but when you're trying to tell people to adopt your "open standard" which apparently relies on some proprietary magic faerie dust which is so secret that you're not even willing to tell us what technical purpose does it serve, that just smells rotten.

The back-end magic isn't "secret", it's just incredibly difficult and expensive to do well, and having put a huge amount of sweat and money into it, we're keeping those cards against our chest for the moment. Anyone could now write their own soul back-end based on what we've already released, just like anyone could write an openGL driver. But these are incredibly difficult things to do, and there aren't many people in the world who can do it, and we need to pay a team of those people to make this a long-term success. So if we can commercialise the project by licensing our best-in-class soul backends to major hardware vendors, then that's a pretty sensible plan to make it happen, and has no negative impact on developers or users. There's nothing cynical or exploitative about that, so I resent the kind of language you're using there, as if we're somehow trying to trick people.

[mystran]

We've been really honest and open about this project right from the start! And you can look at the repo now to see exactly what's there! All the front-end compiler code is open. The JIT engine is still closed (and is still a work in progress anyway, we're not ready to let that go public). That'll probably remain the situation in 2020, after which I'm not going to predict where we'll end up - it'll depend on many factors like our resources, partners etc, and we're keeping our options open to see how things unfold. Likewise as far as licensing goes, we're keeping our options open to see what partnerships come along. But that's literally the whole story. There aren't any nefarious secret plans we're holding back!

Thank you for this clarification, I'm sure I'm not the only one that appreciates it. This is already much more information than I could find on your repository without actually looking into the individual source files. The text-files in the repo that describe the project are very vague about this though.

[Vertion]

The compiler is out under a permissive ISC license. The EULA allows anyone to freely build hardware or software that uses it.
...
Your point about wanting its creators to renounce control is mistaken though - on the contrary, that'd kill the whole thing. We want large companies to commit entire product lines to this technology over many years. For them, the idea of committing to a rudderless project would be a complete no-starter. Open-source, yes, but nobody would risk committing to a project that could suddenly go off in a random direction or be abandoned or splintered into competing factions.. They need to know exactly who's in charge, and to trust them, and they'll often even pay for the privilege of being involved in the roadmap.

Jules,

Thank you for responding. We have something in common, we both want to keep this market growing. We both understand strategy. I see you have made a lot of movement in a top down way. KVR is the garden and the raw heart of our industry. While things may not always be rational here, we are true to ourselves and move the market in subtle ways. Most of us are the little guys, who are just trying to make it by in a very saturated market.

Larger companies often make selfish choices that make it difficult for us to adapt to. Countless hours wasted just to accommodate the whim of a company who controls a standard. So, we are wary when someone comes along hyping their solution, but wants to maintain control. It has proven to always translate to loss at some point.

I agree that there should be direction, but it’s not a one person show. Developers have families to support and another revolution is not always welcomed.

One solution would be a non-profit entity, controlled equally by those both large and small to vote on changes followed by a public trial phase to see how everyone likes the change before it is finally kept. An authority over these standards. This maintains both direction and inclusion. Exclusivity is never the hero. Make the committee number large and odd (lest a coin toss on even), so votes to change always move forwards.

This approach may frustrate more aggressive revolutionary forward thinkers, but it will gain the blessing of a larger base over time.

You are a leader. Do right not just by those at the top, but those at the bottom and you will find success comes easily. The seeds you place in the garden are the ones that can grow the tallest. Good luck.

I believe.

[jules]

Thanks for your wise words!

Do right not just by those at the top, but those at the bottom and you will find success comes easily. The seeds you place in the garden are the ones that can grow the tallest. Good luck.

Absolutely, and 98% of my effort on the project has been from the perspective of what would make me, as a developer, want to use this thing. The other 2% has been "how the hell do we justify paying for this". But I've always taken the view that if we can build something that everyone wants, a business model will emerge somehow. (You may be able to sense that I don't have an MBA..)

It kind of goes without saying that over the course of the last two years, we've explored every possible strategy, including creating a consortium, or a non-profit group, or going fully open, or going fully closed, etc etc. I think in a perfect world some wealthy benefactor would fund a team of 10 people to work on this with no strings attached, but no sign of that just yet!

If it does gain a big enough foothold to become a critical part of the infrastructure of many large companies then I think it'd inevitably end up morphing into some kind of consortium or committee, as has happened with lots of other languages and platforms. But if we get to the point where industry giants are pressurising us to make that kind of move, I'll personally be very happy to have got there! Right now our mission is just to make this as awesome as possible, and get people using it..

[mystran]

For what it's worth, I'm looking at the VCS3 example and trying to resist the temptation to write an improved one..

As it turns out, with LU and one-sample of latency, one can do a predictor-corrector scheme where you only factor the matrix once, but solve the linear equation twice per sample. The trick is to perform the corrector step (and state updates) first, then use the same matrix to compute the predictor (ie. new non-linear conductances) for the next sample (which is where the single sample latency comes in, so you don't need to store the matrix-factorisation)...

...and this brings us to the real reason why I'm writing this post: when writing these filters (especially when they get slightly more complex) in C++ one can save a ton of time by writing a small code-generator (eg. the one I have is 168 lines of commented Lua) that takes the symbolic matrix and unrolls the LU factorisation into code (such that constant folding can then work it's magic). If you assume that the input is properly pivoted, then the whole thing is completely mechanical. You might even be able to get it out of C++ templates, although with a rather ugly syntax.

In fact, this same thing can be done by compiler optimisations as well: if you force the compiler to fully unroll all the nested loops, then perform "scalar replacement of aggregates" on the whole array (which then enables constant-folding, dead-code elimination, loop invariant code-motion and register allocation to properly work their magic), you could get the same performance out of a regular LU loop... but the problem here is that trying to convince a general purpose compiler to actually do such aggressive optimisation is "problematic" since this kind of optimisation is profitable in only very specific scenarios.

So what I would like to see in an audio-processing language would be explicit control for these kinds of optimisations. If I could specify that "really unroll this loop no matter what" and "really replace this array with scalars no matter what" then I could just throw that Lua-script into recycle bin.

[xhunaudio]

Hi Jules,

I'll try to repost my question because unfortunately I didn't receive an answer for this. It is just to understand if SOUL could be considered future-proof for the End-User.

The DAW/Host of the future will be a software including many parts (a sequencer, an interface for input/output audio drivers, etc.). As well today one ASIO driver can be choosen from a list to be used in a DAW/Host (choosen from a list of many different 3rd party ASIO drivers), a future DAW/Host with support for SOUL patches *must* let you choose what 3RD PARTY SOUL RUNTIME DRIVER to use from a list. Below you'll find an example :

"Welcome to Steinbeg's Cubase of the future. Please choose your SOUL driver : "
- SOUL Driver "A" (made by Microsoft Corp, to run SOUL patches on Windows / on any x86-64 CPUs)
- SOUL Driver "B" (made by Intel Corp, to run SOUL patches on AVX vector units on x86-64 CPUs)
- SOUL Driver "C" (made by Intel Corp, to run SOUL patches on Altera FPGA)
- SOUL Driver "D" (made by Avid, to run SOUL patches on their DSPs)
- ...
- SOUL Driver "Z" (made by the Open Source community, to run SOUL patches on Windows / x86-64 CPUs)

Here's my question :
Will - today and forever - the SOUL DRIVER "Z" development scenario be always allowed (or better "encouraged" with an open-source example of a SOUL back-end driver) by ROLI/JUCE/SOUL policies without requiring a money amount/fee to be paid ?

This is a crucial point, please take the time you need to reply but let us know !

[jules]

Hi Jules,

I'll try to repost my question because unfortunately I didn't receive an answer for this. It is just to understand if SOUL could be considered future-proof for the End-User.

The DAW/Host of the future will be a software including may parts (a sequencer, an interface for input/output audio drivers, etc.). As well today one ASIO driver can be choosen from a list to be used in a DAW/Host (choosen from a list of many different 3rd party ASIO drivers), a future DAW/Host with support for SOUL patches *must* let you choose what 3RD PARTY SOUL RUNTIME DRIVER to use from a list. Below you'll find an example :

"Welcome to Steinbeg's Cubase of the future. Please choose your SOUL driver : "
- SOUL Driver "A" (made by Microsoft Corp, to run SOUL patches on Windows / on any x86-64 CPUs)
- SOUL Driver "B" (made by Intel Corp, to run SOUL patches on AVX vector units on x86-64 CPUs)
- SOUL Driver "C" (made by Intel Corp, to run SOUL patches on Altera FPGA)
- SOUL Driver "D" (made by Avid, to run SOUL patches on their DSPs)
- ...
- SOUL Driver "Z" (made by the Open Source community, to run SOUL patches on Windows / x86-64 CPUs)

Here's my question :
Will - today and forever - the SOUL DRIVER "Z" development scenario be always allowed (or better "encouraged" with an open-source example of a SOUL back-end driver) by ROLI/JUCE/SOUL policies without requiring a money amount/fee to be paid ?

This is a crucial point, please take the time you need to reply but let us know !

I don't think it'd even be possible for us to stop people writing a soul-compatible driver if they want to.

...however, I also can't really imagine that being something that more than a handful of ideologically-motivated open-source people would care about doing.

Because if we do succeed at making it ubiquitous and you want to play sound through your SOUL-enabled audio i/o box, or your SOUL smart-speaker, or your SOUL-powered headphones, etc, then you'd have to use the drivers provided by those manufacturers for their own specific hardware. An open-source implementation could only render the code on your CPU and play it via an old-fashioned audio device, so wouldn't provide any of the latency or DSP acceleration advantages that dedicated devices can give.

[jules]

So what I would like to see in an audio-processing language would be explicit control for these kinds of optimisations. If I could specify that "really unroll this loop no matter what" and "really replace this array with scalars no matter what" then I could just throw that Lua-script into recycle bin.

Yes, we'll certainly support performance hints like that, although of course they'll only ever be hints as we obviously don't want to over-prescribe what capabilities the back-end compiler must support. We already have a [[ ]] annotation syntax kind of like the C++ one for this sort of thing.

The other nice thing about SOUL is that as a combination of being a very pure language and having a JIT means there's the opportunity to do some really aggressive compile-time evaluation. We envisage being able to write code that looks like it'll build some e.g. lookup tables at runtime based on the sample rate etc, but where the compiler is actually pre-running all that code and baking the results directly into the compiled code.

[xhunaudio]

An open-source implementation could only render the code on your CPU and play it via an old-fashioned audio device, so wouldn't provide any of the latency or DSP acceleration advantages that dedicated devices can give.

Thank you so much for your quick reply Jules, very nice to hear that...

I think opening the SOUL back-end drivers to any development (without any $ amounts to pay) would be very important because of both CPUs and FPGAs.

Writing it for a CPU apparently doesn't add any benefit to the current state of the art C++ development, but it would (silently) keep SOUL alive in the pro-audio industry on the very long-term scenario. A sort of assurance policy for all SOUL patches coders/developers - for the decades to come.

Instead, applied to FPGAs, open-source SOUL back-end drivers initiatives would bring very important and exciting features.

An example in 2 words : when FPGAs diffusion will be stronger enough, their internal "arrangement assets" (the software instructions needed to re-arrange their internal memory, FP, INT, etc. blocks) will be available for any specific purpose (vector graphics, computing systems emulation, pro-audio dsp, etc..,).

Consequently, an existing SOUL back-end driver may need to be slightly modified to adapt for that specific "arrangement update" for that FPGA semiconductor.

By "opening up" the drivers development with no kind of limitations to (for example) open-source initiatives, that will probably let very interesting projects to emerge, for example :

offering open-source FPGA "arrangements" suited for pro audio/dsp purposes AND at the same time an appropriate SOUL driver to benefit for that re-arranged hardware. In this scenario is mandatory to keep SOUL drivers development open to anyone; nice to hear from you that you'll never obstruct this.

By the way thank you for your clarification, my fears now seem to be gone away...

[jules]

An open-source implementation could only render the code on your CPU and play it via an old-fashioned audio device, so wouldn't provide any of the latency or DSP acceleration advantages that dedicated devices can give.

Thank you so much for your quick reply Jules, very nice to hear that...

I think opening the SOUL back-end drivers to any development (without any $ amounts to pay) would be very important because of both CPUs and FPGAs.

Writing it for a CPU apparently doesn't add any benefit to the current state of the art C++ development, but it would (silently) keep SOUL alive in the pro-audio industry on the very long-term scenario. A sort of assurance policy for all SOUL patches coders/developers - for the decades to come.

Instead, applied to FPGAs, open-source SOUL back-end drivers initiatives would bring very important and exciting features.

An example in 2 words : when FPGAs diffusion will be stronger enough, their internal "arrangement assets" (the software instructions needed to re-arrange their internal memory, FP, INT, etc. blocks) will be available for any specific purpose (vector graphics, computing systems emulation, pro-audio dsp, etc..,).

Consequently, an existing SOUL back-end driver may need to be slightly modified to adapt for that specific "arrangement update" for that FPGA semiconductor.

By "opening up" the drivers development with no kind of limitations to (for example) open-source initiatives, that will probably let very interesting projects to emerge, for example :

offering open-source FPGA "arrangements" suited for pro audio/dsp purposes AND at the same time an appropriate SOUL driver to benefit for that re-arranged hardware. In this scenario is mandatory to keep SOUL drivers development open to anyone; nice to hear from you that you'll never obstruct this.

By the way thank you for your clarification, my fears now seem to be gone away...

Yep. You're not saying anything here that we haven't already been discussing internally in depth for the last couple of years. But I've been in this industry too long to go around making promises about where we'll end up years in advance

[xhunaudio]

Great, keep up the good work, I wish all the best with SOUL !

If you have some additional infos and sneak-previews about it in the future, any comment/post here in this thread is welcome

[resynthesis]

Quite interesting the effect the domain has on the tech. I spent quite some time on a not totally dissimilar project aimed at improving portability in safety-critical, hard real-time systems. The big difference in that domain is that you _do_ need to make promises about where you'll be years in advance

[xhunaudio]

At this point (2019), I don't think Jules / the JUCE team / ROLI will be more specific about this, in the short-term. I take his :

I don't think it'd even be possible for us to stop people writing a soul-compatible driver if they want to.

as a good starting point...