Ciaramella DSP language goes public and open source

[stefano-orastron]

Hi all,

I am happy to announce that Orastron, the company of which I am founder and CEO, has released a new and experimental audio DSP programming language called Ciaramella and its source-to-source compiler called Zampogna under the ISC license.

You can check out the official website - which also contains a web playground for trying it out directly in the browser - and read about its history and motivation here: https://www.orastron.com/blog/ciaramella-smc. The compiler source code is available at https://github.com/paolomarrone/Zampogna.

We'll also present a scientific paper that describes it in the detail on Friday 10th June 2022 at the Sound and Music Computing conference in Saint-Etienne, France (https://smc22.grame.fr/).

As it's something we wish could be actually picked up by the industry, unlike many other such tools, and since it's in an experimental state I'd say for at least another couple of years, any feedback/comment/question etc. is more than welcome.

[mystran]

I wish your language well, though I've never been quite convinced about these declarative languages myself (even though I've even tried to design my own). They seem great for simple things, but it's never been entirely clear to me how to handle situations that require things like scatter filtering or possibly varying sampling rates efficiently. I just always feel like it's kinda easier to just use C++ directly.

Speaking of C++ ... it's relatively simple to write a few templates that allow you to construct efficient WDFs simply by template composition. The resulting syntax is arguably kinda horrid, but it works fine and the efficiency is fine. I used this approach a few times years ago before I realized that WDFs are kinda waste of time and solving arbitrary systems (eg. just stamp your networks MNA style, no need to mess with adapters) with LU is pretty much just as efficient (as long as you unroll the whole thing with a code-gen; obviously running a generic LU-function on a matrix stored in an array would be slow) as the WDF impedance matching computation is basically LU in hiding.

Anyway, back to the topic ... since my other interest (besides DSP) is programming languages, it's always interesting to see more languages; even if one never becomes popular, perhaps there are some new great ideas there that the next batch of languages can steal.

[stefano-orastron]

I do agree with you on the limitations of declarative languages and indeed that's what we're concentrating on at the moment. For multi-rate processing in particular we believe it actually boils down to getting control flow right, specifically branching and looping. To give you an idea, we're now experimenting with branching and the syntax should be something like

Code: Select all

a,b,c = if (x > y) { a = 1; b = 2; c = x; } else { a = y; b = 3; c = 0; }

and the curly braces actually delimit "blocks", which means they can contain their own state, and are executed conditionally (so the outer sample rate is always >= than the internal sample rates, and none are required to be uniform. Summing all up, this should be sufficient to actually implement downsampling. With a similar reasoning, loops should be sufficient to implement upsampling.

Now, this is the theory. In practice this introduces significant complications to the compiler, as each branch multiplies the number of potential data flow paths by 2, so you get an exponential growth of potential schedules in the worst case (imagine the mess when doing code generation...) We're trying to solve this by graph cutting, so that we can identify separate branches that are "unrelated". Plus, we need to figure out if this syntax is "understandable enough" and convenient, or if it's better to introduce some syntactic sugar.

What I'm actually more worried about at the moment is typing - at the moment there's none and I don't think it will be easy to come up with something that is both flexible and precise - and "concurrent" writing to tables (being declarative + index-based read/write seems hard).

Pragmatically speaking, it's out of question that general purpose languages such as C++ are better at dealing with general problems and will always be useful no matter how extensive a DSL can be. As long as it is easy to interface code both ways (GPL<->DSL) and the DSL stays simple, elegant, useful, and tastefully limited I think we're good.

[mystran]

and the curly braces actually delimit "blocks", which means they can contain their own state, and are executed conditionally (so the outer sample rate is always >= than the internal sample rates, and none are required to be uniform. Summing all up, this should be sufficient to actually implement downsampling. With a similar reasoning, loops should be sufficient to implement upsampling.

Well, part of the problem is how to get efficient code. Think something like a modulated oscillator triggering the sampling clock at essentially arbitrary instants (within some bounds of where the sampling rate can vary) and a sinc-kernel with something like 32-64 taps per branch. This sort of thing is absolutely not a problem at all in an imperative language that has deterministic memory layouts and gives direct access to SIMD intrinsics (ie. when the clock triggers, fetch two closest branches, lerp the kernel, multiply accumulate with signal in buffer, can be done either gather or scatter depending on which rate's Nyquist the kernel needs to match) ... yet how would one go about expressing this sort of thing in a declarative language?

I've thought about this problem some and I feel like these kinds of things would probably be best expressed by moving to a higher level, treating signals-over-time as primitive types so that something like a gather-FIR application could be directly expressed as a primitive inner-product between maybe an abstract slice of the signal and similarly a scatter-FIR could be treated as a sum of two signals and such operations would then somehow be translated by the compiler into efficient SIMD operations on buffers rather than naive sample-by-sample dataflow graphs...

Yet... I know enough about compilers that I realise actually coming up with a design that's realistic both from the user point of view and the compiler point of view is not entirely straight-forward.

Now, this is the theory. In practice this introduces significant complications to the compiler, as each branch multiplies the number of potential data flow paths by 2, so you get an exponential growth of potential schedules in the worst case (imagine the mess when doing code generation...) We're trying to solve this by graph cutting, so that we can identify separate branches that are "unrelated". Plus, we need to figure out if this syntax is "understandable enough" and convenient, or if it's better to introduce some syntactic sugar.

I would need way more details about your specific approach to really comment on what you're trying to describe, but at least thinking in terms of traditional compilation your above example branch construct is basically already in SSA form (where "assignments" out of the branch just become phis), so it would seem as if standard compiler theory applies.

What I'm actually more worried about at the moment is typing - at the moment there's none and I don't think it will be easy to come up with something that is both flexible and precise - and "concurrent" writing to tables (being declarative + index-based read/write seems hard).

Well, it's usually the combination of "flexible, precise and practically implementable" that's the real problem, right?

[stefano-orastron]

and the curly braces actually delimit "blocks", which means they can contain their own state, and are executed conditionally (so the outer sample rate is always >= than the internal sample rates, and none are required to be uniform. Summing all up, this should be sufficient to actually implement downsampling. With a similar reasoning, loops should be sufficient to implement upsampling.

Well, part of the problem is how to get efficient code. Think something like a modulated oscillator triggering the sampling clock at essentially arbitrary instants (within some bounds of where the sampling rate can vary) and a sinc-kernel with something like 32-64 taps per branch. This sort of thing is absolutely not a problem at all in an imperative language that has deterministic memory layouts and gives direct access to SIMD intrinsics (ie. when the clock triggers, fetch two closest branches, lerp the kernel, multiply accumulate with signal in buffer, can be done either gather or scatter depending on which rate's Nyquist the kernel needs to match) ... yet how would one go about expressing this sort of thing in a declarative language?

I've thought about this problem some and I feel like these kinds of things would probably be best expressed by moving to a higher level, treating signals-over-time as primitive types so that something like a gather-FIR application could be directly expressed as a primitive inner-product between maybe an abstract slice of the signal and similarly a scatter-FIR could be treated as a sum of two signals and such operations would then somehow be translated by the compiler into efficient SIMD operations on buffers rather than naive sample-by-sample dataflow graphs...

If I undestand you correctly, and correct me if I'm wrong, you are considering arbitrary- and variable-rate resampling, right? If that's the case I can tell you that we haven't thought about it.

The longer answer would be, though, that:

1. the most compelling use cases for us are time-domain algorithms that are commonly found in audio effects and synthesizers;
2. efficiency is of extreme importance, but we accept that certain things are better developed using other tools (as said, if we can easily interface with those we're ok);
3. no DSL that I know of today gets you that far (which is not an excuse, especially since we want to do better, but still).

The language exists primarily to save us from doing the same boring things over and over (e.g., checking which part of each expression is constant vs control-rate vs audio-rate, or which expression depends on which parameter, etc.) - and it already handles all of that (perhaps I should have pointed it out more clearly in the blog post, damn!)

We also want to keep things relatively simple and intuitive, at least for the user.

Given that these primary constraints are respected, we can then try to make the thing as useful and complete as possible. But TBH at the moment there are more pressing and basic aspects to be taken care of.

Yet... I know enough about compilers that I realise actually coming up with a design that's realistic both from the user point of view and the compiler point of view is not entirely straight-forward.

I've given a look at your projects and I see quite some overlap with what we are doing here. I'm contacting you privately.

Now, this is the theory. In practice this introduces significant complications to the compiler, as each branch multiplies the number of potential data flow paths by 2, so you get an exponential growth of potential schedules in the worst case (imagine the mess when doing code generation...) We're trying to solve this by graph cutting, so that we can identify separate branches that are "unrelated". Plus, we need to figure out if this syntax is "understandable enough" and convenient, or if it's better to introduce some syntactic sugar.

I would need way more details about your specific approach to really comment on what you're trying to describe, but at least thinking in terms of traditional compilation your above example branch construct is basically already in SSA form (where "assignments" out of the branch just become phis), so it would seem as if standard compiler theory applies.

Probably so. My personal knowledge in SSA and compiler theory is quite rusty right now. Paolo, the guy actually working on this project for his PhD, knows more (I'll perhaps ask him to join the conversation).

What I'm actually more worried about at the moment is typing - at the moment there's none and I don't think it will be easy to come up with something that is both flexible and precise - and "concurrent" writing to tables (being declarative + index-based read/write seems hard).

Well, it's usually the combination of "flexible, precise and practically implementable" that's the real problem, right?

Absolutely. But I prioritise like this: 1. flexible, 2. practically implementable, 3. precise.

[mystran]

1. the most compelling use cases for us are time-domain algorithms that are commonly found in audio effects and synthesizers;

Well, I suppose it depends on how you look at it, but from my point of view things like BLEP scatters and sinc-interpolation are commonly found in synthesizers and (especially modulation) effects... and basic oversampling ofcourse is everwhere, though that's an easier problem from a language design point of view.

3. no DSL that I know of today gets you that far (which is not an excuse, especially since we want to do better, but still).

That's the thing. I'm aware no such language exists, but as a programming language nerd, I'd love a language that made it easier and more convenient to express the hard things. That doesn't mean you should specifically try to do it, I'm just throwing my random thoughts in the air in case they happen to be useful.

I suppose good enough interop can go a long way though. Simple scripting languages (eg. Lua) can be very powerful as it's very easy to interop with C/C++ code both ways, especially if one can add missing primitives into the DSL from the host language side. For dynamically typed garbage collected languages this is relatively easy (well, at least if you don't need to deal with the GC from the C/C++ side), but it's not entirely clear (to me anyway) how to do it well for a statically typed real-time safe language.

ps. just so I don't give the wrong idea, I'm by no means a compiler expert, just that I've written a few interpreters and compilers ... 'cos that's kinda what programming language nerds tend to do

[Z1202]

3. no DSL that I know of today gets you that far (which is not an excuse, especially since we want to do better, but still).

ReaktorCore as a DSL was aiming in a similar direction. Unfortunately no new updates have been released for about a decade. Not sure how many of hard cases listed by mystran would have been implementable, but things like BLEPs were definitely on the list (nevermind the current examples, the BLEP implementation there is more like a workaround due to missing language features). Obviously, I can't say much more...

[stefano-orastron]

1. the most compelling use cases for us are time-domain algorithms that are commonly found in audio effects and synthesizers;

Well, I suppose it depends on how you look at it, but from my point of view things like BLEP scatters and sinc-interpolation are commonly found in synthesizers and (especially modulation) effects... and basic oversampling ofcourse is everwhere, though that's an easier problem from a language design point of view.

Oversampling is absolutely covered as said before. That sort of arrangement can happily also represent polyphase filtering, so no worries (well... once tables are available actually).

W.r.t. BLEP, to be honest in my career I've only implemented PolyBLEP (actually also with quite some improvements over the published papers) as old school BLEP is obsolete IMO (much worse performance/CPU cost ratio). Anyway, I don't see why BLEP would actually require resampling at all to be implented, but I admit I've not looked much into how other people do it. I'd go like this:

1. you add the needed delay for lookahead (so I'll describe the rest of the approach in anticausal terms)
2. you use current phase and frequency (phase increment) to determine whether there has been a discontinuity and you can compute exactly "when" it happened.
3. you add/remove terms by computing/taking them from tables w/ some interpolation etc.
4. optionally you post filter to compensate for the high-frequency losses.

Such a thing should be easily doable in Ciaramella once we have loops and tables. I don't think a trivial translation to your target language of choice would be optimal, but that's an "optimisation" problem if you ask me, and actually much more related to FIR filtering in general than resampling. Unless I miss something.

3. no DSL that I know of today gets you that far (which is not an excuse, especially since we want to do better, but still).

That's the thing. I'm aware no such language exists, but as a programming language nerd, I'd love a language that made it easier and more convenient to express the hard things. That doesn't mean you should specifically try to do it, I'm just throwing my random thoughts in the air in case they happen to be useful.

Thank you for that. We need food for thought.

[stefano-orastron]

3. no DSL that I know of today gets you that far (which is not an excuse, especially since we want to do better, but still).

ReaktorCore as a DSL was aiming in a similar direction. Unfortunately no new updates have been released for about a decade. Not sure how many of hard cases listed by mystran would have been implementable, but things like BLEPs were definitely on the list (nevermind the current examples, the BLEP implementation there is more like a workaround due to missing language features). Obviously, I can't say much more...

But can you actually picture your-favourite-plugin/hardware-company using Reaktor (Core) for their DSP and integrating it their build system etc. etc.?

On the technical (and abstract) merits, though, Reaktor Core, like many other such languages, implicitly adds unit delays in feedbacks... which is the #1 problem we solved by using a declarative approach. As replied already to mystral, BLEPs (and related) are on the list here too and if we manage to get a couple more features right I believe our language would get really interesting and actually usable. And it's open source, we want people to participate, not only for the adoption, but also since, pragmatically speaking, there's a whole lot that needs doing and our resources are extremely limited.

[mystran]

W.r.t. BLEP, to be honest in my career I've only implemented PolyBLEP (actually also with quite some improvements over the published papers) as old school BLEP is obsolete IMO (much worse performance/CPU cost ratio).

This is non-sense. PolyBLEPs scale poorly with kernel length (and therefore poorly with quality) so they only really win in terms of performance for relatively short kernels where taking a few cache misses for the LUT fetch isn't worth it. As such, they are purely a performance option when CPU cost is more important than quality. Once you're pushing for quality and need a longer kernel, it is literally faster to just fetch it from a table.

Anyway, I don't see why BLEP would actually require resampling at all to be implented, but I admit I've not looked much into how other people do it. I'd go like this:

You don't really need resampling for BLEPs, rather you want to be able to scatter the residues into a buffer when you encounter a discontinuity (since tracking the residues over time with potentially arbitrary number of overlaps would be rather infeasible and terribly inefficient). This in turn means that this sort of algorithm needs to have access to both future and past samples, preferably as a continuous block.

Now, the case of BLEPs is not necessarily the most important thing, rather I just brought it up as a good example of a common algorithm that doesn't easily map into traditional data-flow formulations based on individual samples in an efficient way. Like I said, I'm a programming language nerd and I'd love to see some higher-level constructs that can express such things while still allowing the compiler to generate competitive code.

[Z1202]

But can you actually picture your-favourite-plugin/hardware-company using Reaktor (Core) for their DSP and integrating it their build system etc. etc.?

Technically there's no problem at all whatsoever. It's just a question of a product decision and adoption. (I'm referring here to ReaktorCore language specifically, not the entire Reaktor). And it already has been used for that by NI, just within Reaktor, not outside. Some of outstanding products (Monark is my personal favorite) have been both prototyped and implemented in Reaktor, using ReaktorCore as the DSP basis.

On the technical (and abstract) merits, though, Reaktor Core, like many other such languages, implicitly adds unit delays in feedbacks... which is the #1 problem we solved by using a declarative approach.

I'm not sure what are you referring to here by "solved", is it the "delay1" operator? ReaktorCore has a similar one, and that one is even not built-in, it is built from lower-level language features. Yes, there is also an implicit option for quick-patching, where the compiler will auto-highlight the feedback loops, as a warning means. And, there is (again, built from lower-level features) a ZDF toolkit, although that one is somewhat limited, as the language was not really designed for that

Please get me right, I'm not trying to downplay your creation, on the contrary, I'm quite happy to see a new audio DSL language following something more than a slightly tuned imperative approach. I mentioned ReaktorCore, because I think it's another language of the family (unfortunately unfinished). It is using an integral mixture of reactive, imperative and declarative approaches. And some of the other things (like branching) are also already solved there, and implicitly support the "undersampling" feature, which can be used to implement control rates.

[kippertoffee]

I've thought about this problem some and I feel like these kinds of things would probably be best expressed by moving to a higher level, treating signals-over-time as primitive types so that something like a gather-FIR application could be directly expressed as a primitive inner-product between maybe an abstract slice of the signal and similarly a scatter-FIR could be treated as a sum of two signals and such operations would then somehow be translated by the compiler into efficient SIMD operations on buffers rather than naive sample-by-sample dataflow graphs...

Can you elaborate on this scatter/gather terminology? It's not something I've come across before.

[stefano-orastron]

W.r.t. BLEP, to be honest in my career I've only implemented PolyBLEP (actually also with quite some improvements over the published papers) as old school BLEP is obsolete IMO (much worse performance/CPU cost ratio).

This is non-sense. PolyBLEPs scale poorly with kernel length (and therefore poorly with quality) so they only really win in terms of performance for relatively short kernels where taking a few cache misses for the LUT fetch isn't worth it. As such, they are purely a performance option when CPU cost is more important than quality. Once you're pushing for quality and need a longer kernel, it is literally faster to just fetch it from a table.

Sorry, I meant that PolyBLEP with a good kernel (4th order B-spline at least) at a higher sample rate + downsampling gives you results that are comparable to using long BLEP residuals at a lower computational cost. At least that's what I've experienced (and to be honest, I've compared informally my PolyBLEP implementations vs other people's BLEPs, so that's not completely "fair" in a sense).

Furthermore, especially in common subtractive synths, you might want to downsample after the filter anyway if it contains nonlinearities.

Anyway, I don't see why BLEP would actually require resampling at all to be implented, but I admit I've not looked much into how other people do it. I'd go like this:

You don't really need resampling for BLEPs, rather you want to be able to scatter the residues into a buffer when you encounter a discontinuity (since tracking the residues over time with potentially arbitrary number of overlaps would be rather infeasible and terribly inefficient). This in turn means that this sort of algorithm needs to have access to both future and past samples, preferably as a continuous block.

Now, the case of BLEPs is not necessarily the most important thing, rather I just brought it up as a good example of a common algorithm that doesn't easily map into traditional data-flow formulations based on individual samples in an efficient way. Like I said, I'm a programming language nerd and I'd love to see some higher-level constructs that can express such things while still allowing the compiler to generate competitive code.

I tend to disagree. Accessing "past" and "future" samples can be done with regular delay lines, possibly implemented by circular buffers. On the optimality of the thing... well, that's another chapter, but in most cases I bet a simple approach wouldn't be horrible.

[Z1202]

I tend to disagree. Accessing "past" and "future" samples can be done with regular delay lines, possibly implemented by circular buffers. On the optimality of the thing... well, that's another chapter, but in most cases I bet a simple approach wouldn't be horrible.

I'm personally not sure which of the approaches is better, and one could definitely want to do both. Also generating the entire BLEP transient at once is definitely simpler to think of (YMMV), even though it can be rewritten in purely causal terms. Thus, it may affect the usability of the language.

[stefano-orastron]

But can you actually picture your-favourite-plugin/hardware-company using Reaktor (Core) for their DSP and integrating it their build system etc. etc.?

Technically there's no problem at all whatsoever. It's just a question of a product decision and adoption. (I'm referring here to ReaktorCore language specifically, not the entire Reaktor). And it already has been used for that by NI, just within Reaktor, not outside. Some of outstanding products (Monark is my personal favorite) have been both prototyped and implemented in Reaktor, using ReaktorCore as the DSP basis.

Last time I checked you couldn't export C/C++ code or make your own VSTs, let alone having a decent CLI or dealing with all the legal issues that could be attached to such a thing. Of course it's different if you're NI. But I admit I am ignorant and may have overlooked something.

On the technical (and abstract) merits, though, Reaktor Core, like many other such languages, implicitly adds unit delays in feedbacks... which is the #1 problem we solved by using a declarative approach.

I'm not sure what are you referring to here by "solved", is it the "delay1" operator? ReaktorCore has a similar one, and that one is even not built-in, it is built from lower-level language features. Yes, there is also an implicit option for quick-patching, where the compiler will auto-highlight the feedback loops, as a warning means. And, there is (again, built from lower-level features) a ZDF toolkit, although that one is somewhat limited, as the language was not really designed for that

Unluckily many details will only be available once the first paper is out (in a few days now), but essentially what is "solved" is that you can express whatever algorithm and grouping parts of it in "blocks" in arbitrary ways, and that has no impact on how operations are scheduled. AFAIK, no current language can do that. This means that you can express very clearly and concisely things like wave digital filters (even if I do agree with mystral that in practice they're not as good as they look on paper most of the time), or more in general you can abstract things away parts of algorithms as freely as possible.