CLAP: The New Audio Plug-in Standard (by U-he, Bitwig and others)

[camsr]

I have a simple question that hopefully isn't off topic, but have not seen addressed by anyone who is currently compiling CLAP plugins. Are symbol exports destined to follow a prescribed rule on decorations? I think some of my plugins in the past had problems because of MinGW not following the MSVC symbol decorations.

There’s only one symbol you need to export, which is the clap entry data structure. I don’t know if anyone has built with mingw on windows yet but if the export macros in the clap headers need adjusting for that toolchain just hit us up on github and I’m sure we can figure it out.

Maybe I will do that, I am just avoiding unnecessarily creating a github account to post comments. Although I don't think anything in private/macros.h in in error, based on my limited experience.

I have had an error with numerous VST2 plugins compiled with MinGW where in some hosts, or while being bridged in the working hosts, one of the stereo channels fails to process. I thought it may be related to the processReplacing() parameters, such that my function is not receiving the correct pointer for one of the channels, or there is some kind of calling convention mismatch, or just something I was never able to isolate the problem. But when used in hosts that work, my plugins just work without problems. For example, my plugin SmoothSlew (hosted here on KVR in the effects forum) fails to process some of the channels in BespokeSynth, but it works flawlessly in Reaper, Foobar2000, and FL Studio. The plugin is not doing anything weird WRT pointer handling, except for maybe NOT copying the function parameters onto the local stack (since it should already be on the local stack since it is a parameter), and then modifying the pointer address in place to increment. I don't see how that could be an error. If anyone would like to take a look at it and help me, I will forever be in your debt

[S0lo]

https://github.com/surge-synthesizer/cl ... o.cpp#L522

That code shows how the little demo synth does it. Basically voice lookup depends on the coordinates you are giving. That synth (and surge too) support mono, pck, and noteid based modulation

Two questions,

1. Can we assume that channel = voice number. Or at least channel indicates a particular voice? Edit: or Port/Channel indicate a voice

2. Is there a reason why PCK can’t be a note ID by itself in the standard. I mean, Is there a situation PCK is not unique enough, ie, two different playing notes are having the same port, same channel and same key? If not, then why have note ID in the first place.

[Urs]

https://github.com/surge-synthesizer/cl ... o.cpp#L522

That code shows how the little demo synth does it. Basically voice lookup depends on the coordinates you are giving. That synth (and surge too) support mono, pck, and noteid based modulation

Two questions,

1. Can we assume that channel = voice number. Or at least channel indicates a particular voice? Edit: or Port/Channel indicate a voice

Not necessarily - the standard does not define this, because it simply mimics MIDI ports, channels and keys.

However, I presume in many MPE implementations the channel is identical with the voice. I don't know. That really depends on the instrument itself.

2. Is there a reason why PCK can’t be a note ID by itself in the standard. I mean, Is there a situation PCK is not unique enough, ie, two different playing notes are having the same port, same channel and same key? If not, then why have note ID in the first place.

Yes, PCK can be ambiguous if two notes on the same channel overlap. E.g. if you quickly hit the same key on a MIDI keyboard, release times of previously pressed keys may overlap with the more current notes.

Note_ID avoids that ambiguity, as even overlapping notes will be assigned different IDs by the host.

A special Bitwig operator highlights the issue: The Voice Stack option can play 5 voices of the Surge synthesiser at once per note with different modulations, because Surge implements Note_ID. It does not yet work with ACE, Diva or Hive because the implement PCK.

[S0lo]

Thank you Urs for your elaborate answer

The thing is even when using note ID to resolve all ambiguities, most if not all plug-ins are practically limited for a parameter mod by the number of voices that are playing. So if 3 voices are playing, and say all 3 cutoffs are being modulated. A 4th cutoff can’t possibly be modulated even if the host asks for it due to a repeated note. Which means now that the plug-in has to decide what to do with that 4th request. it means more book keeping on the plug-in side before it decides it ignores it or applies it. Not a problem, but I just feel that if the host were to simply address modulations per voice (like port/channel) instead of per note played it would have been more straight forward. Since the host already must have some book keeping towards which notes are playing under which port/channel, relieving the plug-in from having to do that.

The way I see it, modulation for a parameter is either global, per timbre (a group of voices) or per voice. Including the key or specific note ID just adds complexity. Not a problem though.

I need to start debugging to see how bitwig works.

[baconpaul]

Thank you Urs for your elaborate answer

The thing is even when using note ID to resolve all ambiguities, most if not all plug-ins are practically limited for a parameter mod by the number of voices that are playing. So if 3 voices are playing, and say all 3 cutoffs are being modulated. A 4th cutoff can’t possibly be modulated even if the host asks for it due to a repeated note. Which means now that the plug-in has to decide what to do with that 4th request. it means more book keeping on the plug-in side before it decides it ignores it or applies it. Not a problem, but I just feel that if the host were to simply address modulations per voice (like port/channel) instead of per note played it would have been more straight forward. Since the host already must have some book keeping towards which notes are playing under which port/channel, relieving the plug-in from having to do that.

The way I see it, modulation for a parameter is either global, per timbre (a group of voices) or per voice. Including the key or specific note ID just adds complexity. Not a problem though.

I need to start debugging to see how bitwig works.

So I think you have it a wee bit backwards. The host issues the note ids not the plugin. The plugin, if it works in note I’d space, has a responsibility to send an event back to the host when the voice for that note id is ended. So if you think of “note I’d” as “voice id” it becomes a bit clearer perhaps?

What’s happening with polyphony is the daw is getting access to the actual voice state of the synth. It can do it in two ways. It can assume that a pck uniquely identifies a voice, or it can assume each note on starts a voice with an id and wait for that voice to end. (Internally that voice may be a compound timbre - and in surge it in fact is - but the host doesn’t need to know that).

The “wait for voice to end” in id space requires a return message from the plug but allows essentially infinite polyphonic addressing which is why things like voice stack can work. The pck does not require the notification (a note off is the end of the voice) but doesn’t allow overlap or modulation in the release phase.

As far as book keeping, in surge adding note Id to the voice was just adding an int to our voice structure. Not that bad. And adding modulation was pretty natural since we already have a lot of it. And as you can see in the demo it’s also pretty small bit of code to just find the targeted voices.

[baconpaul]

If you are really curious here’s the diff where I first added mod and note expression support to surge. https://github.com/surge-synthesizer/su ... 17fd5153b6

Had a few bugs to clean up after that but as you can see it’s mostly threading data through the api and handling voice termination. Plus some tricky mono stuff and a test

[S0lo]

Thanks for the details @baconpaul

So I think you have it a wee bit backwards. The host issues the note ids not the plugin. The plugin, if it works in note I’d space,

Yeah, That part I knew.

has a responsibility to send an event back to the host when the voice for that note id is ended.

That I didn't know.

The “wait for voice to end” in id space requires a return message from the plug but allows essentially infinite polyphonic addressing which is why things like voice stack can work. The pck does not require the notification (a note off is the end of the voice) but doesn’t allow overlap or modulation in the release phase.

That bit pretty much makes PCK a bit lame. Every one will want note id then.

As far as book keeping, in surge adding note Id to the voice was just adding an int to our voice structure. Not that bad. And adding modulation was pretty natural since we already have a lot of it. And as you can see in the demo it’s also pretty small bit of code to just find the targeted voices.

It wasn't the additional int value I was worried about. It's the fact that note ID is arbitrarily chosen, not index-able, not addressable value. Which means that you have to do "search" (i.e the for loops in that piece of code). Now thats not an issue if you have say 5 voices with 5 modulations where the modulation are low frequency. But suppose you have 16 voices playing, with 25+ running automations per voice, each running at high rate LFOs of 100Hz+. This would be tasking. It's a search for each modulation incident. People might even want to modulate at audio rate sometimes and as I understand bitwig allows for external CV ? To avoid aliasing the modulation, it had to be run at double those rates.

Now sure you can use something like std::unordered_map to gain some performance indexing the note_id. But that uses internal trickery. nothing is really faster than a plain indexed array.

ps. I tried to disable _PER_NOTE_ID in that clap_saw code but bitwig didn't work per-voice when I tried that. Is there some thing else I need to do to try PCK in clap_saw ?

[EvilDragon]

PCK is there really for (as Urs already mentioned) MPE and MIDI/MIDI 2.0 support, and easier transitioning of existing projects to voice ID method (if it even fits into the instrument's internal voice management architecture). It is obviously not as flexible and clean as voice ID method.

[baconpaul]

Thanks for the details @baconpaul

Thanks for the questions!

That bit pretty much makes PCK a bit lame. Every one will want note id then.

I was very much in the 'lets put note id in the 1.0 spec' because it is way more powerful. But PCK isn't lame. It's just polyphonically capped and requires things like channel rotators. One of our explicit design criteria with CLAP has been to allow people to phase in features too. Start with the just simple "CLAP-as-a-VST2" approach of no mod, no polymod, etc... and then you can walk into features over time. So I wouldn't say "PCK" is lame - i would say it is *absolutely amazing* compared to what we had 4 weeks ago! Just note id is *super* absolutely amazing for high polyphony synths.

It wasn't the additional int value I was worried about. It's the fact that note ID is arbitrarily chosen, not index-able, not addressable value. Which means that you have to do "search" (i.e the for loops in that piece of code).

Yup. Surge has 128 max voices. A linear search over 128 voices is very small compared to the cost of, say, our filters - especially since modulation is control rate not audio rate - but indeed the approach we took would not work for audio rate high density polymod.

ps. I tried to disable _PER_NOTE_ID in that clap_saw code but bitwig didn't work per-voice when I tried that. Is there some thing else I need to do to try PCK in clap_saw ?

I think (think) you need to put a bitwig device up front which does channel rotation for you. But I've been in note id land since day one so am not *entirely* sure.

[Urs]

Psst... PCK also requires the Note_End message, but only 1 for all overlapping instances of the same port/channel/note.

[S0lo]

Ok I might have been harsh on PCK

So I looked further into PCK and the channel values are int16_t. Thats 2^16 values so it's way more than enough for anything. The host could just keep a stack of the currently used channel numbers from 0 to n. And when ever an note is off, remove n from the top of the stack and put it in that empty space (the new top will be n-1). This way the numbers don't have to be arbitrary, and plugin can easily index such a thing without search. A similar thing can be done to note_id ? or not ?

I think, the Key can be considered just additional information. Port/Channel is the ID. Actually, MPE does that. It associates a channel for the life time of every note. Knowing the Key doesn't add to the uniqueness of the Port/Channel. In other words, you can't have modulation p:1/c:1/k:c5 and modulation p:1/c:1/k:e5 playing at the same time. Not in MPE, I'm sure. The 2nd mod has to be p:1/c:2/k:e5. Or not ? Running out of channels that causes overlap is a different story. It's hardly going to happen with CLAP. It only happens with MIDI because of the 16 channel limitation. Sorry I'm just thinking out loud.

One thing worth mentioning here, is that detecting release times on the plugin side might sometime be very hard to impossible. Consider reverbs, or complex delay lines. Modulars with self evolving patches. So a question pops here, can the host just be allowed to keep sending modulations until the user him self stops it in his tracks? You know a user will know better when modulation should be stopped. He can hear the release. Thats actually how automation tracks work. The run time is manually limited by the user.

[EvilDragon]

In theory you can have that with MPE. Try setting the polyphony above 15 in Surge and you will eventually get events overlapping over channels, if they have long release times. MPE has no way of dealing with this, so whatever modulation happens on channel, say, 2, happens for all note events that are on that channel.

[mystran]

Now sure you can use something like std::unordered_map to gain some performance indexing the note_id. But that uses internal trickery. nothing is really faster than a plain indexed array.

std::unorder_map is not ideal, because it's an open table (ie. collisions chained into a linked list) and therefore has to allocate the nodes for said linked list.

For this type of thing a closed table (ie. collisions probes again for another slot) would be better. Unfortunately there isn't one in STL, but they aren't that hard to write yourself. These don't ever need to allocate anything as long as the load factor stays below some chosen maximum load factor and given that the reasonable maximum load factor is typically around 70% (eg. I think my own table resizes at 66%) you can just make the table 2-3 times the number of maximum voices you support and you'll be fine. There's some subtleties with the probing to avoid clustering if you suspect the host is going to be sending really nasty sequences of note IDs.. but mostly you should be able to get performance that's more or less O(1) lookup even if you had 1k voices.

[S0lo]

In theory you can have that with MPE. Try setting the polyphony above 15 in Surge and you will eventually get events overlapping over channelss, if they have long release times. MPE has no way of dealing with this, so whatever modulation happens on channel, say, 2, happens for all note events that are on that channel.

Oh yeah, that’s exactly what I meant. MPE will reuse the channels AFAIK when 16 is reached. The synth will not be able to differentiate between mods meant for this note or that note because they are all using the same channel. So the Key is not a differentiator. Even if we force it to be a differentiator, if these channel colliding voices happen to use the same note, then again the Key won’t help. Which is I believe what Urs and others have mentioned above.

If MPE is not in use, then the host can just use channels from 1 to 2^16. Again the Key is just additional information. It can be useful though.

[EvilDragon]

MPE reuses channels when 15 is reached - one channel is global and not supposed to take any voices.