CLAP: The New Audio Plug-in Standard (by U-he, Bitwig and others)

[S0lo]

Right if you are making polyphony in hardware in audio and not in cv then you have a very hard time detecting voice end. Or really even defining a voice at all.

I'm sorry I don't follow

[mystran]

2. The plugin doesn't have to search for the voice_id

Oh come on... it's really not that bad...

Well, yeah. Unless you have a barrage of voices and modulators PER VOICES. Ok, it's arguable

If you're seriously worried about linear search, then use a closed hashtable. Unless you have hundreds of thousands of events per second, the lookups aren't going to be very meaningful expense, because on average a hash lookup really isn't that much more expensive than a regular array lookup (eg. when using something strings as keys, the bigger expense tends to be hashing the string).

[baconpaul]

Right a one block delay
If your block is 256 that means your amortized voice on envelope will be 128 samples late
That is way too much

Right, let me ask a question. Is the voice_id completely useless from a host perspective, if it arrives late ?

Yes

[S0lo]

Right a one block delay
If your block is 256 that means your amortized voice on envelope will be 128 samples late
That is way too much

Right, let me ask a question. Is the voice_id completely useless from a host perspective, if it arrives late ?

Yes

In that case, I agree that there no point of it.

[Jeff McClintock]

Right, let me ask a question. Is the voice_id completely useless from a host perspective, if it arrives late ?

A plugin-assigned voice_id is completely useless if you can't send the voice any modulation till "some unspecified future time" (depending on block-size).

plus - how many ways do you need to identify a voice anyhow?

You've already got key/channel (aka 'PCK'). You've also got 'note_id'. Which somewhat overlaps the functionality of PCK. And now someone proposed a third way? 'physical' voice number? This is feeling very confused.

1. A keyboard has many keys. (aka 'virtual voices')
2. An instrument has fewer real voices. (aka 'physical voices')

Someone has to create an unambiguous mapping between the two.
That someone can be the plugin (traditionally), or it can be the DAW.

The object doing the mapping needs to know the number of 'physical' voices (how many voices can truly play at the same time).

I understand the CLAP 'note_id' to be what I call the 'virtual voice ID' i.e. DAW wants to play a note, the DAW generates a note_id, and it's up to the instrument to allocate some physical voice. From then on, the two are mapped. the DAW can modulate the physical voice via the 'note_id'. There is absolutely no need for the DAW to care what physical voice is allocated because 'note_id' unambiguously maps to the physical voice. Once mapped, until the note ends or whatever, they are the 'same thing'.
I classify CLAP note_id as a type of "Virtual Voice ID" because it's allocated by the DAW.

Alternate scenario: DAW manages physical voices.
In this case the DAW needs to know up-front how many physical voices exist. And the DAW can trigger sound on any physical voice directly, via a (DAW chosen) 'voice_id' (key/channel, note_id all become secondary). In this scenario, an instrument with 16 physical voices would accept 16 voice_ids (0->15), and it's up to the DAW to cycle/steal/reuse them as needed.

The important takeaway is: In either scenario, you need only one identifier to uniquely identify a voice. Don't overcomplicate this with three different ways of doing the same thing.

[S0lo]

The important takeaway is: In either scenario, you need only one identifier to uniquely identify a voice

True, but that’s from the point of view of the plugin. The note_id (or PCK without any additional assumptions) is not sufficient to identify the voice from the point of view of the host.

Now you could argue that the host doesn’t need to identify the voice. Or you could argue that it’s useless if it arrives late. Or that it’s an over complication. Valid arguments. I’m not stubbornly trying to convince any one. The hell with the voice_id. Bad idea (ie. sending it late). My bad

[Jeff McClintock]

True, but that’s from the point of view of the plugin. The note_id (or PCK without any additional assumptions) is not sufficient to identify the voice from the point of view of the host.

I would aim for a scenario where the note_id does uniquely map to exactly one physical voice.
CLAP already added note_id to the model to account for PCK not being unique enough. Adding more complexity would be a mistake.

[SLiC]

I don't know if this has been mentioned before, but Io think it would be useful if a CLAP could have a small thumbnail of the plugin embedded so that DAWS that show a picture of the VST (in browser or track) can us it without having to take a snapshot. I know its just eye candy and not for everyone, but it would be a 'nice to have'

[] Peter:H []

I thought it's an interesting exercise to reason what a unit of work is and what the ID, aka Note_ID represents.

When I think about device chain building, i.e. the chain starts with something the creates sound (aka a synth) followed by a seperate EQ followed by a seperate Distortion (Voice --> EQ --> Dist) and all participants in this chain work "per voice", then how would you orchestrate that for instance in terms of minimal CPU core switches?
Like this way:
<Voice[NoteID1], Voice[NoteID2], Voice[NoteID3]> --> <EQ[NoteID1], EQ[NoteID2], EQ[NoteID3]> --> <Dist[NoteID1], Dist[NoteID2], Dist[NoteID3]>
Or like this way:
<Voice[NoteID1]-->EQ[NoteID1]-->Dist[NoteID1]>, <Voice[NoteID2]-->EQ[NoteID2]-->Dist[NoteID2]>, <Voice[NoteID3]-->EQ[NoteID3]-->Dist[NoteID3]>

<> denotes a sub unit of work of the chain, probably fed in to the worker pool.
[] denotes NoteID index...
A --> B action B follows A

What would be the most efficient way in terms of CPU Core Switches / Cache misses?

BTW: To be able to do this sort of chaining the Host must be able to begin/end units of work.

What's interesting is - when all plugins that are chained together and "voiced" like I imagined, they should have the same capability, i.e. if EQ can handle up to 3 voices and Dist only 2 but Voice can handel an arbitrary ammount the chain overall can only handle 2 ...
And this kind of per voice chaining is interesting when modulation comes in to play. plugins might change behaviour by modulation, i.e. "latency" if a EQ band is switched of or on depending on the incoming modulation ... (Bitwig EQ+)
...
Funny is btw, that that each participant in the chain might do it's own up and downsampling.
...
Sorry for my brain doing exercises...I know CLAP has other goals.
---
BTW: Just skimming through VST3 ... they use a noteID as well attached to their Events, i.e. NoteEvent and NoteExpressionValueEvent. Seems like a common succesful pattern.
So let's say Bitwig sends 5 NoteEvents, pitch is A4, tuning is from -20cents to 20 cents in 10 cents steps and gives them ids A4_1 up to A4_5 ...

[mystran]

Installed Bitwig demo to see if the clap-wrapper I wrote a while ago would work... and it doesn't seem like Bitwig wants to send me any MIDI? The same synth happily plays stuff in MultiTrack Studio (loaded as CLAP) so... I'm puzzled? That wrapper only advertises DIALECT_MIDI.

edit: Ok.. so after some debugging and figuring out where my debug prints go (redirected to engine.log) ... it looks like Bitwig is calling note_ports->count twice (once for input, once for output) .. but it literally never calls note_ports->get to actually fetch the info.. and just blindly assumes DIALECT_CLAP is fine..

[Jeff McClintock]

Is there a specific reason why clap separates automation from modulation? I mean if the final value of the param is the sum of both, Then why can’t we just rely on the host to give us that final summed value?

Yeah, It's clear that the DAW knows what the preset is, and knows what all the modulation sources are. Why make all the plugins do the summing?

This is a smart insight.

A plugin Processors job is to transform the parameters instantaneous values into an audio stream. Imagine how much simpler and cleaner a plugins process code would be in your scenario. Plus one could just delete and simplify some of the CLAP API, while retaining 100% of the functionality.

CLAPS retort is - because then the Processor wouldn't know the state of the preset (before the modulation was applied).

But why is it the Processor that needs to manage the state of the preset? It can do its job just fine by knowing only the instantaneous values of the parameters.

Sure, something needs to keep track of the preset state. But there is no reason that has to be the audio Processor part of the plugin.
The DAW already holds a copy of the preset. That copy is already kept in sync with the plugin at all times. That state can easily be made available to the plugin and its GUI.
Because when an API duplicates the preset state in two places (the DAW and the Processor), it's kinda redundant and violates the 'single-responsibility' principle of software design.

It is too late to redesign CLAP of course. But it's an interesting discussion to have.

[S0lo]

CLAPS retort is - because then the Processor wouldn't know the state of the preset (before the modulation was applied).

But why is it the Processor that needs to manage the state of the preset? It can do its job just fine by knowing only the instantaneous values of the parameters.

Sure, something needs to keep track of the preset state. But there is no reason that has to be the audio Processor part of the plugin.
The DAW already holds a copy of the preset. That copy is already kept in sync with the plugin at all times. That state can easily be made available to the plugin and its GUI.
Because when an API duplicates the preset state in two places (the DAW and the Processor), it's kinda redundant and violates the 'single-responsibility' principle of software design.

I'll try to state the pros and cons as far as I understand them and see where it goes from there.

Pros:
1. Preset state (or automation) is reflected on the GUI while modulation is not. This allows the user to use say a hardware controller to tweak the state (automation) by hand, while not being visually confused by the modulation that the host is doing concurrently. This is most useful with alive performance.

2. If the user saves the state of the plugin that he accidentally stumbled upon in the midst of a performance. It will not be affected by whatever modulation the host is doing.

3. Host modulation may allow some types of modulation that the plugin is not capable of.

4. Host modulation helps the user to quickly modulate any plugin without having the knowledge of how to operate that plugin's mod matrix.

5.....?

Cons:
1. Most plugins already have some form of internal modulation capabilities that don't change the state of the preset. So, is another layer from the host really adding something?

2. Both automation and modulation (AFAIK) can be done poly-phonically. This is a pro, but it also complicates matters further.

3. Some plugins may support only automation, others may support only modulation. Some may support both. Host vendors, Be prepared for customer complaints like "Why is modulation not working?" , "Why is automation not working?". "Whats the difference between this and that?". etc.

ps. I'm not with, or against. I take CLAP as it is and try to understand it.

[mystran]

Is there a specific reason why clap separates automation from modulation? I mean if the final value of the param is the sum of both, Then why can’t we just rely on the host to give us that final summed value?

Yeah, It's clear that the DAW knows what the preset is, and knows what all the modulation sources are. Why make all the plugins do the summing?

Because the plugin might want to show the base value as the knob position on the GUI?

Sure, something needs to keep track of the preset state. But there is no reason that has to be the audio Processor part of the plugin.

As far as the host is concerned, there is no "processor" in CLAP. There's just a plugin.. and this is one of the reasons just about everyone likes CLAP a whole lot more than they like any of the dozen APIs that try to enforce some sort of separation, because plugin developers want to write plugins rather than "processors" and "editors." Certainly most of us have some internal concept of the two, but it's no damn host business to worry about it.

[baconpaul]

Is there a specific reason why clap separates automation from modulation? I mean if the final value of the param is the sum of both, Then why can’t we just rely on the host to give us that final summed value?

Yeah, It's clear that the DAW knows what the preset is, and knows what all the modulation sources are. Why make all the plugins do the summing?

I'm glad you understand there are two values

Since the user wants to set either value from the DAW (sometimes adjust the preset; sometimes transiently modulate), you need two messages clearly, also.

If the messages are PRESET_VALUE or MODULATED_PRESET_VALUE vs PRESET_VALUE and MODULATION_VALUE seems to be a choice of sending (a, a+b) in one hand or (a, b) in the other.

Since, as I mentioned, most electronic instruments have a concept of a knob and a modulation overlay, it seems (a,b) is more natural and it is the choice CLAP made. But it would be trivial to write a CLAP which converted to (a,a+b) with a custom event type in a different space if you wanted.

[baconpaul]

1. Most plugins already have some form of internal modulation capabilities that don't change the state of the preset. So, is another layer from the host really adding something?

If you read the other KVR Clap forums you can find lots of places where people can accomplish things using the augmented modulators they can't with the internal mod matrix. With the clap saw demo, for instance, i didn't even code a filter envelope I just made the cutoff and resonance poly-mod.

3. Some plugins may support only automation, others may support only modulation. Some may support both. Host vendors, Be prepared for customer complaints like "Why is modulation not working?" , "Why is automation not working?". "Whats the difference between this and that?". etc.

This is why the CLAP spec requires a plugin to advertise exactly what it supports on a per parameter basis. A host which allows a user to configure polymod on a param which doesn't support it has a bug.