About CLAP

[Urs]

Dear Juho,

which email-address did you send your request to? The correct one would be info@cleveraudio.org - which unfortunately is not handled by myself but by a team that is responsible to answer in a timely manner. If that has not happened, I would love to know, so we can remind the team to do so.

Thank you,

- Urs

[BlindMusician]

Hello all members discussing in this thread,
@baconpaul (to baconpaul): Thank you for your information about the standard! Finally, I would like to clarify to u-he (Urs Heckmann) that I tried to send my message of the accessibility to info@cleveraudio.org so I would like to hope that a team member from there also would answer to me. Even though CLAP would not have the way of controlling the accessibility as baconpaul said, the developers would be required to use the UTF-8 encoding and, also, there would be a mandatory statement of making all the plugin parameters accessible to the hosts. The Music Accessibility Standard (a link to that is on my previous post at page 56) would, therefore, be a better place for discussing about the topic, so we would continue the discussion there. Even though I have limited resources as an ICT student, I would like to take part in the discussion as my time allows and develop the groundwork for the first stable version of Music Accessibility Standard.
Kind regards,
Juho Tuomainen

[mystran]

Even though CLAP would not have the way of controlling the accessibility as baconpaul said, the developers would be required to use the UTF-8 encoding and, also, there would be a mandatory statement of making all the plugin parameters accessible to the hosts.

The thing is that CLAP is an open standard and there are no strict requirements (beyond what is required to interoperate), only specifications on how you're supposed to implement things. Unless you give up the whole open "free as in freedom" nature of the standard, any "mandatory requirement" will never amount to anything more than a strong recommendation. This is a fundamental property of any truly open standard, for better or worse.

[baconpaul]

Even though CLAP would not have the way of controlling the accessibility as baconpaul said, the developers would be required to use the UTF-8 encoding and, also, there would be a mandatory statement of making all the plugin parameters accessible to the hosts.

The thing is that CLAP is an open standard and there are no strict requirements (beyond what is required to interoperate), only specifications on how you're supposed to implement things. Unless you give up the whole open "free as in freedom" nature of the standard, any "mandatory requirement" will never amount to anything more than a strong recommendation. This is a fundamental property of any truly open standard, for better or worse.

mystran: that is exactly correct in the sense of 'what you choose to build', but as you know the spec does mandate some things, like UTF-8 encoded strings everywhere, a model for labeling parameters and ports (if you have them) etc...

blind musician: CLAP requires you to use valid UTF-8 for all your parameter information and requires you to describe it to any host which you expose it to.

The idea that "all" parameters should be exposed to the host is not a good one; there's plenty of parameters you just can't expose really (for instance, the surge modulation matrix is not host exposed and would be very difficult to do so in most standards and most daws since it is a dynamic vector of tuples; samplers dont expose their samples at the edges for automation (although clap does have an extension for this)).

While some plugins can expose all their parameters to automation, and so work well with parameter readers like Osara and so forth, many others can't. The work of making these complicated non-parameter gestures accessible is hard work, and it falls on the plugin author at a level above the plugin-spec. It's simply not a problem CLAP (or any other standard) can solve in the way we have laid out the interop separation of concerns.

(Although just a plug: JUCE has done a great job making these higher level abstractions work in a cross platform way, and its what we used in surge to great effect)

[mystran]

Even though CLAP would not have the way of controlling the accessibility as baconpaul said, the developers would be required to use the UTF-8 encoding and, also, there would be a mandatory statement of making all the plugin parameters accessible to the hosts.

The thing is that CLAP is an open standard and there are no strict requirements (beyond what is required to interoperate), only specifications on how you're supposed to implement things. Unless you give up the whole open "free as in freedom" nature of the standard, any "mandatory requirement" will never amount to anything more than a strong recommendation. This is a fundamental property of any truly open standard, for better or worse.

mystran: that is exactly correct in the sense of 'what you choose to build', but as you know the spec does mandate some things, like UTF-8 encoded strings everywhere, a model for labeling parameters and ports (if you have them) etc...

For all intents are purposes these are still really just "strong recommendations" because given the license these things cannot be enforced, at least not beyond technical detection of misbehaviour on the API level and refusing to co-operate.

With regards to exposing all parameters in particular, such a requirement would serve no purpose, because failure to comply would have no effect on technical interoperability (it would not even really be detectable to the host) and (as far as I can tell) there is by design no central authority that would be in a position to enforce such requirements. In a non-open standard you could make such requirements part of the licensing agreement if you so desired, but with an open standard with a free license this is simply not an option.

So in a sense, from the legal point of view, there is no reason you can't have a plugin with strings using an encoding other than UTF-8. Doing so would make little sense as such strings would then fail to work correctly in a host expecting UTF-8, but as far as I can tell there is no central authority that could revoke your CLAP license if you choose to do this anyway.

The point I'm trying to make here is that trying to enforce something like accessibility by bolting it onto a free, open standard as a "mandatory requirement" is simply not useful. I'd imagine most developers consider accessibility a desirable feature, but like any other feature it does come with an implementation cost and I simply fail to see how listing something like this as an unenforceable "mandatory requirement" would have any meaningful effect on anyone decision making on whether to support it. Such a "requirement" here is simply the wrong tool for the job.

Actual recommendations on how various interactions should be implemented and how things should be organized could probably be quite useful for those considering or looking to support accessibility. Such recommendations and guides could both help people with no previous experience in such things to actually understand what they are trying to implement and also help provide consistency especially in non-visual interfaces. I have actually looked into this topic somewhat in order to add some support into my own toolkit, yet the available documentation to help one actually support these things is simply terrible to the point that it is probably the biggest obstacle to wide-spread accessibility support. A lot could be done, but I simply think unenforceable "mandatory requirements" are not going to have the desired effect.

[baconpaul]

Yeah that’s right. The validator tells you if your plugin will work. But you can write one that doesn’t

I was thinking about this some overnight and I think the right approach to accessible synths is to not have a mandate or standard but rather use a certification approach. Have a set of tests which you apply to a plugin and give it a score 1-5 with 5 being most accessible and 1 being least. Publish the rubric. And only publish 345 and communicate 12 only to the devs. That sort of approach has worked well in other fields where you want to layer a market force on top of optionality.

We shouldn’t hijack this thread for that idea but I’ll drop this comment in the surge discord accessibility channel too if folks want to brainstorm.

[syntonica]

I haven't done a deep dive into CLAP yet, but isn't there a contract (explicit or implied) between host and plugin? Meaning host offers a minimum number of services, plugin offers the same, and they negotiate on anything else?

I'm not where this concern about public vs private parameters comes from.

[ghettosynth]

I haven't done a deep dive into CLAP yet, but isn't there a contract (explicit or implied) between host and plugin? Meaning host offers a minimum number of services, plugin offers the same, and they negotiate on anything else?

I'm not where this concern about public vs private parameters comes from.

I don't know, but, do disabled musicians, specifically blind, rely on their host to know and engage with plugin parameters? If so, I can see why they would want every parameter relevant to interacting with the instrument to be reported to the host.

[syntonica]

I haven't done a deep dive into CLAP yet, but isn't there a contract (explicit or implied) between host and plugin? Meaning host offers a minimum number of services, plugin offers the same, and they negotiate on anything else?

I'm not where this concern about public vs private parameters comes from.

I don't know, but, do disabled musicians, specifically blind, rely on their host to know and engage with plugin parameters? If so, I can see why they would want every parameter relevant to interacting with the instrument to be reported to the host.

Ah, I see. Unfortunately, not every value can or should be automated and by that fact, not made public to the host. Perhaps every string and value should be available to a reader? I don"t have any experience with them. I just try to make sure the contrast on my GUIs is sufficient to make them discernible to those with impaired vision.

That may be a useful extension to CLAP in that case.

[Urs]

Some parameters such as drawable waveforms can be edited in a visual editor, but they can't be automated. To do so would require thousands of parameters to be published to the host.

[MrJubbly]

Why no Bazille?

Very nice news!

And Zebra 2, hopefully!

... and please somehow make 'automatic-migration' a real possibility within these plugins and DAWs, from 'VST/VST3 to CLAP'.

It would make transferring existing projects away from Steinberg's proprietary stranglehold, over to this new open standard, SO much easier and far less of a hassle for end users.

[baconpaul]

I haven't done a deep dive into CLAP yet, but isn't there a contract (explicit or implied) between host and plugin? Meaning host offers a minimum number of services, plugin offers the same, and they negotiate on anything else?

I'm not where this concern about public vs private parameters comes from.

I don't know, but, do disabled musicians, specifically blind, rely on their host to know and engage with plugin parameters? If so, I can see why they would want every parameter relevant to interacting with the instrument to be reported to the host.

Ah, I see. Unfortunately, not every value can or should be automated and by that fact, not made public to the host. Perhaps every string and value should be available to a reader? I don"t have any experience with them. I just try to make sure the contrast on my GUIs is sufficient to make them discernible to those with impaired vision.

That may be a useful extension to CLAP in that case.

Writing an accessible synth is a huge amount of work. We did it in surge for almost every feature and it was months of work with a team of musicians who knew screen readers. It’s quite a lot harder than just show all the strings to the screen reader. You really are making a gui which is navigated in a totally different way.

Parameters with well defined names and typeins and proper formatting are an important part of it. Clap has that in the spec for things you expose as parameters. But some other things aren’t params and then you have to worry about an awful lot. Juce Makes this doable. But like any other ui work, it still requires thought about workflow albeit with a totally different tool set.

But the clap api for the gui is miles below this. The way the screen readers work have to do with the low level ui features on the os. The clap gui api hands you an hwnd and let’s you do whatever; windows lers you populate it with components which expose themselves to readers or not. But you don’t want an api at claps layer intermediating that.

If you are interested in doing this for your synth though we did a load on surge. Happy to chat or introduce you to the other team members who designed and tested it with me.

[syntonica]

Writing an accessible synth is a huge amount of work.

I imagine it is. It's bad enough just handling UTF8 vs ASCII. I was rather tired when I wrote that reply, but I was thinking of an extension that queried strings and values from the plugin GUI and passed them on to the OS screen reader (or whoever is receiving.) I'm assuming this would all be a push operation based on the current cursor location.

I would think that if your GUI stuck to system widgets, this operation would be free, but since most plugin GUIs are custom from top to bottom, it would indeed be quite a mess.

As I look at it harder, it seems this isn't really a CLAP issue but a graphics-library issue and while CLAP might be able to provide some pointers and recommendations, it's not really its bailiwick.

[mystran]

Writing an accessible synth is a huge amount of work.

I imagine it is. It's bad enough just handling UTF8 vs ASCII. I was rather tired when I wrote that reply, but I was thinking of an extension that queried strings and values from the plugin GUI and passed them on to the OS screen reader (or whoever is receiving.) I'm assuming this would all be a push operation based on the current cursor location.

I would think that if your GUI stuck to system widgets, this operation would be free, but since most plugin GUIs are custom from top to bottom, it would indeed be quite a mess.

This presentation (assuming I got the right one?) about accessibility in JUCE gives some good insight into the whole thing: https://www.youtube.com/watch?v=BqrEv4ApH3U

[baconpaul]

I was thinking of an extension that queried strings and values from the plugin GUI and passed them on to the OS screen reader (or whoever is receiving.) I'm assuming this would all be a push operation based on the current cursor location.

That’s not really how the os apis work. Objects have accesibility handlers which are way more fully featured than just a string which interact with the windows messages on win and the nsobject layer on Mac.

I would think that if your GUI stuck to system widgets, this operation would be free, but since most plugin GUIs are custom from top to bottom, it would indeed be quite a mess.

This is correct. Juce makes it easier by abstracting the things you need for custom widgets but you still have to do things to handle this alternate ux including labels and options if you don’t use the built in widgets, which no one does.

As I look at it harder, it seems this isn't really a CLAP issue but a graphics-library issue

That is what I’ve been saying from the start

It is exactly that. Among the wide spread graphics libraries used in audio development today, only juce supports screen readers. So basically if you want an accessible gui (1) be single platform and use native widgets, (2) use juce or (3) write a lot of code. But all that code is outside clap. (You can do well with accesibility with just params as mentioned, but not for every product)