CLAP: The New Audio Plug-in Standard (by U-he, Bitwig and others)

[robbert-vdh]

In contrast, with CLAP only the main clap_plugin and clap_host objects have query interfaces. And there is no lifetime management beyond creating a plugin instance and destroying it. In fact, on the plugin side there is zero lifetime management for host resources at all.

Yeah, that could be an advantage, not having to manage the reference count of an interface.
However, coders manage reference counts all the time, via smart pointers (e.g. std::shared_ptr). And smart pointers exist for COM too. So perhaps it's not such a big deal.

I agree that it shouldn't be an issue, but with VST3 I don't think I've come across a single DAW that implements it entirely correctly. The problem is that the model only works when both the plugin and the host actually use the internal reference counting for everything. When they don't (and allocate things on the stack, or use the equivalent of an std::unique_ptr) then things will blow up. Take the IEventList or IParamValueQueue pointers in ProcessData for instance (this is a one that hosts commonly get wrong, but I can name a dozen other places where hosts get it similarly wrong). If you interact with that object like you should be interacting, which is you'd use the SDK's included IPtr<T> smart pointers to increment the reference count by implicitly calling addRef() and then drop that IPtr<T> again (which implicitly calls release()), then in a lot of hosts that will cause the entire object to be deallocated. Which means that to be safe, you'll need to know exactly where to pretend that this reference count simply doesn't exist. And I've also seen hosts use the query interface on an object to get another interface pointer, and then call release() on the object they queried instead of the returned object when they're done. That only works if both objects share the same reference count and release() implementation. There are just so many things here that can go wrong, even though none of these things should go wrong.

[mystran]

For what it's worth, I basically never voluntarily use reference counting for anything anymore [edit: actually that's not quite true; I just realized I use it sometimes for deduplication, such as with glyph caches.. but that's low-level stuff] unless it saves me from having to serialize multi-threaded code (eg. sometimes std::shared_ptr is handy if you need to keep an object alive in another thread, but would prefer not to hold a mutex). In my experience, whenever you take something built around reference counting and redesign it so that reference counting becomes unnecessary, you'll end up with a much cleaner design.

To emphasize, I don't think the real problem with reference counting is having to adjust the counts (that's trivially solved with smart pointers, even if it's some pointless overhead), but rather the fact that it just encourages (and sometimes forces) overly complicated architecture.

[robbert-vdh]

It does, yes, which is probably why most host and plugin implementation chose to ignore it. But that in turn causes 'correct' implementations that use the API as intended to directly or indirectly cause crashes (usually use-after-frees by the host). Which is why I'm very glad that CLAP doesn't force you to use reference counting anywhere: the plugin needs to do zero lifetime management, and the host only needs to worry about creating and destroying plugin instances.

[mystran]

That's true.. but you don't need to go through the vtable at all if you instead just populate the extensions with pointers to regular (non-virtual) or static methods. Then it's just as efficient (just one indirect call) and if you really want to get fancy, you could probably populate the structures automatically with the help of some ATL-style CRTP.

I started thinking about this approach ... and came up with this sort of thing:

Code: Select all

    template <typename Plugin>
    struct ClapPlugin : clap_plugin
    {
        Plugin  plugin;
        
        ClapPlugin(const clap_host * hostPtr) : plugin(hostPtr)
        {
            desc                = &plugin.plug_desc;
            plugin_data         = 0;
            init                = _init;
            destroy             = _destroy;
            activate            = _activate;
            deactivate          = _deactivate;
            start_processing    = _start_processing;
            stop_processing     = _stop_processing;
            reset               = _reset;
            process             = _process;
            get_extension       = _get_extension;
            on_main_thread      = _on_main_thread;
        }

    private:
        static ClapPlugin * _cast(const clap_plugin *self)
        { return static_cast<ClapPlugin*>(const_cast<clap_plugin*>(self)); }
        
        static bool _init(const clap_plugin *self)
        { return _cast(self)->plugin.plug_init(); }
        
        static void _destroy(const clap_plugin *self)
        { delete _cast(self); }
        
        static bool _activate(const clap_plugin *self,
            double sr, uint32_t minf, uint32_t maxf)
        { return _cast(self)->plugin.plug_activate(sr, minf, maxf); }
        
        static void _deactivate(const clap_plugin *self)
        { _cast(self)->plugin.plug_deactivate(); }

        static bool _start_processing(const clap_plugin *self)
        { return _cast(self)->plugin.plug_start_processing(); }

        static void _stop_processing(const clap_plugin *self)
        { _cast(self)->plugin.plug_stop_processing(); }

        static void _reset(const clap_plugin *self)
        { _cast(self)->plugin.plug_reset(); }

        static clap_process_status _process(
            const clap_plugin *self, const clap_process * proc)
        { return _cast(self)->plugin.plug_process(proc); }

        static const void* _get_extension(const clap_plugin *self, const char * id)
        { return _cast(self)->plugin.plug_get_extension(id); }

        static void _on_main_thread(const clap_plugin *self)
        { _cast(self)->plugin.plug_on_main_thread(); }
    };

The idea then is that one implements a plugin like this:

Code: Select all

struct Test
{
    static clap_plugin_descriptor plug_desc;

    const clap_host * host;

    Test(const clap_host * host) : host(host) {}

    bool plug_init() { return true; }

    bool plug_activate(double, uint32_t, uint32_t) { return true; }

    void plug_deactivate() {}

    bool plug_start_processing() { return true; }

    bool plug_stop_processing() { return true; }

    void plug_reset() {}

    clap_process_status plug_process(const clap_process *)
    {
        return CLAP_PROCESS_CONTINUE;
    }

    void* plug_get_extension(const char *id) { return 0; }

    void plug_on_main_thread() {}
};

clap_plugin_descriptor Test::plug_desc =
{
    .id             = "org.example.test",
    .name           = "test",
    .vendor         = "",
    .url            = "",
    .manual_url     = "",
    .support_url    = "",
    .version        = "0",
    .description    = "test compilation",
    .features       = (const char *[]){
        CLAP_PLUGIN_FEATURE_AUDIO_EFFECT,
        0
    }
};

.. and then to actually create an instance, one does this:

Code: Select all

clap_plugin * create_test(const clap_host * host)
{
    return new ClapPlugin<Test>(host);
}

I like this concept, because it doesn't need any real vtables (although it'll still work if you have one), it's "typesafe" (with the sole exception of ClapPlugin<T>::_cast obviously), you can still have a base-class for the actual plugin type that provides default implementations (just don't make them virtual and a derived class implementation will hide the base class implementation)... and you except all the wrapper methods to either inline (which is what clang does with Test above) or compile into trivial tail-calls (ie. just simple jumps).

I think this can probably be extended to do extensions in an equally nice way... but the point I'm trying to make is that C++ can do a lot more than Java-style OOP and sometimes thinking outside the traditional OOP box will give you more or less equally convenient, but (in this case marginally) more efficient constructs.

[A_SN]

You had me at "C-only ABI" I always hated C++ and having to use it to make plugins (using WDL-OL). How does it work for plugins on macOS, do we still have to sign and notarise them?

[DRMR]

How does it work for plugins on macOS, do we still have to sign and notarise them?

As with all software on MacOS: yes.

[OBSOLETE160530]

[A_SN]

As with all software on MacOS: yes.

Damn shame, I thought maybe whoever makes a new plugin format in the 2020s would consider making a portable interpreted plugin format, so you could have a plugin that works on any platform, even on a WebAssembly host (I feel strongly that WebAssembly is the future that will allow me to soft-ditch macOS).

*actually..

If we have JIT for plugins that are compiled at runtime, such restriction can be bypassed.
So for example JSFX "plugins" used in REAPER that are actually just text files, can be installed without issues.

Only binaries need to be signed. If you dont use binaries, you dont need to sign anything

Yeah that's what I had in mind!

[DRMR]

*actually..

You still have to run them inside a program/plugin that is .. notarized.

CLAP describes a binary interface between .. binaries .. so yes notarization is required.

[A_SN]

You still have to run them inside a program/plugin that is .. notarized.

CLAP describes a binary interface between .. binaries .. so yes notarization is required.

I'm sure there's a way to make it work, it only takes a little bit of imagination. Looking at the plugin template I see mostly functions and then one function that fills up a struct with pointers to all the functions that make up the plugin's expected functionality, plus other filled out structs. It should be possible to make a JIT plugin format that does all this. Having this would be quite wonderful for having plugins that work on every platform in the same way. And perhaps more importantly it would be good for futureproofing, you'll find no shortage of people on KVR who hanged on to 32-bit for as long as they could because of abandonware plugins that weren't updated. I guarantee you that in 2038 you'll be happy to be able to run a CLAP plugin that hasn't been updated since 2024. If we go the native-only route then that won't be the case, mostly not with macOS, we'll just continue the cycle of everything breaking and requiring being updated every few years. The desktop software world isn't moving at a breakneck pace anymore so we have to think about how to keep things running at the scale of half-centuries.

[mystran]

How does it work for plugins on macOS, do we still have to sign and notarise them?

As with all software on MacOS: yes.

Well... except locally compiled software. I think the way it works is basically that when you download something it sets some "evil" bit in the filesystem and then refuses to load it if it's not properly signed and notarized.

If you locally compile something and don't sign it, then what seems to be happening is that the first time you run an application or load a plugin, it'll automatically add some sort of local signature to the bundle. Once the signature is there, it'll get checked as usual, so if you recompile you'll need to delete the old signature or it'll refuse to load... but in general you don't need to bother with signatures or notarization for local development, only for distribution.

[A_SN]

I think the way it works is basically that when you download something it sets some "evil" bit in the filesystem and then refuses to load it if it's not properly signed and notarized.

Yep, true, which means that when you set up the whole process you need a second machine (well the second machine is mainly to be sure about dependencies) to test the software and you need to download it over the Internet. I have two plugins I haven't updated since 2015 (they're unsigned because back then that wasn't a problem) so I just tell my users to sign it themselves locally and that works.

[OBSOLETE160530]

[mystran]

I came up with this sort of extension mechanism for the approach above:

Code: Select all

    template <typename Plugin>
    struct Clap_AudioPorts
    {
        static void * check(const char * id)
        { return (!strcmp(id, CLAP_EXT_AUDIO_PORTS)) ? (void*) &ext : 0; }

    private:
        static const clap_plugin_audio_ports ext;
        
        static ClapPlugin<Plugin> * _cast(const clap_plugin *self)
        { return ClapPlugin<Plugin>::_cast(self); }
        
        static uint32_t _count(const clap_plugin *self, bool is_input)
        { return _cast(self)->plugin.plug_audio_ports_count(is_input); }

        static bool _get(const clap_plugin *self,
            uint32_t index, bool is_input, clap_audio_port_info *info)
        { return _cast(self)->plugin.plug_audio_ports_get(index, is_input, info); }
    };

    template <typename Plugin>
    const clap_plugin_audio_ports Clap_AudioPorts<Plugin>::ext =
    {
        .count  = Clap_AudioPorts<Plugin>::_count,
        .get    = Clap_AudioPorts<Plugin>::_get,
    };

Then in Test plugin:

Code: Select all

    void* plug_get_extension(const char *id)
    {
        void * ext = Clap_AudioPorts<Test>::check(id);
        // if(!ext) ext = SomethingElse<Test>::check(id);
        return ext;
    }

    uint32_t plug_audio_ports_count(bool input) { return 0; }

    bool plug_audio_ports_get(
        uint32_t index, bool input, clap_audio_port_info * info)
    {
        return false;
    }

Here when we call Clap_AudioPorts<Test>::check() we'll triggers the template expansion, which then throws a compilation error from the expanded wrappers if the plugin didn't actually fully implement the extensions... and again nothing virtual anywhere, so everything gets inlined perfectly fine.

I think I'm seriously going to adopt this approach (though possibly with some of the logic moved to the wrappers directly)...

[A_SN]

Congrats to its creators, I wish all the best with their CLAP format, but I think this happened too late. In my opinion it was something which could have a chance something like 15 years ago, normalizing the multi-format "war".

I think this is a good time. VST 2.4 is ancient, last time I checked VST 3 failed to dethrone it, and everything else (AU and AAX, did I forget anything relevant?) is locked to something proprietary. So devs are happy to see CLAP and not dismissive because something new is sorely needed.

Not to be prophet of doom, but I think if it is based on the same "pre-compiled binaries" approach used until now, it may not have a future, unfortunately. But who knows...

The "pre-compiled plugins" approach made a great job in the last 20-25 years, but I think the future of pro audio / plugins is related to JIT.

Totally agree, this is a great time and opportunity to create a universal JIT plugin format and a bad time to rely on a dated approach of "one platform = one pre-compiled binary".

there's a reason that C/C++/Rust are still leading the way in performance when necessary.

I don't know about the practical aspects of accomplishing this, but compilers these days turn all code into an intermediary representation, then the IR is turned into another IR (same format) that is optimised depending on the target, then that IR is turned into whatever type of bytecode. In theory you could have plugins that represent an IR (or something that can be parsed by a LLVM parser), then the DAW could call CLAP's LLVM compiler which would compile the plugin IR into a binary, and cache the compilation for later.

This is what I do with my OpenCL GPGPU kernels. They're embedded in my program in plain text (but SPIR-V which is an IR is a possibility, and loading from a file is of course also an option) so at that point the whole thing is just a string in memory, it's compiled by the GPU vendor's OpenCL driver (which uses LLVM, although I seem to recall Nvidia using something different), then I myself cache the binary so that next time I run the program and nothing changed (I include a bunch of parameters into a hash and put the hash in the filename) I don't have to compile it again.

My point is, if you do it this way, with LLVM compiling an IR, then all you did was shift the same compilation from the dev's computer to the user's computer, and you get the advantage of compiling to their specific platform (so while the dev might only want to compile for up to SSE2 for maximum compatibility, the IR on the user's machine can be compiled to AVX-512 if that's what the machine can handle), so in that case the result might well be faster than running the dev's compiled binary.

I don't know if this approach would work with a WebAssembly host though.