About CAT

DSP, Plugin and Host development discussion.
Post Reply New Topic
RELATED
PRODUCTS

Post

Regarding the Community Audio Technology format, I would like to write down thoughts and feelings I have. I hope CAT goes on. Sorry, I don't want to "go on against" the will of Vertion, when she has locked the thread. But as I am interested in technical things and in things where community plays a role in getting "audio work" done.

From my perspective the pace of development on the Github page could be a bit slower. I believe even the plugin format design and raw description would get done because of long term interest, like a year, not until end of July.

GainExample.c has the .c ending, and I assume it was also tried - in some way - to provide definition of the C ABI, C binary interface for the plugin format. But there is C++ code inside the C file. I'm not sure what this means. Is this a way to provide the usual 'extern "C"' parts by using the .c extension, or something like that?

The binary interface "in C" would be a bit easier to understand if it was going on with C, wouldn't it? C++ among all other languages should be encouraged for sure, but when OpiPluginEntrypoint() returns "new OpiGain", which is derived from OpiPlugin, I get the feeling that there might be the need to deal with a class instance, even if it is currently a POD. Okay, sorry, if I'm just saying here that it scares me instead of really finding "a bug". I trust you, it's C ABI, and also OpiPluginEntrypoint is being an unmangled name in the DLL and is easy to access, but it could be made more clear that it certainly is. I was a bit like "Comment me through the C++ please", I'm leaning towards newer languages, am unskilled in C++.

I agree on many points or suggestions Vokbuz made initially about the plugin interface. E.g. I do see the possibility that the plugin should, at least, be allowed to allocate memory itself. It might be that using memory allocated by the host is not as easy to convert to the memory model of the language used by the plugin developer.
E.g., if you have chosen Golang as your C++ replacement, you would need to convert such host memory to a slice by using Go's "unsafe" package, possibly losing some guarantees about memory safety, later when Go should change. It's currently easy to use host memory, but using the unsafe package actually means, you're formally no longer in vanilla-Go so to say.
A "it does it for you - babysitting" done by the host is in general a good thing, because plugin developers want to reuse stuff made for them, I guess, but the so called babysitting maybe would have to stop at this point where it comes to memory allocation.

I believe separation of concerns and making the features being very orthogonal would be good. If there is e.g. memory allocation done by the host, it should not mean the plugin has to use it. And the other things should as independent as possible, too.

And, as far as I understand the request that was made, I believe providing thread-safe communication - if added to the plugin format - is a thing that should try to not interfere with the inner signal format. If there is a universal way then it should be considered, but I believe there isn't such a thing that is easy enough. Multi-threading-safety actively managed by the host with some sort of communication channels might be really interesting. But in a newer language you have it's own multi-threading tool-set, and both things might become hard to combine, and you might want to continue using e.g. Golang goroutines, but then there is the channels way the host provides. It's a conflict, isn't it?

I hope the format, if it goes on to be developed, becomes minimal and orthogonal. And the topics automation and separation of audio and display should be discussed longer. My personal belief is that the current market of plugins is dependent on a "audio-graphical", monolithical way, and I have not really liked how controlling plugins is done when you make music, nor do I think that it's the one right way extend host-features that way. Audio should be decoupled, e.g. automation at samplerate should also allow automating pitch, making the plugin - if it agrees - unaware of the notes being played. The plugin would accept the data the stands for the pitch slides and vibrato I like to use when composing.

Post

beginzone wrote: Thu Jul 09, 2020 1:58 pm GainExample.c has the .c ending, and I assume it was also tried - in some way - to provide definition of the C ABI, C binary interface for the plugin format. But there is C++ code inside the C file. I'm not sure what this means. Is this a way to provide the usual 'extern "C"' parts by using the .c extension, or something like that?
It has .c extension by mistake.

Post

I just wrote a simple example of a possible API (in C, because specifying a binary API in anything else is just retarded) and a simple example plugin (in C++) as a starting point for discussion.

The thing is, if you want to get something done, you need to propose something concrete, something that you can throw at a compiler and something you can implement and test in practice. You expect to throw away at least a couple of dozen revisions and you might even replace the whole thing with a better proposal, but if you want something that doesn't communicate using XML over CORBA, with special considerations for IP over avian carries, you need to just write something and see if it works, if there are any pain points and what are the things that people feel are missing.

Post

mystran wrote: Thu Jul 09, 2020 4:16 pm I just wrote a simple example of a possible API (in C, because specifying a binary API in anything else is just retarded) and a simple example plugin (in C++) as a starting point for discussion.
Do you mean you wrote the header and example c file on github, or did you write another one?

I must admit I did not see that the extern C declarations are #included, I have used too much Go lately. The format and example is alright, someone might add documentation that explains it a bit here and there. For folks like me. But now I guess the CAT/Opi project is paused.

I have just seen that Vertion is banned. I'm under the impression that the misunderstanding has been she thought SOlo tried to say "no such community", like adding to the critique that was alreay there (xkcd standards argument meme, no willing community for a new standard). How unfortunate that "none such" reads like "no such". I'm from Germany and have my problems with English, but I think maybe it's all just a temporary issue.

I have been reading KVR for quite some time, also posted, and there have already been proposals comparable to CAT. I was pretty sure I would have no use for a plugin format (I see it as some sort of a bottleneck), but when I yesterday noticed my personal DAW project todo list has exploded, I thought the idea to talk about worksharing in a technical sense (one person writes a host, another person somewhere else writes a plugin) is a good one. I usually like everyone of these ideas, and I'm - depending what CAT actually could become - interested in the whole thing. But it depends.

Thanks for replying!

Post

mystran wrote: Thu Jul 09, 2020 4:16 pm The thing is, if you want to get something done, you need to propose something concrete, something that you can throw at a compiler and something you can implement and test in practice. You expect to throw away at least a couple of dozen revisions and you might even replace the whole thing with a better proposal, but if you want something that doesn't communicate using XML over CORBA, with special considerations for IP over avian carries, you need to just write something and see if it works, if there are any pain points and what are the things that people feel are missing.
Oh, hehe, I understand. Yeah, it's also the thing where one starts to understand how it has to start and might start thinking more about it. I did that. I would like a situation where I could read on KVR about very "orthogonal" features in that standard. But as I'm not a any skilled (standard) designer by no means, I tend to just read what you all say. All I know is my last project and how it compares to the CAT idea.

And I'm sometimes a little bit frustrated by DRM and my personal issues with mouse usage and how computer things are in general. I saw that Vertion mentioned some mild critique on Steinberg and Apple, so I thought, something that would be freely licensed and still standardized, would be nice. And some order (atomic components and some minimalism) in the design would be good.

Post

beginzone wrote: Thu Jul 09, 2020 4:39 pm
mystran wrote: Thu Jul 09, 2020 4:16 pm I just wrote a simple example of a possible API (in C, because specifying a binary API in anything else is just retarded) and a simple example plugin (in C++) as a starting point for discussion.
Do you mean you wrote the header and example c file on github, or did you write another one?
I wrote the API "header" into a forum post. Then wrote the example gain plugin in another forum post. Vertion put them on github, even though my main intention was to get people to disagree on something concrete, rather than to just endlessly debate high-level details that you can't throw at the compiler.

Post

Formally, threading and memory are implemented by the OS, not any particular programming language. Any lack of functionality regarding these things is a limitation of the language.

Plugins are designed for heavy workloads, much of the time. In some cases they can also use quite a lot of data, the most extreme of that I know is loading giant preset banks into RAM for fast switching/browsing.

The host is responsible for making it all come together to do what we want it to do. Control is exerted over plugins through the host. There are times when the only thing we want, is to run the transport and not have any technical glitches. During that time, we do not want plugins making weird decisions about using system resources. This is a real-time priority scenario.

Then there are times when the user is making decisions about making sounds. This may not be a real-time priority scenario. There may be plenty of CPU time available to do other things the system needs to do to manage itself.

Plugins doing resource-intense things, while not being under control of the host, is a conflict to managing these scenarios. In reality the user wants everything to work smoothly. In a fragmented software-ecosystem like the current, the user has little control of the interoperation of most programs. In a scenario like making music or recording audio, this could lead to audio buffer underruns, and these are generally unacceptable for many reasons.

A new spec may be able to make the system run more efficiently, and also defer heavy system-management related workloads to times when the user is not expecting real-time audio priority.

Post

Ok, I also want to suggest an idea with code. Like, with VST2 it has been no pleasure and there have been tricks and hacks. I did touch VST3 and landed in COM hell (tried to use it via C ABI). And that's what made me stay high level when writing about CAT. But if I try to make no mistake, I would say the types are the thing of importance for me. How the control of the plugin is "typed".

Code: Select all

// Can automate anything, at samplerate or slower (plugin has to interpolate linearly).
struct linear_automation_segments_t { 

	int number_of_segments;
	floating_point_number_t segment_duration_in_seconds;
	
	 // range of a number = [0..1]
	 
	 // example of a list: 0.f .. 0.3f .. 0.7f .. 0.4f
	 // The first element in the array is the first number to be interpreted.
	 // The last element in the array is the last number to be interpreted
	 // The last element is *not* the last start of a segment.
	 // There is never an open segment (without endpoint).
	 
	floating_point_number_t *pointer_to_first_segment;
}
Depending on linear_automation_segments_t.segment_duration_in_seconds the segment could also represent only a sample as in just a plain old samples buffer. There's no linear interpolation done then. Like this, a minimal set of types would be there for two usage scenarios, but actually for a range of scenarios from "samples" to "long linear segments".

Is this an agreeable thing? I feel, since some people point out the automation topic quickly, it could be a core part of the design. Is it an atomic thing that might be part of CAT or is there already a mistake in it?

I mean, would you say this is good, or is it better to have a separate struct type for a buffer of samples? CAT would probably repeat itself at "number_of_elements" (number_of_segments) and at "element_duration_in_seconds".

For the latter you could assume it's just "at samplerate". But I, for a first try, like this struct and not much beside. And linear interpolation has some property that enables it - with many segments - to represent many different curvature types, which is also used in graphics rendering, e.g. the core of Antigrain. When it's not sufficient to have linear segments, you scale it from "coarseness" to "at samplerate", all with this single structure.

Post

camsr wrote: Thu Jul 09, 2020 5:31 pm Formally, threading and memory are implemented by the OS, not any particular programming language. Any lack of functionality regarding these things is a limitation of the language.
I agree on all points you made below and find a host as a "capable host" very interesting. But, just for example, Golang presents itself to some extent as a language for fast code and could sit between some even faster code and the plugin host. Go has its mantra for stability and provides goroutines, channels. For someone who chooses this to improve response times and user experience in general, they still might see themselves in a conflict, even if Go "should formally not wrap the resource management" so to say.

The runtime is tightly integrated into the language and talks to the OS. So, it provides functionality. But it has no functionality to resolve this conflict indeed.

As a concrete example: If I do streaming from disk in Go and double my samples buffer just in time, Go will do that in a stable fashion and handle possible race conditions in a good way (race detector, clean modern language). If, when reading from a file, for doubling the memory my plugin asks the host for more memory, then CAT should at least contain help for the user of the modern/memory-managed language. Because you say the goal is to get rid of out-of-resource problems for a better user experience. Then the plugin *must* allocate all it's memory via the host, at some point. In some way, without enforcing it there would be no point in an allocator.

This help from the side of CAT would have to be a working example. But then other languages could become relevant too, and I'd like to point out that "forcing to use an allocator in a host" is a thing that looks to me like something that is to blame, if the help from CAT should not be enough.

Post

The problem with these heavyweight sample-accurate automation schemes that involve linear segments is that nobody implements them. The VST3 example code doesn't provide an implementation of the sample-accurate ramps. JUCE doesn't use either VST3 or AU sample-accurate ramps.

Sure, there's a handful of people who go out of their way to design their parameter abstraction in such a way that they can make perfect, full use of the sample accuracy, but not only are those instances the exception rather than the rule, I actually can find no code for implementing it properly. CoreAudioUtilityClasses I guess, but you have to subscribe to their entire framework to get that, and the whole thing's a mess.

Everybody has parameter smoothing in their plugins anyway. The best option is to have sample-accurate discrete parameter changes, and the parameter change has a flag for whether the change should be considered "instant/fast" or not. Smooth the parameter faster (or just disable smoothing) for that particular point.
owner/operator LHI Audio

Post

wrl wrote: Thu Jul 09, 2020 6:26 pm Smooth the parameter faster
This will lead to clicks in some cases. Generally even ramping changes should be smoothed inside for sensitive parameters.

Post

Vokbuz wrote: Thu Jul 09, 2020 6:33 pmThis will lead to clicks in some cases. Generally even ramping changes should be smoothed inside for sensitive parameters.
Oh, absolutely it will. Still, I've had this specific feature request several times now. People really want to sync up their automation points to transients, so in my opinion it's better to give them the option so that the smoothing on the non-fast points can stay slower.

I mean, if we're all just sharing ideas. I'm mostly just opposed to the "list of linear segments" approach. Too complicated, (almost) nobody implements it.
owner/operator LHI Audio

Post

+sub.
www.solostuff.net
Advice is heavy. So don’t send it like a mountain.

Post

wrl wrote: Thu Jul 09, 2020 6:38 pm Still, I've had this specific feature request several times now.
It does not mean they would really enjoy it if it is there :) I saw this so many times.

Post

beginzone wrote: Thu Jul 09, 2020 6:21 pm Because you say the goal is to get rid of out-of-resource problems for a better user experience.
Not just out-of-resource, but anything that degrades the real-time priority of the audio.

For instance, when loading a new plugin in a DAW host currently, there may be momentary pause in the audio. I would like to see that eliminated.

And when changing the sample rate of the host, or even the audio device, do the best to eliminate any interference.

Post Reply

Return to “DSP and Plugin Development”