About CAT

[mystran]

I mean, if we're all just sharing ideas. I'm mostly just opposed to the "list of linear segments" approach. Too complicated, (almost) nobody implements it.

Right. I feel like a better approach would be just events with a smoothing interval (eg. number of samples). Even if you don't bother implementing the exact intervals, you can use the host provided value to guide your heuristics and most importantly it allows for the differentiation between "smooth" and "jump." Even if you use such a scheme to implement full "list of linear segments" it is usually easier to deal with a single list of events as opposed to a bunch of separate arrays for different parameters.

I should probably add that if the host generates an event for every 32 or 64 or whatever samples, then the actual nature of smoothing doesn't usually make a whole lot of difference either, but it is useful to explicitly know when the next event should be expected if the automation is supposed to be continuous.

[beginzone]

Is there a set of types that allows both approaches ("sending curves", alignment to transitions) to be in CAT? If I understand it correctly, the struct type I suggested is already too big and/or complicated. I think that double precision might be enough to get alignment, but the structure might just look plain wrong in your eyes, I get that.

The possible situation that bothers me, is a plugin that in principle cannot get data that it knows how to exactly interpret. This is also because I here do not distinguish between "automation" and "modular signal flow" (by which there is no separation of concerns, my fault).

If for example I want to automate the pitch or volume of a note to create a vibrato or tremolo, how do I as composer tell the plugin to do so? The reason I did not distinguish mentioned concerns, is that I do not yet know what set of "one instance" signal changes (aligned to transition, smoothly ramped if needed) and what set of "modern automation" there is. For the latter I think of VST3 and Bitwig (pitch slides). I don't know implementations either, but I have grasped the "advertisement" and was glad that the idea exists. I don't know how note expressions are implemented, but I think there must be a way to get quite exactly defined signals to the plugins.

I'm currently not knowing enough about "single instance" scenarios as I tried to paraphrase what you said (it's no good phrase, sorry). I can visualize how the points align to samples of transitions. But I don't know what the smoothed signal samples would be, which might end up not quite deterministic? Even a smoothing interval does not tell how the smoothed samples around the point of interest look like.

Can you elaborate on "heavyweight"? I do believe there's a situation possible that the struct I suggested is too big (in bytes) already, and too complicated if your needs (alignment to transition, etc) have to be considered. Different needs need different structures.

Consider my struct as being subject to "obsolete". I like the discussion, and I also think, it currently just ideas and there is quite some time for the plugin format to evolve.

[beginzone]

Because you say the goal is to get rid of out-of-resource problems for a better user experience.

Not just out-of-resource, but anything that degrades the real-time priority of the audio.

For instance, when loading a new plugin in a DAW host currently, there may be momentary pause in the audio. I would like to see that eliminated.

And when changing the sample rate of the host, or even the audio device, do the best to eliminate any interference.

I don't know if this of any help, but there is some idea. It's also in the FL Studio plugin format, probably in more formats. The resource heavy parts must be outside the "inner" instancing. It was the first thing I ever tried with VST2, which does not define it in the plugin format, but FL Studio does.

Although, when loading the DLL or calling "create the big one instance", that should probably be handled, too. In another thread? Another process (plus inter-process communication, unfortunately for the rest of the plugins lifetime)? Could it then still create a moment that blocks the main audio thread from going on? Are there specific reasons like file or memory procedures in the OS?

[matt42]

I think the inner/outer instancing could be useful to save time when the host scans its plugin directory. Where as to avoid audio glitches the host should never try to load a plugin in the audio thread. Which I'm pretty sure FL used to do. Its been a while since I tested in FL, but it was the only host, of the few I tested with, that I could reliably glitch by loading a plugin.

Also perhaps the plugin could instead just have an associated file with plugin data the host could scan without the need to load the plugin at all. I dunno just thinking out loud.

[matt42]

And when changing the sample rate of the host, or even the audio device, do the best to eliminate any interference.

I don't think there is any way you can do sample rate change on the fly and guarantee glitch free performance

[camsr]

And, hypothetically, if all cores were busy with other threads, loading/unloading the plugin may take time away from those. I would think that most of the work of instantiating a plugin could be defered until there is a timeslice available, and only loading small increments until then.

This idea may be a host-only concern, although keep in mind that there may be cases that the plugin will need to be managed by the host to accomplish tasks like this.

[mystran]

And when changing the sample rate of the host, or even the audio device, do the best to eliminate any interference.

I don't think there is any way you can do sample rate change on the fly and guarantee glitch free performance

Just not going to happen. Even the audio driver itself is probably not going to handle it gracefully most of the time.

[matt42]

And, hypothetically, if all cores were busy with other threads, loading/unloading the plugin may take time away from those.

You can have more threads than cores just make sure the audio threads are high enough priority.

I would think that most of the work of instantiating a plugin could be defered until there is a timeslice available, and only loading small increments until then.

This idea may be a host-only concern

This is what the thread scheduler does. I don't think the host should be reinventing this functionality

[camsr]

And when changing the sample rate of the host, or even the audio device, do the best to eliminate any interference.

I don't think there is any way you can do sample rate change on the fly and guarantee glitch free performance

Just not going to happen. Even the audio driver itself is probably not going to handle it gracefully most of the time.

This is true, but there's at least 4 modes of state change that can happen, while sample rate details are being sorted. What has the final say in this operation is vague.
1) Hard transition. Will result in discontinuities most of the time, and general user unhappiness
2) Fade transition. Will result in drops and rises to/from silence. Should not generate the very unpleasant discontinuities of a hard transition.
3) Overlap transition. Will result in constructive and destructive interference of the audio signal, due to any phase mismatch between signals of different sample rates. Somehow it may also spike CPU load due.
4) Continuous transition. WOULD result in a perfect signal, but the details are once again vague, and even I say esoteric.

If some audio device was even capable of switching sample rates without interruption, modes 3 and 4 might be feasible. So far, I look to at least support mode 2, with an undefined amount of time between fadeout and fadein.

[beginzone]

I mean, if we're all just sharing ideas. I'm mostly just opposed to the "list of linear segments" approach. Too complicated, (almost) nobody implements it.

Right. I feel like a better approach would be just events with a smoothing interval (eg. number of samples). Even if you don't bother implementing the exact intervals, you can use the host provided value to guide your heuristics and most importantly it allows for the differentiation between "smooth" and "jump." Even if you use such a scheme to implement full "list of linear segments" it is usually easier to deal with a single list of events as opposed to a bunch of separate arrays for different parameters.

I did not read your post good enough. It makes sense. Having what I called "duration of a segment" defined as "smoothing interval" works. The difference in your approach is that you suggest to implement it with a number of samples instead of a duration in seconds.

Both things could be used to tell the plugin "immediately, please". The duration in seconds could be more stable when the host wants to change samplerate? But I don't want to defend the floating point type. I like integers quite much.

I had thought you mean "smoothing interval" as a global setting, sorry. Probably you mean it's mentioned per each event.

Your event structure might look a bit like part of my automation_segments_t. You say, it's easier to deal with events than with a *list of lists* over time. I agree.

I think, in your posting "event" probably means also other types of events like notes? If for events you propose an approach that in the end resembles a similar thing that can be used for the same as my idea (I guess so), then there's no big difference somehow, and I'm all for it.

So, VST2 hosts send events in a structured way (a *bit of list-of-lists here, too, list of calls with list of events as argument, to say precisely). The structure contains a number of elements and a pointer to a (MIDI) event. And the list refers to the time of an audio block. The event.delta means number of samples.

I guess, the only thing that is *not* list-of-lists (and by that hard to use) is direct parameter change by dispatcher call. But it has the problem that there cannot be enough calls to the dispatcher. So VST2 hosts decided what smoothing interval they like with the parameter automation API (not event API, no problem there). So, easiness resulted a bit of complication here. VST2 hosts slice audio blocks to align parameter changes to notes, and maybe to control overall precision, too.

The latter host behaviour looks to me like a hack. I think it might be good to watch the easiness of direct changes in a dispatcher call, and the "realtime" part of this happening when you use the knobs of your MIDI keyboard. It's all important, but there has been a "hackish" approach by hosts.

May I suggest to make it a unified approach for all? Is there a plugin format to refer to here?

And if you had to choose one of both these VST2 API parts, which is it?

Anyway, because of slides,vibrato,tremolo/note expression there should be such an event type constant that the event represents the start of such a linear segment (smoothing interval as you say). So, just some enumeration constant added to VST2 and an expectation what a plugin should do with it.

But along with it should also go a destination. A note, an instance of a tone. So, I think the event should have to *option* to refer to a note-instance, shouldn't it? Is that maybe a core place of concern? I don't know how VST3 does it, but there must be something like this inside VST3, right? I'm still talking about note expressions, but also about a unified approach if that should be a good idea.

All properties of a unified approach (see non-unified approach in VST2):
1) Realtime-capable
2) Exact alignment
3) Curvature is defined exactly, also at samplerate precision if the musician wants it
4) No list of lists, not more things than needed
5) No completely different structure types for events of all kinds. Keep the abstract event type from VST.

[beginzone]

I would think that most of the work of instantiating a plugin could be defered until there is a timeslice available, and only loading small increments until then.

This reminds me of a game where you would do a lot of things when the player reaches the end of the level. You would do texture loading or garbage collection at this point, and it's indeed about available time slices.

I did not think about the problem that all cores might be busy, I have to admit.

That would indeed need a "free moment" at some time. Or maybe plugin loading in a low priority thread in the background? Like this:

When music plays + the user wants to load a plugin => loading with background priority thread
When no music plays + the user wants to load a plugin => loading with high priority thread

[camsr]

That is how I would describe it yes. I am not a threading guru, so don't blame me if I make faulty assumptions, but I am seeing it from the perspective that loading a plugin may involve large memory transfers. This may be the domain of DMA if it's going storage/ram and vise versa.

Plugin states should be built as fast as possible (with lower priority than audio processing), but not interrupt the audio processing OR the control (user interface) processing. Graphics may be the same or lower priority, not sure about it.

[beginzone]

Mystran, I had read a few things in the other thread and was inspired to spend the whole day creating a plugin format. In the evening I looked at your code and decided that everything about it is better than my try. I think, Opi as it stands on Github should really be a thing just as it is. I was a bit misguided in thinking "it just compiles". No, your prototype is actually complete.

One thing this whole afternoon brought me is the idea that:
1) The host might want to ask the plugin (via opcode) if it is okay with unserialized things (where indexes are being used), e.g. unserialized calls (from multiple threads) to the sound producing dispatcher. And the plugin format might distinguish if the plugin is rather a orchestra plugin that expects to be able to do channel (bus?) processing in parallel, or if it wants to do voice processing in parallel, or both.

2) I tried to find out what I actually tried to do and found that putting exactly as many automation signal samples into buffers as the host wants to receive audio signal signal samples is a pure approach. All of the slice-for-alignment and setParameter-methods, and also linear interpolation things, looked like ambiguous things that not everyone would accept and use (after I have read here).

Is it only me who find it hard to document, what VST2 allows for? I'm frustrated and will go on doing my own monolithic synth app, which would be open, and any binary interface like a plugin format would be superfluous. But It's *also* a frustrating thing to try to write a DAW instead of buying one or buying plugins.

But anyways, I hope I did not work against your approach, I should have read the huge header file instead of the .c file, sorry, if I was like being ignorant, I'm feeling like so.

[mystran]

2) I tried to find out what I actually tried to do and found that putting exactly as many automation signal samples into buffers as the host wants to receive audio signal signal samples is a pure approach. All of the slice-for-alignment and setParameter-methods, and also linear interpolation things, looked like ambiguous things that not everyone would accept and use (after I have read here).

The problem with sending full audio-rate automation is that it is memory bandwidth intensive if you have a lot of parameters automated and there is a good chance that a plugin isn't going to do it's modulation at full audio rate (but rather some lower modulation rate) for performance reasons anyway (eg. in some cases it is simply not possible to do so in real-time). At the same time, in most cases the actual automation data is very low-frequency, so the interpolation error that you end up when you only send events every 32 or 64 (or whatever) samples is negligible.

When people ask for "sample accurate automation" what they almost always means is that they want to align certain events exactly with the audio, not that they necessarily need exact automation for every sample. If you try to create something like a trance-gate with automation, then having the attacks align to the beats is important (at least within a dozen or so samples; what don't want to do is round them to the closest processing buffer boundary, which is typically several milliseconds if not longer), but you don't really need every sample to be exactly correct, as long as the general timing is there.

[beginzone]

Thanks. I didn't grasp it at first, and even didn't think about interpolation errors.

I understand now, that Opi can do what I wanted to have. A smoothFrames of 0 will set (snap) the parameter to the targetValue. And so, a second event with the *same* delta value but smoothFrames=10, would create a ramp from the first event's targetValue to the second event's targetValue.

That would in effect be like a "line segment". And the interpolator is not defined, but it will in virtually any case look like a line, because smoothFrames==0 probably clears the smoothing filter? And then there's nothing to make it go anywhere else than straight to the target value.

So, if there's nothing like "don't do it" in the Opi documentation, hosts can do it like so, or they could even use 32 events for 32 audio samples with smoothFrames=0 for a "heavyweight" solution, if the host developer is convinced that something can only be done like so.

I hope I didn't write nonsense. Thanks a lot for your patience.

PS: About the interpolator or filter. Is it going to determine how to reach targetValue exactly after smoothFrames frames, doing some calculus for this, or is it rather a system where targetValue is just feeded into the filter, but after smoothFrames "nothing has to be"?