Multiple instances and memory space ( VST 2.4 )...

DSP, Plugin and Host development discussion.
RELATED
PRODUCTS

Post

I am in the planning stage of a plug-in that I'm developing and I have a question regarding shared memory...

In the host that I am testing with, if I load in two instances of the same plug-in, they exist in the same memory space. They share the same global variables, etc.

Here is my sample code...

int globalInstaceCount = 0;

AudioEffect* createEffectInstance (audioMasterCallback audioMaster)
{
globalInstaceCount++;

return new MyVSTPlugin (audioMaster);
}

When I load the second instance, globalInstaceCount is increased to 2. Awesome...

Is this true for ANY host, or should I assume that some hosts may not exhibit this behavior? My plugin needs to be aware of other instances and share data as a part of it's design. If this is not the behavior of all major host applications, then I'll have to implement another method of sharing information between them.

cpu

Post

I think you could also use a static variable to do the same.

Post

What surprises you? This is the usual behavior. Once the dll is loaded in memory, the static variables will be shared by all instances, this is IT 101. It's just not possible to map in memory two different instances of the same methods, as they have the same name. How would the OS tell the difference between which one it has to load?

camsr> it's actually a global variable, as such a static variable if it was declared in a class. Even if it were declared as a static variable (static per file), it would still be the same address, so it would be a global variable as well (the definition of a global variable being exactly a variable declared in a global scope).

Post

It doesn't surprises me. I understand why it's like that. However, I thought that it may be possible for some hosts to "sandbox" the DLL into it's own memory space, or into another process, etc. I don't want to assume that every host will behave the same way.

I want to be sure to design it in a way to work on all hosts, so I'm just being thorough :) .

cpu

Post

cpu.running.hot wrote:It doesn't surprises me. I understand why it's like that. However, I thought that it may be possible for some hosts to "sandbox" the DLL into it's own memory space, or into another process, etc. I don't want to assume that every host will behave the same way.
Yeah, a host could sandbox plugins if it wanted to (and that's how bit-bridges work, though sometimes it's available otherwise as well). Another thing is whether the host puts all the instances of the same plugin in the same process. It might do that even when bridging (or might not; who knows).

I use global variables (shared between instances of the same plugin) mostly as a form of memory optimization, to share read-only lookup-tables and such, where it doesn't really matter in terms of correctness if they are shared or not (except for some wasted space if not), so .. can't give any specifics of specific host behavior.

OTOH, be careful about making other assumptions even if this one happens to work in major hosts (no idea if it does). There's no reason to believe that all your plugin instances are processing the same block for example (since hosts can process some audio paths ahead), etc ..

Post

On windows I have had some success using shared mapped memory via: CreateFileMapping and MapViewOfFile. That is probably the safest method but statics seem to work fine. Another approach is using network protocols - this seems to be what Rayzoon are doing to communicate between JAmstix and their upcoming Jambass or whatever it is called

Post

Keith99 wrote:On windows I have had some success using shared mapped memory via: CreateFileMapping and MapViewOfFile. That is probably the safest method but statics seem to work fine. Another approach is using network protocols - this seems to be what Rayzoon are doing to communicate between JAmstix and their upcoming Jambass or whatever it is called
For communicating between multiple processes, either shared memory or networking certainly work (and the basic feature set is roughly the same on any platform really).

That said, given the "choice" between the two, I'd always use networking (not necessarily TCP/IP though; named unix sockets are a valid approach too), since it's by far easier to work with (basic client/server setup especially is pretty easy to get right). Shared memory is kinda like shared state in multi-threaded programs: just say no unless you really have no choice. ;)

If you need the low-overhead bulk-bandwidth that shared memory can give you, I'd still use networking as the main "control" protocol and just use shared memory for bulk transfer (eg. copy the large data block into shared segment, then send a message "your data is now here").

Post


Post

So far, by the responses here, I'm guessing that most if not all hosts will exhibit the before mentioned behavior. I'll have to test it on the major hosts to be sure of course.

It seams that all instances of the same plugin will share the same stack and heap.

Just to be thorough, I'll ask the question in reverse... are there any hosts that do not recognize the static and/or global variables between instances of the same plugin? Has anybody had this experience?

cpu

( I'm 95% sure now the answer is no )

Post

As mytran said, the only case is if the plugins are sandboxed. That happens if you are using a bridge IIRC.
Also, I think I remember reading something with AU and Apple wanting to sandbox the extensions/plugins. But I may be mistaken. Otherwise, it's always the same process and the same memory space (that's why 32bits was not enough in a DAW).

Post

Sandboxing on OSX involves something more like a chroot for a particular application. So it applies to the host and all processes/threads associated with the host application, including all plugins.

"Sandboxing" in this case means mostly giving limited permissions to the application to access parts of the system including the user's files. Think of it like this: the user launches the host, the os creates a "host" account and launches the host under that account. That account has no permission to access anything until it is specifically given that permission. This is accomplished via file dialogs and similar which the user then must use manually to grant permission. In other words the host does not have the ability to press "okay" on the dialog, the user must do this. This was a major issue for any developers who were using custom dialogs such as those built in to the "preset browser" in a plugin - that no longer works. (Solution: use a single archive of your preset database and initially authorize access to this file with a dialog or special permission associated with the binary. The more complex you want the more complex you can have... a dialog is most simple. My solution, although it may not be acceptable for others was simply to use the sandbox version of the database. A dialog is then not needed except to import/export the database. When you launch inside a sandbox instance the user must initially import the database or start from scratch with an init database.)

As for whether Apple ever planned on isolating processes associated with individual instances of AU plugins, that would be great if they did do that although I've never heard this was planned.

OP:

The issues you're having understanding how processes are allocated memory (including code and data of various types) are not an issue of plugins or hosts. These are basic programming issues. You need to take a step back and read a book on programming first, then worry about plugins. You'll find there is no reason to distinguish "plugins" in the majority of cases.
Free plug-ins for Windows, MacOS and Linux. Xhip Synthesizer v8.0 and Xhip Effects Bundle v6.7.
The coder's credo: We believe our work is neither clever nor difficult; it is done because we thought it would be easy.
Work less; get more done.

Post

Sandboxing exists on OSX for browser extensions, but I didn't talk about the facilities provided by the OS directly. It wouldn't surprise me if the sandbox developed for Safari would be reused somewhere else.
I don't think it's really that great an idea, as there are less risks than with a browser extension, if the DAW is run with a limited account.

Post

Lets' not get into a debate regarding OS X sandboxing apps. I was using it in the generic sense, as in to "put the plugin in an isolated environment". Sandboxing techniques can range from limiting permissions, all the way up to running a process in a virtual machine.

As Miles1981 said, case where the plugins are bridged was my initial red flag and hence this post. I am currently testing on FL Studio, and it bridges the plugins, most notably in the case where you are using a 32 bit plugin on the 64 bit host. In this case, however, both instances of the plugin are being run in the same process, so there is no problem.

I just need to be sure that two instances of the same plugin will share the same stack and heap ( in most or all hosts )
cpu

Post

cpu.running.hot wrote:share the same stack and heap ( in most or all hosts )
Any design involving sharing stacks is doomed to fail. :)

Post

There is no debate, I was pointing out the only occurrence of the term "sand boxing" I've heard from Apple has been referring to the permissions issue.
I just need to be sure that two instances of the same plugin will share the same stack and heap ( in most or all hosts )
cpu
You're not able to do this unless you control the entire system. If you control only the plugin you need to create a program that functions regardless of the underlying system's configuration.

You know that malloc can return null, right? Same thing.
Free plug-ins for Windows, MacOS and Linux. Xhip Synthesizer v8.0 and Xhip Effects Bundle v6.7.
The coder's credo: We believe our work is neither clever nor difficult; it is done because we thought it would be easy.
Work less; get more done.

Post Reply

Return to “DSP and Plugin Development”