Pious: polyphonic modular plugin framework

[avasopht]

I am beginning to create a polyphonic modular plugin framework (empty github with readme.md).

The purpose is to provide a platform independent poly-modular audio processing framework that can support VSTs, Rack Extensions and Audio Units (and possibly AAX) as well as the Linux plugin systems. The framework allows you to create small or large modules that you can connect together to create composite modules. Composite modules can contain composite modules also, allowing users to create plugins with no code, but also allowing programmers and non programmers to work together in creating new devices rapidly.

On top of that this system separates the DSP from the UI and I will be providing a GUI framework that is powerful, scriptable and easy to use.

I've prototyped the concept enough now and am going to be publishing regularly under an MIT license.

The structure of composite modules is listed below. DSP modules that contain code do not have a DEVICES and CONNECTIONS section.

Code: Select all

/* JSON like structure.
 *  GLOBAL_DEVICES = [
 *    UNIT: {
 *      ID,
 *      port_type*: [ // e.g. audio_inputs
 *        PORT_ID // e.g. "audio_in"
 *      ],
 *      DEVICES: [
 *        UNIT_ID, // ID of the unit
 *        REF_ID, // how this device will be referenced in the connections section
 *        ALIASES: [
 *          [original, alias] // e.g. port named "out" renamed to "gr_out"
 *        ]
 *      ],
 *      CONNECTIONS: [
 *        {
 *          FIRST: [ "path", "to_port" ], // e.g. ["gain_device", "out"]
 *          SECOND: [ "path/to_port" ] // e.g. ["rc_device/audio_in"]
 *        }
 *      ]
 *    }
 *  ]
 */

[avasopht]

(From this thread) This of course sounds really cool, but also like Reaktor (or the core system, anyway). Some dynamic API for something like this could be very cool, but again I believe Reaktor gains magnitudes of speed by statically analyzing the flow graph of the dsp objects and coalescing them together and optimizing them (which of course requires the graph to be non-dynamic and static).

Well I'm thinking more on the lines of a Reason Cominator but maybe slightly lower level. They place complete devices such as synths, compressors and EQs inside and connect them in creative ways to create multi band compressors, sound generators and even more complex synths like a B3 with the Rotary speaker.

Bear in mind that these devices in my system can either be your own compiled plugin, or a composite device, so it's operating on a level somewhere between Reaktor and Reason's Combinator with the option to operate higher or lower. Low level modular design as you say will need to be optimized. It's not a focus but I'm open to it.

With this system you can do really cool things like create a synth and allow the user to replace your filter with one of their own. Or even allow users to insert their own chain of effects your voice voice bank.

On top of that it just makes it simpler to experiment and try out different arrangements really quickly, which was my main motivation, but then I realized it opened up the doors to so much more.

[-Fluky-]

How will this be different from:

http://audulus.com/

or

http://ddmf.eu/metaplugin-chainer-vst-a ... x-wrapper/

or

http://www.sensomusic.org/

[avasopht]

Good question.

1. Allows platform independent c/c++ to write plugins with future API support. As soon as we support a platform, your code will support it too (when sticking with the portable C/C++ subset)
2. Undo API to communicate with DAWs
3. Compatible with Reason Rack Extensions (DSP wise, GUI communication would need to adhere to Rack Extension specification)
4. Save / load state management with user library database
5. Open source
6. Real-time scripting for prototyping in the pipeline.
7. Hyper modularity: write a plugin, providing a port for users to connect an effects chain, which will be processed as if it were within the device, with polyphonic support.
8. More features to be introduced in due time.

There are many similarities and this does also have the potential to power a DAW (or Workstation plugin), which is part of the vision but I want to create a much more lightweight solution - I'm not aiming to reinvent the wheel. My background includes developing gui frameworks and system / asset management for embedded systems as well as creating scripting VM's (with garbage collection, etc.). I have a grand vision in my mind, but for the time being, I am focusing on getting this little helper out in the open.

The initial motivation / inspiration for me is to facilitate my own development and make my own life easier, but in the process I can also add value to others.

Thanks for the links, I'll investigate them further.

[-Fluky-]

I would like a framework though that would combine what the "visual patchers" do with a proper and user-extensible SDK. Such that one could develop plug-ins by combining e.g. C++ and visual patching.

Many have tried something like this, but nothing really works well enough to facilitate e.g. plug-in development. E.g. Reaktor patches are still Reaktor patches.

Usine is pretty good, but it's not aimed for plug-in development per se, but live performance.

[Miles1981]

In some way, pious seems to be a ensemble on top of something like ATK. Just like ATK, you want to have a pipeline, units connected to each other, so why not building on top of it?
There is no C/C++ common subset anymore, so I would suggest targeting C++ directly: faster and more options especially in terms of objects (you want to connect objects, so it makes sense to use an object-oriented language).

The Json interface is really deadly close to what I'm doing to connect my objects in terms of mechanisms.

[avasopht]

That is very much the end goal, visual patcher combined with C++, but first I want to make sure that the framework behind this is as concise as possible. So yes, with this what you create will be its own plugin.

You can even go as far as designing your device using visual patching with placeholder effects to create code that just needs you to supply a DSP init and process method. In the init you bind your inputs and outputs, and then you can go ape with your algorithms. You can use C, C++ or anything that can be transpiled to those languages.

Knowing what people like yourself would need will help me prioritize and also sculpt the feature set.

[avasopht]

In some way, pious seems to be a ensemble on top of something like ATK. Just like ATK, you want to have a pipeline, units connected to each other, so why not building on top of it?
There is no C/C++ common subset anymore, so I would suggest targeting C++ directly: faster and more options especially in terms of objects (you want to connect objects, so it makes sense to use an object-oriented language).

The Json interface is really deadly close to what I'm doing to connect my objects in terms of mechanisms.

Yes it sort of is (not sure why I didn't notice AudioTK before, you started it around the same time I started dabbling with this and I'm pretty sure I've checked it out (maybe I misperceived it as providing only DSP code)).

Building on top of ATK is a real possibility. My initial objective was facilitating the development of Rack Extensions and rapidly prototyping and constructing my plugins. I really like Propellerhead's vision for future proof platform independent plugins that are packaged in LLVM bytecode to be later compiled for other processors and OS's.

The limitations of their SDK however allow no direct OS access, assembly / processor dependent code or static variables. I can create a static to scoped transpiler though. That being said, if this isn't compatible with Rack Extensions I can still create a minimal version for that platform.

I'll take a proper look at ATK to confirm compatibility, would be great to contribute functionality to another project.

[Miles1981]

Yes it sort of is (not sure why I didn't notice AudioTK before, you started it around the same time I started dabbling with this and I'm pretty sure I've checked it out (maybe I misperceived it as providing only DSP code)).

I started it almost 3 years ago IIRC, and yes, mainly DSL code at the moment because I haven't had the time or the knowledge to work more on MIDI interfaces, but the idea was there in my mind, just never could make it happen :/

[Mayae]

Just to clear it up: This will both be a development framework in an isolated ecosystem, but also a modular patching environment?

Anyway, I'm really excited to see how this will turn out and formalize itself. I'll be following.

[-Fluky-]

Here's also something interesting around this genre:
http://www.axoloti.com/

[camsr]

It's important to keep things as simple as possible regarding memory structure. This is where programming languages like C have an advantage, once the structure and layout is defined, it becomes a matter of how it's used. Use as little as possible but keep the possibilities open.
Just some insight into the problem, will follow this project as well.

[-Fluky-]

Also wouldn't this be something that could base on the already good WDL-OL framework?

[soundmodel]

Also,

http://integra.io/integralive/

Which happens to be an open source project:
https://github.com/BirminghamConservatoire/IntegraLive

That could be extended. In one thread somewhere it was requested that if it would support VSTs.

[Miles1981]

Also,

http://integra.io/integralive/

Which happens to be an open source project:
https://github.com/BirminghamConservatoire/IntegraLive

That could be extended. In one thread somewhere it was requested that if it would support VSTs.

It's a GPL project, so there are some limitations there. Also based on pd and can't be made a plugin (lots and lots of static modules instantiated and thus complete redesign required to make them into... Reaktor).
But really neat.