Idea: WebAssembly for audio plug-ins

[Aleksey Vaneev]

This is a topic to discuss an idea of having audio plugins compiled and run under WebAssembly-alike conditions. This isn't exactly a narrow developer-specific discussion as it needs a wider awareness to kick-start.

Currently, WebAssembly is meant to run in web-browsers - a code compiled into pseudo-instructions in a general framework of Von Neumann architecture (contrary to e.g. Java byte-code which requires Java framework).

However, the underlying idea of WebAssembly is quite applicable to such hosted software as audio plugins. The obstacles I see now are both plugin specification-bound and hardware-bound: such WebAssembly plugins will require a subset of OS window event handling and drawing, file IO features implemented in plugin API, so probably no hardware-accelerated graphics will be available as that's too hardware-specific.

What's the purpose? As many of plugin users experienced recently, a lot of useful free plugins remained in the era of 32-bit computing. On MacOS, many plugins won't receive an ARM upgrade. In the future, if architectures and OSes undergo a serious overhaul, existing 64-bit plugins will turn into legacy or even non-functioning plugins. While WebAssembly makes it possible to run compiled software in a hardware-agnostic manner.

The general problem is, even if developer releases a free plugin in source code form, its further maintenance and recompilation is not guaranteed (due to disinterest of any follow-up developers, or due to awkward compilation process). If a commercial developer goes out of business, the maintenance will cease to exist as well.

Of course, WebAssembly software is slower, but audio plugins are usually not CPU-expensive either. Currently, WebAssembly runtimes are about 2.5 times slower compared to native compilation, and I think that's not bad at all since we have extremely powerful processors compared to 10 years ago.

Your opinions?

[Guillaume Piolat]

I think it's a worthwhile target, considering how it's used as conveniency VM by Zig , despite nothing mandated that. As an immediate plus for plugins, you could run as WAM/WAM2 in a web page as Iplug2 demonstrated.
Would be troublesome to target for us, we need to depend less on the language runtime to do so (WASM is not exactly the same as WASI). WASI has POSIX calls, but not `fread`. The C stdlib is just not included in WASM. On the other hand, half of the WASM runtimes are kinda abandoned already. Will it stick?

[Aleksey Vaneev]

I've read some details on WASM runtimes and compilers, while many of them were abandoned, some are going forward. Of course, in WASM model, it's not possible to rely on usual conveniences available in C/C++ compilers and Windows/macOS SDK, that's why some "common denominator" extensions will be necessary that route calls to OS level (not necessarily on plugin API level, but on runtime library level as well). Targeting both DAWs and web-browsers may be overly complex, or impossible as none of web-browsers will ever offer a way to read/write random files, or create OS-level popup windows.

[Guillaume Piolat]

Can also be useful as development tool also, for example we bundle a Wren interpreter in plugin in development, just for styling and tuning variables, but that interpreter needs a VM anyway for anything to run quickly enough. It is = compiler + VM interpreter.

[SamDi]

I would love to see WASM to replace this ugly Jacascript, but this don‘t seem to happen. And that‘s the problem. So WASM is and probably will stay a niche technologie and messing around with niche technologies is always PITA, because you have frameworks and ecosysystems which never reach a kind of stable state where a whole industry does implement maintainable SW on.

Furthermore I don‘t get your point really, TBH. From which WASM DAW you would like to start a WASM plugin? There just exists just not any WASM plugin standard AFAIK. Before I would consider WASM to implement any platform independent SW, I would prefer JAVA first, but even there you would be pretty lonely to find DAWs or plugin standards or a big ecosystem for that.

[diroxe7660]

If I'm understanding correctly, there is no SIMD support for WASM at this moment and Emscripten only supports POSIX threads?

[Cyoon]

just for fun, I asked chatGPT: how would I write an audio plugin like a vst in webassembly. can you give me an example?

It's answer is:

Writing an audio plugin like a VST (Virtual Studio Technology) in WebAssembly would involve several steps. Here is a general overview of the process:

• Write the audio processing code in C or C++. This code should implement the functionality of the plugin, such as filtering, modulation, or synthesis.

• Use a tool like Emscripten to convert the C/C++ code to WebAssembly. This will allow the code to run in a web browser.

• Create an HTML and JavaScript wrapper for the WebAssembly module. This wrapper will handle communication between the plugin and the host application, such as passing audio data and processing parameters.

• Use a library like Web Audio API to handle audio input and output in the browser.

• Finally, use the Web MIDI API to handle MIDI input and output if your plugin needs it.

• Here's an example of a simple WebAssembly plugin that applies a gain to audio input:

C++ code:

Code: Select all

#include <emscripten.h>

extern "C" {
  void EMSCRIPTEN_KEEPALIVE applyGain(float* input, float* output, int samples, float gain) {
    for (int i = 0; i < samples; i++) {
      output[i] = input[i] * gain;
    }
  }
}

JavaScript wrapper:

Code: Select all

const wasmModule = new WebAssembly.Module(bytes);
const wasmInstance = new WebAssembly.Instance(wasmModule);
const applyGain = wasmInstance.exports.applyGain;

const audioCtx = new AudioContext();

const input = audioCtx.createGain();
const output = audioCtx.createGain();

input.connect(output);

// Create an AudioWorkletNode to process audio in the audio thread
const processor = new AudioWorkletNode(audioCtx, 'gain-processor');

// Pass the input and output gains to the audio thread
processor.port.postMessage({ gain: 0.5 });

// Connect the input and output nodes to the processor
input.connect(processor);
processor.connect(output);

// Define the audio processing function
processor.port.onmessage = event => {
  const samples = event.data.samples;
  const inputBuffer = samples.input;
  const outputBuffer = samples.output;
  applyGain(inputBuffer.data, outputBuffer.data, inputBuffer.length, event.data.gain);
  processor.port.postMessage({ samples });
};

Please note, this is a simple example and may not be suitable for actual use in a real world scenario, as it may lack some important features that are required for a real-world audio plugin.

[wangeroge]

So the best idea is probably to wait for chatGPT 4 and let it write a new DAW in a few milliseconds.

[bungle]

Sounds like you want JS, it already exists in Reaper and is platform/hardware agnostic/future proof, just ask Justin to open source the JS VST, then just compile a wrapper for whatever platform/hardware you like, that will be much much much much easier than trying to convince the myriad of DAW developers to support a new format.

[MrBeagleton]

Reaper's JS has nothing to do with JavaScript.

[rafa1981]

Reaper JSFX was to be open sourced already. I don't know the current status.

[Aleksey Vaneev]

How difficult the idea to implement is well understandable, but how advantageous it seems?

JSFX is scripted from what I know, it's not the same thing. Most developers won't bother open-sourcing the algorithms. The same applies to JavaScript.