Login / Register  0 items | $0.00 New#KVRDeals

Web Audio Modules (WAMs)

jariseon
KVRer
 
6 posts since 31 Jan, 2015

Postby jariseon; Wed Nov 22, 2017 6:58 am Web Audio Modules (WAMs)

Hi!

We have been experimenting with Web Audio Modules (WAMs), which are the equivalent of VST plugins for web. The current version is targeting WebAssembly and AudioWorklets, and our goal is to come up with a community controlled open API which is thin, yet extensible to different use cases. Would be great to hear your feedback and suggestions! Links to the community site and some early synth demos are below.

Please note that the demos require Chrome canary 64.0.3275 or later.

https://webaudiomodules.org
webOBXD
webDEXED

Thanks!
Jari and Oli
stratum
KVRian
 
1360 posts since 29 May, 2012

Postby stratum; Wed Nov 22, 2017 7:16 am Re: Web Audio Modules (WAMs)

SDL+emscripten targetting webassembly also works on Firefox in addition to Chrome, I'm not sure if it's suitable for real time audio, though (It was OK for a video player).
What do you use to receive MIDI events?

Thanks
Last edited by stratum on Wed Nov 22, 2017 7:20 am, edited 1 time in total.
~stratum~
hibrasil
KVRian
 
726 posts since 23 Jun, 2002, from Huddersfield, UK

Postby hibrasil; Wed Nov 22, 2017 7:20 am Re: Web Audio Modules (WAMs)

firefox nightly does not have the audioworklet (yet), so the DSP is done in the script processor node, on the main thread which gets pretty nasty. We use the web midi api, which may also only be working in chrome so far (not sure)

https://webaudio.github.io/web-midi-api/
Last edited by hibrasil on Wed Nov 22, 2017 7:21 am, edited 1 time in total.
stratum
KVRian
 
1360 posts since 29 May, 2012

Postby stratum; Wed Nov 22, 2017 7:21 am Re: Web Audio Modules (WAMs)

firefox nightly does not have the audioworklet (yet), so the DSP is done in the script processor node, on the main thread which gets pretty nasty.


Yes, but can handle a lot of workload nevertheless.. it can decode video realtime (on a desktop/laptop cpu that is, hopeless on a tablet)
~stratum~
hibrasil
KVRian
 
726 posts since 23 Jun, 2002, from Huddersfield, UK

Postby hibrasil; Wed Nov 22, 2017 7:23 am Re: Web Audio Modules (WAMs)

yes but you asked if it was suitable for real-time audio!

here are v-0-1 WAMs using SPN and ASM.js for comparison. there is more latency and things get glitchy if the page needs to redraw anything etc.

http://www.webaudiomodules.org/wamsynths/
jariseon
KVRer
 
6 posts since 31 Jan, 2015

Postby jariseon; Wed Nov 22, 2017 7:24 am Re: Web Audio Modules (WAMs)

Still I think we could fall back on ScriptProcessorNode if AudioWorklet is unsupported. Chrome has webMIDI, and if I've understood correctly, it is coming to FireFox as well.
stratum
KVRian
 
1360 posts since 29 May, 2012

Postby stratum; Wed Nov 22, 2017 7:28 am Re: Web Audio Modules (WAMs)

yes but you asked if it was suitable for real-time audio!


That's true, a video player isn't really under much stress in terms of latency. (I had not realized that you had answered that question of mine, so it seemed like an independent post at first)

BTW - ASM.js is very slow compared to webassembly running on a recent browser, don't know why, it's supposed to be the same thing except the code download format.

edit: Both of the synths you have posted for comparison seem to be running OK on Firefox (including the ASM.js version) but while running on Firefox they do not receive midi events. On Chrome ASM.js version has audible glitches as you mention.
~stratum~
hibrasil
KVRian
 
726 posts since 23 Jun, 2002, from Huddersfield, UK

Postby hibrasil; Wed Nov 22, 2017 7:41 am Re: Web Audio Modules (WAMs)

Image

Image
Last edited by hibrasil on Wed Nov 22, 2017 8:09 am, edited 1 time in total.
stratum
KVRian
 
1360 posts since 29 May, 2012

Postby stratum; Wed Nov 22, 2017 7:43 am Re: Web Audio Modules (WAMs)

I had meant this one: http://www.webaudiomodules.org/wamsynths/webcz101/ and this http://www.webaudiomodules.org/wamsynths/webdx7/

Both of these work on Firefox but ASM.js version glitches on Chrome.

edit : Perhaps I have misunderstood these sentences:

Alpha version based on WAM API v0-1 using Script Processor Node and ASM.js.
This is a WAM port of a closed source commercial plug-in available to buy as a VST/AU/AAX at Plug-in Boutique

(This one has a problem with Chrome 62.0.3202.94 and doesn't warn that a newer version is required)

edit2: Both of these have problems with Chrome 62.0.3202.94. Anyway, sorry for the unnecessary confusion.
~stratum~
hibrasil
KVRian
 
726 posts since 23 Jun, 2002, from Huddersfield, UK

Postby hibrasil; Wed Nov 22, 2017 7:57 am Re: Web Audio Modules (WAMs)

thanks for your feedback, this is bleeding edge stuff, and at this stage we are mostly interested and getting feedback on the api design from fellow plug-in developers rather than feedback on performance, so that once these technologies are available in mainstream browsers we have a good way of doing the equivalent of "plug-ins" in DAWs. To that end we should probably do a tutorial soon!
stratum
KVRian
 
1360 posts since 29 May, 2012

Postby stratum; Wed Nov 22, 2017 8:10 am Re: Web Audio Modules (WAMs)

To that end we should probably do a tutorial soon!


Good idea. All WebRTC documentation I had seen sucked to such an extent that I had to reinvent the same stuff using only webassembly which took a lot shorter than trying to figure out their video transfer protocols. Don't make us suffer like that :lol:
~stratum~
hibrasil
KVRian
 
726 posts since 23 Jun, 2002, from Huddersfield, UK

Postby hibrasil; Wed Nov 22, 2017 3:39 pm Re: Web Audio Modules (WAMs)

stratum
KVRian
 
1360 posts since 29 May, 2012

Postby stratum; Thu Nov 23, 2017 1:54 pm Re: Web Audio Modules (WAMs)

[deleted] still reading..

edit: If I have understood correctly from your paper ( https://www.webaudiomodules.org/docs/wa ... mc2015.pdf ) you propose that controller and gui parts to be written in javascript and audio processor to be written in C++ or Javascript. If it was all C++ it would be easier to use. As for the Chrome cannary "AudioWorkletProcessor" api, I guess it's newer than the paper because it is not mentioned in the paper (edit2: "AudioWorkerNode" is mentioned shortly as an forthcoming standard.)

This sentence in the blog is a bit confusing:

The DSP is in WebAssembly, which is a low-level assembly-like language already supported in most popular browsers.


If that's the case, what is AudioWorkletProcessor and what problem it solves isn't clear(*). If this ( https://github.com/pioug/audio-worklet/ ... nerator.js ) exposes it correctly, it seems to be just an audio callback specification for low latency audio and nothing else. Having this wrapped in the SDL support in emscripten would be a better way, but I guess this may not happen anytime soon.

(*)Not that the sentence itself is confusing, but it contradicts some earlier posts in this thread, i.e. about the scripting mechanism fallback and things such (I had thought you were proposing to use the signal processing building blocks mentioned in https://developer.mozilla.org/en-US/doc ... _Audio_API ) . If the audio processor is simply C++ compiled to webassembly, on the other hand, then it's just another code download format, theoretically javascript could be just as fast, but in practice ASM.js works considerably slower but that looks like an implementation issue which should concern web browser developers. In the end we can assume that they will all implement webassembly and ASM.js is irrelevant.

One problem related to a C++ only solution would be the fact that emscripten threading is only supported in firefox nightly build at the moment (if they didn't release a newer version after I have seen that to be the case)

Perhaps I'm just allergic to javascript and html, who knows (have a look at AngularJS and other javascript frameworks and share some of that allergy, you'll hate them all if you aren't already a web developer wasting your life with that stuff ). In any case I'm not saying doing it only in C++ would improve anything in any technical sense, but sure it would be easier to use (i.e. I would rather make a GUI using the webgl wrapper in emscripten which is exposed as just another C/C++ opengl api))
~stratum~
jariseon
KVRer
 
6 posts since 31 Jan, 2015

Postby jariseon; Thu Nov 23, 2017 4:41 pm Re: Web Audio Modules (WAMs)

thanks for the feedback!

what is AudioWorkletProcessor and what problem it solves

The first incarnation of Web Audio API introduced ScriptProcessorNode (SPN) which enabled audio synthesis and processing using JS. The issue with SPN is/was that it runs in the main thread, together with other JS/HTML/CSS content. It was therefore subject to blockouts and audio glitches were not uncommon. Another downside was thread hopping (audio thread pulled data from main thread using double buffering). This increased latency.

AudioWorkletProcessor (AWP) runs directly in audio thread, and is therefore not influenced by what happens in the main thread. There's also no thread hopping, and latency is fixed at 128 samples buffer size. Although both SPN and AWP are able to run JS, asmjs or WebAssembly, AWP is only available in Chrome Canary. With a SPN fallback I meant that with it our demos would also run in other browsers.

I actually just finished writing an AWP polyfill that fallbacks to SPN. Thanks for the tip! The demos seem to work with at least in stable Chrome and Firefox Quantum. Need to finally update Safari to test it out as well.

Perhaps I'm just allergic to javascript and html, who knows

C/C++ is my mother tongue and I used to suffer from similar allergic symptoms :) However, I've later started to like the directness of vanilla JS quite a lot. But I'm still not exactly fond of heavy JS frameworks like Angular. I also do like declarative GUI development for widget based stuff, but also realize that it's just my personal preference.
stratum
KVRian
 
1360 posts since 29 May, 2012

Postby stratum; Fri Nov 24, 2017 4:51 am Re: Web Audio Modules (WAMs)

Thanks for the explanation. I haven't really looked into javascript implementation of things, I don't know what javascript API authors of emscripten wrapped in SDL, for example. It surely runs in the main thread, because at the moment threading support is lacking in general.

But I'm still not exactly fond of heavy JS frameworks like Angular


Such things look like attempts to fix what's wrong with building GUI's with HTML. Once you use something else (a widget library targetting OpenGL for example - the problem disappears and the heavy JS frameworks look like a joke in comparison - they are trying to solve the wrong problem - at least that's the way things look from a desktop developers perspective. It's not that making GUI's declarative wrong, they have been partially declarative for a long time, it's just that taking a text markup language and extending it to give support for things that it was never originally intended to be used for doesn't make much sense. In the end it's a neverending list of missing features because you can't do anything about it unless the browser developers add another missing piece with another formal specification proposal to be incorrectly implemented by others for the foreseeable future until it becomes another "standard". In the end the whole thing looks like display connector standards, VGA first, HDMI, DVI, and now they have added DisplayPort which will surely replace all others and serve all needs (if you believe) ) (OK, we can perhaps buy an HDMI to VGA connector, err, polyfill in javascript jargon).
~stratum~

Moderator: Moderators (Main)

Return to DSP and Plug-in Development