GPU powered audio processing

[lb24569]

I saw this posted on reddit today and thought it might be of interest.

https://www.braingines.com/

Hey everyone! We're working on rolling out GPU audio technology that allows plugins to be run off of any standard gaming GPU to achieve power and speed that surpasses any outboard DSP processing rack or internal CPUs. We've got it down to 1ms of latency, and running on hundreds of instances, and are finally starting to tell people about it, and get beta testers! Right now we're going to be rolling out a suite of our own plugins alongside an SDK and plugin marketplace (with real-time browsing of plugins) and would love for you to join us for the testing! Hit me with questions in the comments

Sign Up Here: www.braingines.com (scroll to request trial)

or heres the form directly: https://docs.google.com/forms/d/e/1FAIp ... A/viewform

The long term implications are huge - exponential power and speed, no CPU bottlenecking (quite literally, it CANT when its not in use) and a lot of remote processing features (wifi6/b3 processing over networks). We're working with some big plugin companies to start porting versions of their stuff over asap.

The pic below shows us running GPU Audio VST3's on a macbook, while using the GPU on a windows 10 laptop.

Very cool IMO, apparently its not been done before due to latency issues but these guys are confident they've cracked it at 1ms.

[Soundplex]

Well, it is also not done because GPUs are good at simple parallel processing while audio data usually needs sequential processing. The mentioned DSP cards furthermore are specialized on these tasks while the GPU isn't. I think they're going with "think big". It seems limited to VST3 only and even networking as low as 1ms latency? Let's see what devs say about it and how it is done at all.

[medienhexer]

Without wanting to start any Windows PC vs Mac discussion - Apple’s new systems starting with the M1 have an architecture where data doesn’t have to be copied to the GPU anymore. I fully expected an announcement for these systems.

But for systems with separate main and graphics memory where copying is inevitable, this would indeed be an impressive achievement. Congratulations to the developers.

[medienhexer]

Well, it is also not done because GPUs are good at simple parallel processing while audio data usually needs sequential processing. The mentioned DSP cards furthermore are specialized on these tasks while the GPU isn't. I think they're going with "think big". It seems limited to VST3 only and even networking as low as 1ms latency? Let's see what devs say about it and how it is done at all.

AVB does 1ms over ethernet or do I remember incorrectly?

[Soundplex]

AVB does 1ms over ethernet or do I remember incorrectly?

I think it was 2ms and that does not include the time to transfer the audio to the GPU or even process it.

[lb24569]

Without wanting to start any Windows PC vs Mac discussion - Apple’s new systems starting with the M1 have an architecture where data doesn’t have to be copied to the GPU anymore. I fully expected an announcement for these systems.

But for systems with separate main and graphics memory where copying is inevitable, this would indeed be an impressive achievement. Congratulations to the developers.

I must admit it would be very cool if developers were able to offload plugins onto apple silicons nurual engine.

[medienhexer]

Yes, that and the GPU cores potentially offer a lot of processing power.

If it happens, it will probably happen in Logic Pro first. Something like saturation in each channel or "analog" summing would be a nice option to be able to flick on/off.

[medienhexer]

AVB does 1ms over ethernet or do I remember incorrectly?

I think it was 2ms and that does not include the time to transfer the audio to the GPU or even process it.

Yes, I think you're right.

Well, perhaps the processing doesn't need huge additional buffers if transfer and processing are that fast. And AVB is more of an open standard which has to cover a variety of use cases.

Waves offer < 1ms latency for monitoring through their plug-ins via Soundgrid:
https://www.waves.com/soundgrid-studio- ... th-plugins

And that's at least including latency from processing on the CPU.

Maybe they were able to cut some corners by limiting themselves to this one use case. Maybe the transfer to the GPU is heavily losslessly compressed?

[thresholdpeople]

There are a bunch of other programs that allow for similar capabilities, though no necessarily with GPU, but but using computers on the same network. Here's an open source one: https://audiogridder.com/.

And in trying to remember the name of Audio Gridder, I stumbled upon this- https://www.dawconnect.com/.

[glokraw]

Fathom Synth will have nVidia gpu processing, not sure the timing, but it's been on the worktable for some time now, and he just released version 3.3 which enables creating skins, and up to 16 presets can have their own unique skins. Discussion:

viewtopic.php?f=1&t=482501&start=5040

[planetearth]

Fathom Synth will have nVidia gpu processing, not sure the timing, but it's been on the worktable for some time now, and he just released version 3.3 which enables creating skins, and up to 16 presets can have their own unique skins. Discussion:

viewtopic.php?f=1&t=482501&start=5040

Not to hijack this thread, but I didn't see anything on the page you linked (or the page before it) about nVidia GPU processing, other than that someone was having a lot of glitches with the nVidia processing, and (perhaps unrelated to "GPU processing"), the new GUI will be (or is) vector-based. (That said, I didn't read all 335 other pages, so there may be more to it.)

Do you know if Fathom will use nVidia GPUs for generating sounds? How stable or CPU-intensive do you think it will be for those who don't have nVidia GPUs? I have an earlier version of Fathom, and it's pretty CPU-heavy. I also don't have an nVidia card and don't want to have to buy one to make the next version work smoothly.

If they can offload some of that to a GPU, it would be interesting, but since GPU processing isn't necessarily designed to calculate and return results in a specific order, I'm still not sure how Fathom/nVidia or Braingines will work. No other "pro-level" VST instrument or effect is using the GPU for processing AFAIK. A couple of reverbs are using it, but one hasn't been updated in 8 years and the other is a freebie with a fixed latency of 1024 samples.

I think it would be great if this takes off, but I also think it would require a change in how nVidia and ATI code their GPUs...and that could piss off a lot of cryptocurrency miners.

Steve

[Basilbasil]

- all things run within 1ms buffer; the current version supports 96 samples buffer @96khz. Even over LAN. EVEN on eGPU over LAN. Including processing and transfer time.
- VST3 at this moment because: 1) this is a starting point, but not a limitation.
- WIN/NVIDIA is at this moment. WIN/AMD will be in the future. MAC versions will be only as a client-side (Au, maybe aax) but the processing has to be on an external WIN server. The server part on Mac isn't possible at this moment due to the current Apple framework limitation also due to obsolete Amd and Nvidia drivers for mac os x.
- Fanthom has nothing to do with calculating the audio algorithms on the graphics card.

[frizzbee]

the website doesn't say much at all. and it seems you can't just get the sdk. then on the other hand audio on GPU isn't something new either.

[glokraw]

Do you know if Fathom will use nVidia GPUs for generating sounds? How stable or CPU-intensive do you think it will be for those who don't have nVidia GPUs?
Steve

viewtopic.php?t=482501&start=3090

This is page 209 or so from Fathom the topic, before Christmas 2018, and the dev makes a lot of replies to a lot of questions in that region of the thread. He mentions that gpu's exist on many motherboards, being used for 3D games, so support will go beyond just slot-filling gpu cards. I fully expect the results to be better and different than any recent mentions, just because there is much uncharted territory to conquer, with a 'self described stubborn perfectionist at work.
Cheers

[planetearth]

Do you know if Fathom will use nVidia GPUs for generating sounds? How stable or CPU-intensive do you think it will be for those who don't have nVidia GPUs?
Steve

viewtopic.php?t=482501&start=3090

This is page 209 or so from Fathom the topic, before Christmas 2018, and the dev makes a lot of replies to a lot of questions in that region of the thread. He mentions that gpu's exist on many motherboards, being used for 3D games, so support will go beyond just slot-filling gpu cards. I fully expect the results to be better and different than any recent mentions, just because there is much uncharted territory to conquer, with a 'self described stubborn perfectionist at work.
Cheers

As I mentioned, I didn't read all 335 other pages, so yeah, the results may be different than any recent mentions -- and I hope they are. But there's probably a good reason that for all intents and purposes, no one else is using GPU processing in instrument or effect plug-ins. I hope the dev doesn't go too far down the "GPU-enhanced instrument" rabbit hole before he finds this out; independent devs can't usually afford such a detour. And I also hope he doesn't exclude those who don't use nVidia processors (and/or all Apple M1 processor users) from any "enhancements" built into the next version of Fathom.

And with that, I happily return this thread to what the intrepid developers at Braingines are working on. I sincerely hope this works out and we can finally harness GPU processing power for something more practical than cryptocurrency mining!

Steve