A Method to process real-time digital audio using graphical processing unit (GPU) of video hardware is announced. The official site is not up yet, and information about the upcoming beta and products will be released soon at BionicFX.com.

This is a public announcement of the technical processing that is employed:

The audio processing software generates digital audio effects including reverberation, impulse response convolution, flange, chorus, delay, echo, pitch-shifting, and harmonizing by (1) transforming audio streams into audio samples, (2) transforming audio samples into video data elements, (3) combining video data elements into video data structures, (4) sending video data structures to 3D video cards, (5) executing audio algorithms as video processing instructions in pixel or fragment shaders on incoming video data structures, (6) storing results of the processed pixels as textures in video memory when multiple iterations are performed (or in the case of a single pass sending results to the buffer), (7) retrieving textures from the video memory when multiple iterations are performed, ( sending final shaded pixels to an off-screen video buffer, (9) retrieving the shaded pixels in the audio software or plug-in module, (10) decoding and translating the processed or shaded pixels into audio samples, (11) combining sequences of audio samples back into recognizable audio streams which possess the characteristics of the aforementioned audio effects.

Different manufacturers of video hardware produce video cards with varying amounts of memory, bandwidth through the video pipeline, processing speeds, and other capabilities to process and generate 2D and 3D video. These factors are taken into consideration by using known and gathered information about the video card to select or determine: 1) which pipeline to employ, 2) the numeric data type to use for video data, 3) the one, two, or three dimensional video data structure, 4) which version of a pixel shading application to load on the video card, 5) the sequence of steps to retrieve and decode processed pixels. The artificial intelligence applied in this adaptive selection of components, pathways, data types, and data structures exhibits decision making to select the most efficient path and process for the desired effect, while factoring the size of the audio samples with the characteristics and capability of the video card.

More information will be available at BionicFX.com soon.

Thanks,
James M. Cann
Cambridge, MA.

Nice to hear that it has finally been done.

I've been looking into Brook GPU for over a year now and slowly starting to understand it. My last year here at the university is partly going to involve researching exactly what you just announced.

Exciting news indeed.

I haven't actually gotten into testing it on any hardware though.

I know the latest gen graphics DSP power is off the map compared to our current CPU generation.

But what kind of plugin i/o latencies are we looking at?

And how do you deal with the accuracy issues of the different pipelines you mentioned? The way it looks is that the 24bit pipeline Ati cards wouldn't suite to audio quite as well as the newer fully 32bit Nvidia cards. Also, the pixel shader precision and the lenght of instructions are way higher in the current nvidia cards.

Never mind the audio-on-GPU technology; where can I get the software you used to turn the patent application into a press release?

To maximize parallellism between CPU and GPU, you need to batch a bunch of data for the GPU at once. Otherwise the GPU stalls waiting for the CPU to feed it. This is one of the biggest hurdles programmers face in a full-featured graphics engine.

I'm not sure what this means for real-time audio processing, but I'm not imagining an ideal scenario here. Then again, using a fraction of the GPU's capability may be better than none, depending on how efficiently the CPU and AGP/PCIExpress can deal with the overhead of shuttling data back and forth.

Also, GPUs are designed, built and optimized for graphics. A lot of the tricks they do to speed up graphics processing don't apply so well to more general cases. For instance, they have one pipeline for the RGB channels and one pipeline for the Alpha channel, and can independently perform operations on both pipelines as if it were a single instruction.

Of course my experience comes from the cards before the GeForce 6800... but that's what the majority of people actually own right now. Even Doom 3 and Half-Life 2 are not likely to push hardcore gamers to upgrade from a Radeon 9800 to a 3.0 card, nevermind the millions of schmucks who get their cards from Wal-Mart and buy the cheapest one that says "EXTREME!!!!" on it

And musicians aren't all hardcore gamers either

No but since musicians are primarily self indulgent arseholes (myself included) a helluva lot of them are gamers!

with this.. seeing (or is that hearing) is believing...

not only am I curious about performance.. but also about quality.. and latency measures..


pure speculation.. but if this is going to require a 3 or 400 dollar videocard then whats the use if I can make a second machine or an audio dsp card for that amount?

latency is what I was wondering about..that seems like a really intricate set of instructions to go through to achieve in time for live useablity..

Agreed, but even 'older generation' technology that can be had for $60 and which most of us have in our machines already will be able to take advantage of it.

My only thought, though, is that this isn't even a blip on the radar yet, so I'm not getting my hopes up for anything.

[edit] Hell, unless it's just temporarily down, BionicFX doesn't even have a splash screen placeholder at that url. Yeesh! [/edit]

Greg

Universal Audio's UAD-1 dsp card is really a hacked graphics card running UA's drivers and software...and this is a very old PCI card. There may be a lot of potential for reworking newer graphics processors to do audio DSP, but I have no idea how difficult this may be or the limitations developers would face.

That sounds really interesting to me.
All new middle range video card should arrive in september/October at both ATI (X700) and NVidia (6600).
Should cost around 200$/E.
If by upgrading to that kind of video card i can also have an accelerated audio processor fo "reverberation, impulse response convolution, flange, chorus, delay, echo, pitch-shifting, and harmonizing "......

Sure i'll buy one...
(And I'll be able to try Doom3 and HalfLife2... )

brianbrian quoth Universal Audio's UAD-1 dsp card is really a hacked graphics card running UA's drivers and software..

Proof?

There's a lot of speculation about this around.

What we do know is that UAD1 is based on a "scrap yard" chip that nobody else but the UAD1 inventor thought would be suitable for audio use. They "quad pumped" it to 1ghz from 250mhz to make it even faster.

Can't remember his name but it's the same guy who designed Oasys (among other things). There's an interview about this on the net.

There are literally thousands of DSP chips around suitable for audio. Who knows what theirs really is...

a GPU isn´t the same as a DSP chip is it?
Most GPUs have very specialised instruction sets also. Isn´t a DSP more like an empty "husk" that can be programmed to be whatever the needs are...granted, most GPUs can be programmed but aren´t they hardwired in production?

DSP = Digital Signal Processor (or Processing)
GPU = Graphics Processing Unit (a specialised form of DSP)

DSP chips are anything from industrial robot sensor controllers to GPUs.

It simply means a specialised instruction set tailored for certain tasks.

There are a few benchmarks around showing the general processing power of a modern GPU. They have tremedous capabilities in things like FFT and anything granular for example, or convolution (think impulse reverbs). Kyma, for example, will look a bit old fashioned processing-wise pretty soon. And as pixel shaders home in on general processing, we will see a lot easier programming for audio, too.

We are already talking about several multiples of the processing power of UAD1 (although modern GPUs are not directly comparable).

But that's why this announcement is so important.

So, what is it that makes a GPU so much more powerfull than a dsp?

As mentioned, the UAD dsp is 1ghz and most GPUs are around 300-500mhz. quite a difference!
Is it the shader processors or is it the fast memory?