Urs' GPU comments

[blue monk]

In a recent interview you said that GPUs are in principal ideal for soft synths but latency is too high.
Do you mean that the data bus between CPU, RAM, GPU is too slow?
I imagine that the transfer speed is probably good in the direction to the GPU but not in the direction back from the GPU to the CPU. Is that correct?

[EvilDragon]

The bus itself is pretty fast, but getting and sending data to/from CPU takes a decent amount of CPU cycles (because it's sent in chunks, I would assume, rather than serially), which is not good for realtime audio processing at all.

[exmatproton]

The bus itself is pretty fast, but getting and sending data to/from CPU takes a decent amount of CPU cycles (because it's sent in chunks, I would assume, rather than serially), which is not good for realtime audio processing at all.

Still, a GPU/CPU combo is capable of 'near-realtime' processing (240Hz/FPS gaming for instance). And this is considering the complete package of data; I/O (mouse, keyboard etc..), frame/3d-rendering, audio etc..
The huge parallel capabilities of a GPU would be amazing for audio, IF they find a way to transfer data chunks more directly (i guess).

[chk071]

Not sure if it's really worth the effort. Not everyone is doing his audio stuff on a desktop machine with a capable GPU. The GPU's of laptops are often quite slow/low spec'd. I'd already be happy if all of the GUI stuff happening in DAW's and plugins would run on the GPU, i guess.

[Urs]

You gotta send data, you gotta wait, then you gotta pull data.

At, say 200 FPS and 48kHz sample rate you gotta process in chunks of 240 samples. So you gotta send 240 samples worth of audio to be processed to the GPU in one audio process call, then wait for the next call to retrieve that data. You also have to sync that with the audio host/driver set framerate, so you have to use the smallest common denominator somewhere. So you're ending up with, dunno a dozen or more milliseconds of plug-in latency, without much of a safety margin. Which, for a synth, is too much.

Last time I dug into this I think there were latencies of up to half a second involved. So maybe it *is* getting better

[blue monk]

You gotta send data, you gotta wait, then you gotta pull data.

At, say 200 FPS and 48kHz sample rate you gotta process in chunks of 240 samples. So you gotta send 240 samples worth of audio to be processed to the GPU in one audio process call, then wait for the next call to retrieve that data. You also have to sync that with the audio host/driver set framerate, so you have to use the smallest common denominator somewhere. So you're ending up with, dunno a dozen or more milliseconds of plug-in latency, without much of a safety margin. Which, for a synth, is too much.

Last time I dug into this I think there were latencies of up to half a second involved. So maybe it *is* getting better

Is the problem that chunks of data have to be used instead of a constant stream?
I imagine, that the chunking would not matter, neither the size, if the processed data would be coming back much faster than it is the case currently.
In multi core CPUs the individual cores are most likely connected via a much faster bus than the GPU-to-CPU connection.
That architecture is obviously no problem for the GPUs original purpose since that may require mostly uni-directional data transfers.
I have to speculate here since I have no know how in the matter. Your thoughts?

[jdnz]

Last time I dug into this I think there were latencies of up to half a second involved. So maybe it *is* getting better

pcie v4.0 doubles thruput, so presumably that would also halve latency? Meant to be available on machines this year.

[blue monk]

Last time I dug into this I think there were latencies of up to half a second involved. So maybe it *is* getting better

pcie v4.0 doubles thruput, so presumably that would also halve latency? Meant to be available on machines this year.

The question is at what speed it is a viable alternative for audio plug-ins.

[jdnz]

Last time I dug into this I think there were latencies of up to half a second involved. So maybe it *is* getting better

pcie v4.0 doubles thruput, so presumably that would also halve latency? Meant to be available on machines this year.

The question is at what speed it is a viable alternative for audio plug-ins.

bigger question is how you can practically use something as massively parallel as current top-end GPUs - spreading your dsp code across 3000+ cuda cores is non-trivial (so far everything I've seen is oriented to convolution reverbs as a result - though I guess someone could finally do that super-ultra-mega-saw synth with 3000 stacked oscillators )

[bmrzycki]

Last time I dug into this I think there were latencies of up to half a second involved. So maybe it *is* getting better

pcie v4.0 doubles thruput, so presumably that would also halve latency? Meant to be available on machines this year.

Interfacing with a GPU is completely different than a CPU from an application perspective. Several developers add in CPU assembly routines to directly manipulate the CPU: especially for instruction extensions like SSE or AVX.

GPUs all adhere to software ABIs like OpenCL, OpenGL, or DirectX. This means for even "trival" requests you have the following layer, at a minimum, between you and the GPU:

[Application] -> [ABI Userspace] -> [ABI Kernelspace] -> [GPU Driver] -> [Kernel transport and memory bus] -> [GPU]

And the whole path has to be reversed to get data back. To make matters worse the ABI might perform some of the operations on the host CPU to process data before dispatching to the GPU driver.

The reason this works so well for graphics is because it's only one way 99% of the time. The application sets up primitives, the stack blasts the GPU, the GPU drives your display. The 1% of the time is where people use the GPU for computation are performing problems where marshalling the data and commands in is a small portion of the actual work computed. It's also at least on reason why GPUs aren't used for more common OS tasks like floating point instruction offloads.

Urs has his work cut out to make this path work for a real-time domain like audio DSP. But he's a smart cookie and may make it work after all.

[EvilDragon]

Basically there needs to be some sort of ASIO-like driver for GPUs that talks directly to them, without additional layers.

[jdnz]

Basically there needs to be some sort of ASIO-like driver for GPUs that talks directly to them, without additional layers.

there is the GPUDirect tech which can massively low latency/increase thruput (https://developer.nvidia.com/gpudirect) - trouble is at the mo it's limited to tesla (and high end quadro) - which means that cards needed to use it make UAD seem cheap

[bmrzycki]

Basically there needs to be some sort of ASIO-like driver for GPUs that talks directly to them, without additional layers.

there is the GPUDirect tech which can massively low latency/increase thruput (https://developer.nvidia.com/gpudirect) - trouble is at the mo it's limited to tesla (and high end quadro) - which means that cards needed to use it make UAD seem cheap

GPU direct is just making use of "locked" memory regions. It still relies on CUDA as the ABI to broker requests. From my understanding even this technology doesn't really lend itself to what an ideal music offload DSP would look like. And, as you correctly said, it's limited to NVidia and certain models geared for compute offload (and not actually GPU) work.

[diaper@ky]

Urs has officially become proficient in Yankee slang.
Gotta is so Bronx.
Yuze guys is also widely used.
The word ask must also always include an "s" on the end as in asks.
If you're a brotha Axed and acts are used.
You'll be able to hang easily with those chops.

[jdnz]

Urs has officially become proficient in Yankee slang.
Gotta is so Bronx.
Yuze guys is also widely used.

also common australian-isms (i.e strine) : "I've gotta shoot down the bottle-o, yuze guys want me to grab you some tinnies while I'm there?"

And as we all know Urs doesn't go on holiday to aus just cos he likes snakes and huge venomous spiders