Efficient modular audio processing?

[mfa]

Oh yes, thanks! This is certainly better than going with raw function pointers.

Note that a virtual function call is 'underneath' a raw function pointer, so switching to a more complicated scheme might not help.

I'm afraid i'm not sure exactly what you mean by "underneath" here. Do you mean that it's lower level functionality, and faster as such?

I'm starting to believe that the virtual call overhead is really not an issue as long as the virtual functions process buffers and not single samples, just as Caco pointed out earlier.

[Chris Walton]

Oh yes, thanks! This is certainly better than going with raw function pointers.

Note that a virtual function call is 'underneath' a raw function pointer, so switching to a more complicated scheme might not help.

I'm afraid i'm not sure exactly what you mean by "underneath" here. Do you mean that it's lower level functionality, and faster as such?

I'm starting to believe that the virtual call overhead is really not an issue as long as the virtual functions process buffers and not single samples, just as Caco pointed out earlier.

Imagine this:

Code: Select all

class VirtClass
{
public:
    virtual void doSomething() { /* do something */ }
};

int main()
{
    VirtClass a;
    a.doSomething();
    return 0;
}

This is about equivalent to:

Code: Select all

struct VirtClass;

struct VirtClassVtbl
{
    void (*doSomething)(VirtClass* this);
};

struct VirtClass
{
    VirtClassVtbl vtbl;
};

void VirtClass_doSomething(VirtClass* this)
{
    /* do something */
}

VirtClassVtbl g_VirtClass_vtbl = { VirtClass_doSomething };

void VirtClass_construct(VirtClass* this)
{
    this->vtbl = g_VirtClass_vtbl;
}

int main()
{
    VirtClass a;
    VirtClass_construct(&a);
    a.vtbl.doSomething(&a);
    return 0;
}

To compare in x86 assembler, this is a virtual function call:

Code: Select all

; ecx is the pointer. eax is now a pointer to the vtable
mov  eax, [ecx]      
; let's pretend our function is at an offset of 0x0d
mov  eax, [eax+0x0d] 
; ... aaand call!
call eax

Regarding the original question - it's rare that virtual function calls will cost you much, unless you're quickly batch processing many items. Don't do it per sample, sure, but per buffer the cost will be negligible. You can optimize by doing raw function calls (one deference) instead of virtual function calls (two dereferences), but the bending-backwards you have to do for that is not worth the cost in headache and in maintenance pain.

One thing you can do, however, is instead of recursively calling your inputs, call it in a loop. That means, instead of the output module calling the effect module which calls the input module, instead just process the input module, then the effect module, then the output module. This gives you several benefits:

• You use less stack space, which means less committed memory usage of your program, which also means its faster since the chance to swap is (minimally) smaller, and less pages need to be committed (which takes time - involves kernel mode calls!)
• You use less of your stack, which means fewer stack cache misses, which means it's faster.
• It's cleaner from an architectural point of view, since your modules now don't know anything about their siblings. They are independent pieces of the chain, just acting when they're told.

Alternatively, you can all wait a bit and then use my perfected modular audio engine

[mfa]

Regarding the original question - it's rare that virtual function calls will cost you much, unless you're quickly batch processing many items. Don't do it per sample, sure, but per buffer the cost will be negligible. You can optimize by doing raw function calls (one deference) instead of virtual function calls (two dereferences), but the bending-backwards you have to do for that is not worth the cost in headache and in maintenance pain.

Thank you very much Chris! Your example sorted this out for me.

One thing you can do, however, is instead of recursively calling your inputs, call it in a loop. That means, instead of the output module calling the effect module which calls the input module, instead just process the input module, then the effect module, then the output module. This gives you several benefits:

• You use less stack space, which means less committed memory usage of your program, which also means its faster since the chance to swap is (minimally) smaller, and less pages need to be committed (which takes time - involves kernel mode calls!)
• You use less of your stack, which means fewer stack cache misses, which means it's faster.
• It's cleaner from an architectural point of view, since your modules now don't know anything about their siblings. They are independent pieces of the chain, just acting when they're told.

Yes, this is exactly what i was looking for. Thanks!

Alternatively, you can all wait a bit and then use my perfected modular audio engine

That sounds exciting! What kind of work is that? What kind of design goals you have? Have you set your focus on clever architecture or on implementing a bunch of building blocks? Is it aimed at "simple" plugins or to someone who's developing another Reaktor?

[Jeff McClintock]

You might find XYZ is faster...

Or you can take a smoke break instead, and when you come back to your computer, CPU upgrades taken by your customer base will have caused the average machine to get that much faster without you lifting a finger.

Yeah, "Don't Optimize" is a legitimate option, but does not answer the question.

[Chris Walton]

Alternatively, you can all wait a bit and then use my perfected modular audio engine

That sounds exciting! What kind of work is that? What kind of design goals you have? Have you set your focus on clever architecture or on implementing a bunch of building blocks? Is it aimed at "simple" plugins or to someone who's developing another Reaktor?

That was meant more in jest than seriously, but I am kinda working on something on and off. Considering I've got a job that keeps me pretty damn busy and a little daughter at home that also keeps me pretty damn busy I should be done in, uuh, about 2037.

[mfa]

Alternatively, you can all wait a bit and then use my perfected modular audio engine

That sounds exciting! What kind of work is that? What kind of design goals you have? Have you set your focus on clever architecture or on implementing a bunch of building blocks? Is it aimed at "simple" plugins or to someone who's developing another Reaktor?

That was meant more in jest than seriously, but I am kinda working on something on and off. Considering I've got a job that keeps me pretty damn busy and a little daughter at home that also keeps me pretty damn busy I should be done in, uuh, about 2037.

Oh, ok. First things first, i know. Glad i asked instead of just waiting for it. I'm kind of a sucker for all things perfect. By 2037 it'll be quite well perfected i'm sure.

[mystran]

You might find counting a loop backwards is faster because repeated comparison to zero is cheaper than loading a constant..

for (int i=numFrames; i> 0; --i) {

..you might find that pre-decrement (--i) is faster than 'post' (i--) because post-decrement uses an additional temporary variable (it returns the value before it was inremented/decremented).
Of course it all depends on you compiler, which may perform these optimisations automatically. Sure it seems a very small thing, but this type of loop occurs thousands of times in audio software, you may as well use the best-of-breed.

You might find that array access in forward order performs better with cache-prefetch though (in theory it works backwards too but forward is supposed to work better). You might also find that when you write a loop like this:

Code: Select all

for (unsigned i = 0; i < numframes; ++i)
{
   buf[i] *= gain;
}

the disassembly of the compiler generated code might actually be closer to this:

Code: Select all

if(numframes)
{
  unsigned i = numframes;
  float* buftmp = buf;
  do
  {
    (*buftmp) *= gain;
    ++buftmp;
  } while(--i);
}

Compilers do funny things, and unless you know that a particular compiler has a really bad optimizer (or none at all) or you're prepared to do several hours of profiling of different variations with several different models and brands of CPUs (well, assuming you're not on a fixed target) it's usually best to write the clearest possible code; clear code tends to be more likely to get properly optimized by the compiler.