Coding 101 question - "double" vs. "float"

[Fender19]

I see a lot of audio code that use "doubles" rather than "floats". I don't know if this is just common practice handed down from 16 bit days or if there's a real need for doubles in some cases. 32-bit processing (24 bit mantissa) seems like plenty to handle anything we can hear (the very best A/D/As can only achieve ~21-22bits).

When is "double" necessary/beneficial and when is just wasted CPU cycles? I know higher bit depth processing has lower errors - but can you HEAR it?

[matt42]

Sometime 64 bit is necessary for iir filters depending on the filter order, the cutoff and topology of the filter.

Here's an example from this thread.

For correct operation at low frequencies you will need double precision numbers to have enough precision so that the accumulation of state can still be resolved. This is easier to see in mystrans += form, so if the += is operating on a large state and trying to increment it by a very small number problems will arrise:

eg:
float v = 1.0;
v += 1e-20f; // this won't work!

As for wasted CPU cycles, in some tests I did I found that 64 bit performance was at least as good as 32 bit.

[MadBrain]

I see a lot of audio code that use "doubles" rather than "floats". I don't know if this is just common practice handed down from 16 bit days or if there's a real need for doubles in some cases. 32-bit processing (24 bit mantissa) seems like plenty to handle anything we can hear (the very best A/D/As can only achieve ~21-22bits).

When is "double" necessary/beneficial and when is just wasted CPU cycles? I know higher bit depth processing has lower errors - but can you HEAR it?

In a lot of applications there's not really any difference.

- Double is twice as large so it takes twice the memory bandwidth and eats up twice as much cache. But unless you're using SIMD that's about the only downside.

- For most applications, float has quite enough precision. The exception I can think of is sample position counters.

[koalaboy]

Another useful example: http://msdn.microsoft.com/en-us/library/c151dt3s.aspx

It all depends on how accurate you want your calculations to be and how well you understand the effects on every single calculation you right.

Personally, unless I know I don't need any precision, I'd use doubles. IEEE is a complete bugbear that always causes unexpected problems.

YMMV.

[mystran]

Doubles make everything sound twice as good. That's why it's called doubles.

Seriously though.. when either works, the performance difference is often negligible, except where the increased cache use becomes a problem. With doubles you can sometimes use cheaper (but numerically less well-behaved) math that would run into precision problems with floats... and sometimes recursive algorithms can accumulate enough errors that it becomes (easily) audible with floats (or even doubles, but at least that takes somewhat more effort).

That said, for about 90% of stuff it doesn't really matter either way and even where it does, you can usually rework things to work with floats too (though in some cases that can be more work than actually writing the problematic code).

[ChewingAluminumFoil]

Years ago I heard that there was overhead converting floats to double precision (where all math is done) so I use floats for big arrays where space is a concern but otherwise use doubles.

[JCJR]

Tis been too long since examining the num clocks per instruction timing charts for x86, and probably it is different nowadays anyway. Load of either float or double to the fpu are variations on the same fld opcode. Memory bandwidth for float would be half that of double, so perhaps fld would work as fast or faster on a float than a double? As best I recall either operation was fairly fast even in the old chips.

The above observation being predicated on always leaving the cpu in the double float math mode. In the dim past reading the manuals, I got the impression that a person would need to be fairly desperate for extra clock cycles in order to put the chip in the single float math mode, because it didn't appear drastically faster in single float math mode. Again, probably different nowadays anyway. Likely irrelevant old information.

Never studied timings for SSE instructions, float versus double.

[MadBrain]

Tis been too long since examining the num clocks per instruction timing charts for x86, and probably it is different nowadays anyway. Load of either float or double to the fpu are variations on the same fld opcode. Memory bandwidth for float would be half that of double, so perhaps fld would work as fast or faster on a float than a double? As best I recall either operation was fairly fast even in the old chips.

The above observation being predicated on always leaving the cpu in the double float math mode. In the dim past reading the manuals, I got the impression that a person would need to be fairly desperate for extra clock cycles in order to put the chip in the single float math mode, because it didn't appear drastically faster in single float math mode. Again, probably different nowadays anyway. Likely irrelevant old information.

Never studied timings for SSE instructions, float versus double.

Fld runs in the same amount of cycles on float and double (even on the original Pentium). What changes is simply the likelihood of cache misses (increased chance of bumping out useful cache lines, increased number of cache lines loaded for a given buffer size, potentially twice the potential cache misses and potentially requiring twice as much operations from RAM to read the data).

Some versions of Directx set the fpu in single mode for faster fdiv and fsqrt. But for fadd/fmul/etc there's no difference.

SSE is different of course but you get the idea.

[sonigen]

Years ago I heard that there was overhead converting floats to double precision (where all math is done) so I use floats for big arrays where space is a concern but otherwise use doubles.

The old x87 (fpu) code did it intrinsically, loads / stores intrinsically converted so there is no conversion cost. For SSE code you need an extra instruction. You load with one instruction and convert with another.

[sonigen]

When is "double" necessary/beneficial and when is just wasted CPU cycles? I know higher bit depth processing has lower errors - but can you HEAR it?

Single precision has a range of about 140dbs, so much higher than any soundcard, and likely much higher than your audio algorithms.

The only times in 10 years that I've run into trouble with single precision was...

1. With an FFT that was using trigonometric recurrence to generate the trig lookup tables.
2. 2nd order filters with certain settings.

IE. It's either or both...

Anything that has high amounts of feedback which can compound the rounding errors.
Anything that has coefs that can not be accurately enough represented in single precision.