DSP code - when to use floats or when to use doubles

[JustinJ]

One case where I definitely needed double was for Nigel Redmon +
style envelopes. With very long attack/decay/release times and only single-precision float, the envelope can get stuck on the same value because the per-sample diff is too low.

For this reason my envelopes segments are calculated to run for 'n' samples rather than relying on the actual value that the envelopes generate. It makes them rock solid and even if the value is slightly off, it's imperceptible.

[Tone2 Synthesizers]

I've only come across a hand full of situations where doubles were really needed during the last decades. This was:
- Summing up very high oder FIR filters
- Highly recursive algorithms like in an infinite reverb
- Precise timing calculations in BPM syncronisation
And here the difference between double/float was nearly inaudible.

In general it is safe to use floats. It is faster and reduces that RAM needed by 50%. If your algorithm really needs doubles to work properly you're usually doing somthing wrong and you should consider redesigning it.

[mystran]

- Summing up very high oder FIR filters
- Highly recursive algorithms like in an infinite reverb
- Precise timing calculations in BPM syncronisation
And here the difference between double/float was nearly inaudible.

For FIR filters, the order of accumulation also matters: if you accumulate small coefficients first you get much better precision than accumulating the largest coefficients first. Sometimes this is inconvenient or inefficient though, in which case just going doubles could be better.

Sort of between FIRs and recursive computations that's also FFTs: these are usually good in single precision up to a few thousand points, but if you want larger FFTs then it's usually good idea to do them in double precision. Large enough FFT size and it's difficult to even keep the algorithm stable in single precision.

For certain types of circuit simulation you might also find that solving in double precision leads to less problems and/or faster convergence.

In general it is safe to use floats. It is faster and reduces that RAM needed by 50%. If your algorithm really needs doubles to work properly you're usually doing somthing wrong and you should consider redesigning it.

Emphasis on "consider" because there's the occasional situation where using doubles makes sense, but I agree that the large bulk of what any given plugin does can usually be done in single precision just fine, especially if you prefer numerically stable algorithms.

[Tone2 Synthesizers]

if you accumulate small coefficients first you get much better precision than accumulating the largest coefficients first.

That's an interesting idea an can be used as a general rule:
If you have to accumulate floats, try to use ones that are from similar size.

[mystran]

if you accumulate small coefficients first you get much better precision than accumulating the largest coefficients first.

That's an interesting idea an can be used as a general rule:
If you have to accumulate floats, try to use ones that are from similar size.

Correct. The suggested approach of (roughly) ordering the order of accumulation is based on the idea that we'll keep the accumulator itself smaller while we're accumulating the small values by avoiding the largest values first.

Parallel accumulation using SIMD helps slightly as well for the same reason: each of the accumulators stays slightly smaller, effectively giving you a few extra bits until the final horizontal sum. So this one is a win-win really. Even if it might not provide a huge benefit in precision, it never really hurts.

[rafa1981]

You are a bag of tricks

[kernaudioio]

rafa1981's point about intermediate computation is the one that actually bites you in practice. you can have two numbers that are each well within float range but their difference is meaningful and tiny, and the mantissa just doesn't have enough bits to represent that difference accurately. catastrophic cancellation is the classic example. the answer looks fine on paper, it's just wrong.

the envelope case declassified raises is a good one. we ran into the same thing with our envelope follower in a saturation plugin. very slow release, very long time constants, the per-sample coefficient gets close enough to 1.0 that float stops resolving changes entirely. switching that specific accumulation to double fixed it cleanly, no other changes needed.

hugoderwolf's take about reformulating the math instead of reaching for doubles is valid for most cases though. if you're relying on double precision to paper over an ill-conditioned algorithm it will probably still cause problems, just quieter ones.

[kerfuffle]

I just want to point out that on a modern desktop or even mobile CPU, doubles are not always slower than floats and can even be faster. (Yes, you can fit fewer in the cache and the throughput is half when using SIMD. Whether these things matter depends on what you're doing.)

[mystran]

I just want to point out that on a modern desktop or even mobile CPU, doubles are not always slower than floats and can even be faster. (Yes, you can fit fewer in the cache and the throughput is half when using SIMD. Whether these things matter depends on what you're doing.)

Division and square root are almost always slower in doubles, but other operations tend to be the same. I don't think there's really cases where doubles would be faster (assuming we're not spending cycles on conversions), but they aren't really slower (for simple multiply-adds at least).

If you suspect you have a case where doubles are faster, make sure you have compiler warnings turned on for float conversions and/or double check the assembly (eg. clang in particular is really reluctant to reliably give you warnings of every double->float conversion that might accidentally get into your assembly).

The most obvious case is when you are accidentally use double-precision constant literals (eg. 3.14 instead of 3.14f) in your code, then this can totally tank your "single-precision" performance, because the compiler then needs to convert other terms into double precision, perform the arithmetics in double precision and finally convert the results into single precision (when assigned into a variable, or stored into memory).

[camsr]

So what is the "fastest" or most optimal way to define an accumulation of 2 floats into a double datatype in C? Anyone got a one-liner?

Code: Select all

float Guitar; // pseudonym for a signal.
float Vocal; // same as above.
double Mix = (double)Guitar + (double)Vocal
// and the optional step of downmixing to float
float Downmixed = (float)Mix.

It's just defining a signal (Mix) as being a lossless mix of Guitar and Vocal. However, the point is that Downmixed COULD be better than that simply adding the two signal floats without converting to double first. And I say could because it's only the values in the sample stream that matter.

If you keep increasing precision with every sum (or even any basic arithmetic operation) you would run out of memory fast Why would anyone ever bother to use increased precision as operations continue? It's because it is an intermediate representation of an answer that could be downscaled in precision at any point.