64 bit summing engine -- how far should it be pushed?

[schnapsglas]

No, this is not a troll thread asking whether 32bit engine sounds worse than 64 bit double precision.

I am taking everyday mixing out of the equation and want to ask the question; how far do you have to push the engine in terms of chained effects and such to push the error in float to even the noticeable level? My guess is that it would have to be an outrageously long chain of effects and computations but my engineering skills don't go that far

If any developers could answer this, I would love to know.

[earlevel]

I'm going to skip past the list of things that affect the accurate of the low bits...I'll assume floating point, which is less sensitive to order of operations for multiplication...and get right to a more intuitive look:

Single precision floating point gives 25 bits of precision (24 plus sign). You can do pretty well recording and mixing in 32-bit float, including a reasonable amount of plug-ins—I'm sure you're familiar with products that do this. When you go to listen to it, you might have a 24-bit D/A converter, but realize that you don't heard the full 24 bits. For instance, if the full-scale swing of the analog output is 1 volt, the last bit is about 0.06 microvolts. Sorry, the thermal noise of any circuit you'll be listening through is a lot higher than that. So, we can afford to have a bit of noise build up before we could ever know it.

Now, enter 64-bit floating point—54 bits of precision. That's 29 more bits, or 2^29th—536,870,912 times as much.

In other words, it's gone, in one step, from "usually good enough" to "insane drastic overkill". But that's the way the hardware is built—you get 32-bit floating point, or you get 64-bit floating point—there are no 40 or 48 choices.

That is, don't worry about it

Or, if you do need to worry about it, you're doing something seriously wrong, or a plug-in manufacturer is doing something seriously wrong.

[Johnny!]

Amen.

[cerberus]

...how far do you have to push the engine in terms of chained effects and such to push the error in float to even the noticeable level? My guess is that it would have to be an outrageously long chain of effects and computations but my engineering skills don't go that far

If any developers could answer this, I would love to know.

nothing needs to be outrageous or out of the ordinary:
a convolution using a 100 sample i.r.
results in 100 potential rounding errors per sample.

[earlevel]

...how far do you have to push the engine in terms of chained effects and such to push the error in float to even the noticeable level? My guess is that it would have to be an outrageously long chain of effects and computations but my engineering skills don't go that far

If any developers could answer this, I would love to know.

nothing needs to be outrageous or out of the ordinary:
a convolution using a 100 sample i.r.
results in 100 potential rounding errors per sample.

I think cerberus is just teasing here, but in case it makes anyone paranoid...what he says is correct, sort of, but has two problems. First, the potential rounding errors don't simply sum—statistically, the low bits are going to look a bit random, as far as whether they will round up or down. But even if you're so paranoid that you think that every operation will go against you and round in the same direction, consider that 64-bit floating point has a billion (2^30) times the precision of your 24-bit converter. Yes, there will be rounding every time you multiply one 64-bit number with a 64-bit coefficient and it rounds (or truncates—no difference in the argument except a bias of half a bit) to fit it back into 64 bits. But the rounding is—again—at a billionth of a level that's already not possible to hear.

So, when will CPUs be fast enough to do enough processing within a sample period to grow that error to the point where it creeps into your 24-bit DAC output if all operations are 64-bit floating point? I'd be pretty happy with my computing power the day that happens.

[jens]

...how far do you have to push the engine in terms of chained effects and such to push the error in float to even the noticeable level? My guess is that it would have to be an outrageously long chain of effects and computations but my engineering skills don't go that far

If any developers could answer this, I would love to know.

nothing needs to be outrageous or out of the ordinary:
a convolution using a 100 sample i.r.
results in 100 potential rounding errors per sample.

Even if this was true, it actually has got nothing to do with the topic of this thread, since these rounding errors you mention would happen within a plugin.

[Richard_Synapse]

The difference is marginal, probably inaudible in most cases.

But then again, the 64-bit summing as such is very cheap. What's costly is when the whole DAW signal path is 64-bit, including all instruments and effects. This will massively slow down things and I don't think this can be justified just yet.

Richard

[cerberus]

Even if this was true, it actually has got nothing to do with the topic of this thread, since these rounding errors you mention would happen within a plugin.

A typical VST plug-ins' internal precision could be fixed-point, floating point, or double precision floating point. The OP mentioned "chained effects", IMO effects may be a better place to start examining questions about mathematical precision and audibility. If the effects chain is not "clean" then IMO the rest matters a lot less, garbage in...

[Xenakios]

The difference is marginal, probably inaudible in most cases.

But then again, the 64-bit summing as such is very cheap. What's costly is when the whole DAW signal path is 64-bit, including all instruments and effects. This will massively slow down things and I don't think this can be justified just yet.

Richard

The whole path being 64 bit isn't a huge performance hit at all, unless some 3rd party stuff messes things up in the middle. Reaper's developer has done test builds of Reaper so that it uses 32 bit floating point numbers instead of 64 bit ones to see if there's some notable performance difference, and there wasn't. "Massively slow down" is thus grossly exaggerated. Audio buffers being 64 bit floats double the memory usage of the buffers obviously, though. (And perhaps with really old hardware that additional memory bandwidth use might slow down actual DSP calculations too, but we really should always talk about realistic average current day hardware and not some junk from 2001.)

I am however of the conviction that 64 bit summing and signal paths in the host application don't contribute anything significant to the audible result unless things are on purpose set up to cause the differences to become audible. (These kinds of tests are possible to do with some care.) But that wouldn't happen during any normal and sane music production situations.

DSP modules (plugins/built-in effects of the host) may or may not benefit from using 64 bit floats internally. If memory resources are at a premium, the developer would then need to consider the tradeoff between memory usage and audio quality, if it was determined the 64 bit floats make an audible difference.

[Xenakios]

garbage in...

The problem with this is that 32 bit floating point is already way way past the "garbage" point. It just happens that today's hardware can efficiently deal with 32 bit floating point numbers, so it wouldn't make technical sense to use, say, 24 bit calculations though that might be plenty enough to get audible results that are clean. You will disagree with that and can come up with all kinds of counter examples (which obviously do exist), but I am speaking of average often encountered cases here. In DSP development these things have to be considered on a case by case basis, making tradeoffs between CPU usage, memory usage and the audio results. (The choice that would most often practically be encountered in DSP development for x86/x64 architectures is to whether to use 32 bit or 64 bit floating point numbers. Mentioning the 24 bits case is an unrealistic example that a developer would almost never need to consider these days, it's just to illustrate a point : that if we had commonly available hardware that dealt efficiently with 24 bit numbers, it might in many cases be completely reasonable to use those kinds of numbers, as even those are already way past what is needed for to get an audible result that is not "garbage".)

[Fixed Error]

I was perfectly happy with 8-bit. Mmmmm, 8-bit...

[earlevel]

garbage in...

The problem with this is that 32 bit floating point is already way way past the "garbage" point. It just happens that today's hardware can efficiently deal with 32 bit floating point numbers, so it wouldn't make technical sense to use, say, 24 bit calculations though that might be plenty enough to get audible results that are clean.

Agree with most of what you say, Xenakios, except this...

32-bit float is 25 bits of precision and you're comparing that with 24-bit (I assume integer, because a 24-bit floating point format wouldn't be adequate) and calling the former "way way past..." and the latter "might be plenty"—in reality they are pretty close to the same thing. (As a side note, people forget, when touting how great 32-bit float is compared to 24-bit int, that the popular 24-bit hardware has 56 bits of accumulation.)

[cerberus]

You will disagree with that and can come up with all kinds of counter examples (which obviously do exist)

You're correct. I'm especially concerned with the nature of the signal near and below the noise floor, and how very small distortions in louder signals can alter one's impression of the material. I once heard of someone else who thought similarly (who happens to be a famous gear designer, but that doesn't make me right).

I'll be trying to prove it with my company's first fx plug-in, aiming for comments like:
"Remarkable, Your plug-in does sound OK" or even: "OK, Your plug-in does sound remarkable".

I was perfectly happy with 8-bit. Mmmmm, 8-bit...

IIRC Xenakios' app hourGlass has some bit-crushing capacities among it's many mangling possibilities. You might like it.

[Fixed Error]

Yeah, but 8-bit because of lack of better technology is the real shizzle, as opposed to fabricated 8-bit

[Xenakios]

garbage in...

The problem with this is that 32 bit floating point is already way way past the "garbage" point. It just happens that today's hardware can efficiently deal with 32 bit floating point numbers, so it wouldn't make technical sense to use, say, 24 bit calculations though that might be plenty enough to get audible results that are clean.

Agree with most of what you say, Xenakios, except this...

32-bit float is 25 bits of precision and you're comparing that with 24-bit (I assume integer, because a 24-bit floating point format wouldn't be adequate) and calling the former "way way past..." and the latter "might be plenty"—in reality they are pretty close to the same thing. (As a side note, people forget, when touting how great 32-bit float is compared to 24-bit int, that the popular 24-bit hardware has 56 bits of accumulation.)

Yes, it's of course more complicated in the details. Mentioning 24 bits was quite arbitrary and I could have said as a controversial bait for example 18 bits.