Accuracy of your Synth's wave generation

[Nowhk]

Hi there,

since I'm making a "synth" (drum synth), learning DSP and (recently) using optimization with SIMD, I'd like your opinion about two points, speaking about accuracy.

1 - on generating waves, which fp precision do you use?
Float or double? I'm really not able to "perceive" any differences generating a sin (or square) with basic functions like this:

Code: Select all

   double value = sin(phase) * gain; // sine
   double value = (phase <= PI ? 1.0 : -1.0) * gain; // square
   double value = (((2.0 / TWOPI) * phase) - 1.0) * gain; // triangle

The introduced error its very low; but probably, later, it will impact the whole "quality"? i.e. after processing it on a fx.
Or float is enough for a generated signal? Because processing floats with SIMD make stuff really lighter on CPU

2 - which kind of approx fuction error is required? I'm trying the "out of the box" IPP sin function right now, and I'm noticing that the performance really change between ippsSin_64fc_A26 (approximately 8 exact decimal digits) and ippsSin_64fc_A53 (maximum guaranteed error within 1 ulp). Would the first be enough, or its ALWAYS better stay at higher precision?

The idea is to abandon IPP and replace with a fancy SSE2 library sin approx, but I'd like to got any idea of what I need.

Or it really doesn't matter? For example, the (scalar) sin() function on math.h (for what I've read) seems to be dispatched due to the input vaue range, choosing the correct sin math approx (which I don't know the accuracy).
What do you use in your plugins?

Your opinios are always great

Thanks

[BertKoor]

Since even 16bit audio is good enough for precision errors not to be of any relevance (in rendered end result, not talking about an intermediate processing format inside your filters) I am not surprised: 32bit is sufficient.

[Nowhk]

Since even 16bit audio is good enough for precision errors not to be of any relevance (in rendered end result, not talking about an intermediate processing format inside your filters) I am not surprised: 32bit is sufficient.

I think its enough as "final step" of the chain (i.e. render the final result).

But my generated signal is supposed to be processed through many FX stages; it become the first element of the chain. Doesn't matter?

[mystran]

1 - on generating waves, which fp precision do you use?
Float or double? I'm really not able to "perceive" any differences generating a sin (or square) with basic functions like this:

Code: Select all

   double value = sin(phase) * gain; // sine
   double value = (phase <= PI ? 1.0 : -1.0) * gain; // square
   double value = (((2.0 / TWOPI) * phase) - 1.0) * gain; // triangle

If I'm not mistaken, you are actually giving up some precision here by using 2*PI as your period. Modulo by power of two (eg. typically 1.0 for phase wrap-around) is exact (in the sense that it just bit-shifts mantissa and adjusts exponent), where as modulo by 2*PI will introduce additional rounding (not that it's going to make a huge difference in practice, but still).

That said, if we assume phase wrap around at 1.0 (for simplicity of the computation that follows), then your pitch accuracy for single precision is on the order of 1/(2^23) cycles per sample and for double precision on the order of 1/(2^52) cycles per sample. So for single precision at 44100Hz sampling that means roughly 0.005Hz precision, which is probably sufficient for audio most of the time, but could become a problem for slower LFOs if you're running them at audio rates.

The actual drift is somewhat trickier to compute for floating point, as the accumulation is more accurate near zero and then hits the worst-case just before wrap-around, but that should still give you a rough estimate. In terms of audio quality, single precision is almost certainly enough here; it's mostly in recursive computations where you might (or might not, depending on the situation) benefit from double precision. In this case the recursive computation of interest is the phase accumulation which mostly affects pitch accuracy (probably good enough) or LFO drift (might or might not become a problem).

Also note that for waveforms like saw/pulse/triangle, the audio quality is typically dominated by the quality of your anti-aliasing, rather than the computational precision.

2 - which kind of approx fuction error is required? I'm trying the "out of the box" IPP sin function right now, and I'm noticing that the performance really change between ippsSin_64fc_A26 (approximately 8 exact decimal digits) and ippsSin_64fc_A53 (maximum guaranteed error within 1 ulp). Would the first be enough, or its ALWAYS better stay at higher precision?

Each decimal digit is worth 20dB of signal to noise, so 8 decimal digits equals 160dB which is certainly enough for pretty much all audio purposes (although I'd guess it's really 26 bits of mantissa, so about 162dB). That's more than you can fit into single-precision, and honestly twiddles for long FFTs are pretty much the only thing I can think of that might need more. In fact, for general audio use you might even want to opt for less accurate approximation.

[vortico]

Float32s are fine for virtually all audio purposes except for unstable numerical methods, e.g. having catastrophic cancelation. If your audio is scaled between -1 and 1, the coarsest granularity of float32s in that set is between 1 and the next smallest value ~0.99999994, which is a 6e-8 or -144dBr difference. A rule of thumb is that the set of 24-bit signed integers can be embedded into the set of float32s on [-1, 1] (and remember we have more granularity for numbers closer to 0), and 53-bit ints into float64s.

[kryptonaut]

Also note that for waveforms like saw/pulse/triangle, the audio quality is typically dominated by the quality of your anti-aliasing, rather than the computational precision.

As Mystran says, if you are using the wave as an audio source then a naively generated saw/pulse/triangle will sound very rough regardless of whether you use floats or doubles, due to the presence of harmonics with frequencies greater than Nyquist. This is not a precision issue, but is caused by the abrupt changes in the waveform value/slope.

I'd suggest you look at wavetables or BLEP synthesis if you are interested in this area.

[Nowhk]

it's mostly in recursive computations where you might (or might not, depending on the situation) benefit from double precision. In this case the recursive computation of interest is the phase accumulation which mostly affects pitch accuracy (probably good enough) or LFO drift (might or might not become a problem).

I've started with another "problem" in mind once I open this topic:

Code: Select all

  y1 = sin(x1)
  y2 = sin(x2)
  y3 = sin(x3)  
  ...  

my concern was more about the drift of y1, y2, y3 between calculating it using float instead of double, rather than the accumulation error of x1, x2, x3 at each step, which is another trouble, of course; let me talk about this below. First back to "original" trouble.

As said to BertKoor, starting with a signal "already" with less precision, couldn't be a problem for future processing of itself?

I mean...
If the sinfloat(x) already have more error rather than sindouble(x), wouldn't this error increment if I apply other FX to the signal later (wave shaping, for example)?
Or (even more) if I use that signal as modulation source (FM, for example)?

Or the "number" of following stages are irrelevant since it would need a very huge amount of post process to introduce noticeable drifts (i.e. a 100 FM chain, which is uncommon)?
That's my concern about precision on math functions in audio

Now, lets talk about phase accumulation...

If I'm not mistaken, you are actually giving up some precision here by using 2*PI as your period. Modulo by power of two (eg. typically 1.0 for phase wrap-around) is exact (in the sense that it just bit-shifts mantissa and adjusts exponent), where as modulo by 2*PI will introduce additional rounding (not that it's going to make a huge difference in practice, but still).

That said, if we assume phase wrap around at 1.0 (for simplicity of the computation that follows), then your pitch accuracy for single precision is on the order of 1/(2^23) cycles per sample and for double precision on the order of 1/(2^52) cycles per sample. So for single precision at 44100Hz sampling that means roughly 0.005Hz precision, which is probably sufficient for audio most of the time, but could become a problem for slower LFOs if you're running them at audio rates.

Yes, I calculate phase (x1, x2, x3, ecc) this way:

Code: Select all

	double bp0 = mNoteFrequency * mHostPitch;
	...
	for (int sampleIndex = 0; sampleIndex < blockSize; sampleIndex++) {
		// sin(phase) here
	
		phase += std::clamp(radiansPerSample * (bp0 * pB[sampleIndex] + pC[sampleIndex]), 0.0, PI);
		if (phase >= TWOPI) { phase -= TWOPI; }
	}

And indeed using double instead float will introduce more errors on phase accumulation, right I've just ignore it until now, hehe.
Not sure about your suggestion of the "trickier way" to remove this phase drift: reset... on wrap?
How do you know which value "should it be" at wrap moment? Uhm, can you provide to me an example?

Each decimal digit is worth 20dB of signal to noise, so 8 decimal digits equals 160dB which is certainly enough for pretty much all audio purposes (although I'd guess it's really 26 bits of mantissa, so about 162dB). That's more than you can fit into single-precision, and honestly twiddles for long FFTs are pretty much the only thing I can think of that might need more. In fact, for general audio use you might even want to opt for less accurate approximation.

Well, yes: I think ippsSin_64fc_A26 is enough so! I'll looking for a sin math approx with this kind of error (approximately 8 exact decimal digits), which can work on SSE2 and also double (for testing purpose). Found this, but it only works with float.

As Mystran says, if you are using the wave as an audio source then a naively generated saw/pulse/triangle will sound very rough regardless of whether you use floats or doubles, due to the presence of harmonics with frequencies greater than Nyquist. This is not a precision issue, but is caused by the abrupt changes in the waveform value/slope.

Yeah I know. Right now I'm more interested on "generating" precision; I know the problem of aliasing, I'll challenge it later, thanks for remark.

I'd suggest you look at wavetables or BLEP synthesis if you are interested in this area.

BLEP and Wavetable are my next cases of study Now I'm getting the basics!

[mystran]

my concern was more about the drift of y1, y2, y3 between calculating it using float instead of double, rather than the accumulation error of x1, x2, x3 at each step, which is another trouble, of course; let me talk about this below. First back to "original" trouble.

I don't really understand any of what you are trying to say here. Phase accumulators will drift as they accumulate error over time, where sin() is essentially just a waveshaper.

I mean...
If the sinfloat(x) already have more error rather than sindouble(x), wouldn't this error increment if I apply other FX to the signal later (wave shaping, for example)?
Or (even more) if I use that signal as modulation source (FM, for example)?

Yes, but the question is whether it matters. The thing is as a rule of thumb you can generally assume that the most significant source of error will dominate the SNR to the point where other sources of error are essentially irrelevant. So the question you really want to ask is whether increasing the precision will reduce the most significant source of error by a meaningful amount.

Not sure about your suggestion of the "trickier way" to remove this phase drift: reset... on wrap?

Not suggesting anything tricky, but rather pseudocode that looks like this:

Code: Select all

for samples:
   phase += freq / fs
   if phase >= 1: 
      phase -= int(phase)
   out = sin(2*pi*phase)

The 2*pi factor cannot be represented exactly, so every time you use it for anything you introduce some error. Since it is not even useful for anything other than sines, it's better to just run your phase in [0,1] and put the scaling into the argument to sine.

[Max M.]

If the sinfloat(x) already have more error rather than sindouble(x), wouldn't this error increment if I apply other FX to the signal later (wave shaping, for example)?

In general it's almost safe to say "an effect" does not "increment input errors" - it's only "adding" its own errors - unless an increment of the input level (obviously this will boost errors too) is exactly what the effect does. E.g. think of a high-gain distortion, bit-crushers and so on, but in these cases the "original" error (even if increased) is still most likely to be totally masked by the errors introduced by the effect itself (like in "-120dBerror + -96dBerror = -95.5dBerror" <- values are purely abstract of course).

Or (even more) if I use that signal as modulation source (FM, for example)?

Sure a signal modulated by "float" sine is more distorted than the same signal modulated by "double" sine. But just like above the question is how significant the difference between them compares to the level of errors introduced by the modulation itself (aliasing and so on).

---
There's no simple rule like "use floats for this, doubles for that etc." (Well, not counting obvious well-known edge-cases like "beware of float direct-form biquads"). It's all about understanding where and how exactly the errors are introduced (incl. how they can actually "sound"), estimating (and/or isolating) the corresponding bottlenecks and then balancing between "error-level/performance". And it's an art of its own (there're whole theses/books dedicated to the error analysis of just a single-multiply/single-add schemes , not even counting more complex algorithms).

---
In this particular case (sine via `sin`) I'd say: use `floats` and don't bother (yet). You'll face much more hard/tricky stuff the further you go.
In a perfect world and in a simple case I'd suggest to write your code so that it can be easily switched between doubles/floats (it's pretty trivial in C++), so when the plugin is nearly finished you could compare them by ears or simply measure (and even make "high-performance/high-accuracy" option). But that's a perfect world (and a simple case).
Probably you'd be interested in some sort of statistic (like what author X uses in his plugin Y) but I doubt they would be very motivated to expose such information directly. However it's not that difficult to guess by finding the corresponding posts of the people of interest (it's pretty obvious - but always remember: they are very good at understanding where a potential error-bottlenecks could be).

[mystran]

(Well, not counting obvious well-known edge-cases like "beware of float direct-form biquads").

Even this "well-known edge-case" is not really that straight-forward, because if the poles and zeroes are far from the unit-circle (or at special positions such as +/- 1 and +/- i), then single-precision is usually good enough even for direct form biquads. Really it's only the low-frequencies and high-Q filters that cause problems most of the time, although both of these are fairly common in audio.

[Nowhk]

Not suggesting anything tricky, but rather pseudocode that looks like this:

Code: Select all

for samples:
   phase += freq / fs
   if phase >= 1: 
      phase -= int(phase)
   out = sin(2*pi*phase)

The 2*pi factor cannot be represented exactly, so every time you use it for anything you introduce some error. Since it is not even useful for anything other than sines, it's better to just run your phase in [0,1] and put the scaling into the argument to sine.

Uhm I suppose phase here is still double (or float), and thus freq / fs (and than sum) got cumulative errors, anyway. Modulo is exact, yes, but the sum and division isn't anyway. It will still drift a bit. Are you saying that's unnoticeable?

In general it's almost safe to say "an effect" does not "increment input errors" - it's only "adding" its own errors

Of course That's why I used to think that if the chain is err1+err2+err3+err4=errFinal... if I can reduce err1 (which is the generated signal, by increasing precision double vs float), of course errFinal is inferior.
But for what I'm seeing from these replies, it seems it doesn't matter (until I find that it matters, haha).

Sure a signal modulated by "float" sine is more distorted than the same signal modulated by "double" sine. But just like above the question is how significant the difference between them compares to the level of errors introduced by the modulation itself (aliasing and so on).

But doesn't the "level of errors introduced by the modulation itself" ALSO depend by the initial error level? If its less, the lesser "errors introduced by the modulation itself" later.
The two "multiply/add" each other I think, they are not "overlapping", such as one that "mask" the other (there's 1 noise level, not 2 differents noise that run in parallel)

Probably you'd be interested in some sort of statistic (like what author X uses in his plugin Y) but I doubt they would be very motivated to expose such information directly. However it's not that difficult to guess by finding the corresponding posts of the people of interest (it's pretty obvious - but always remember: they are very good at understanding where a potential error-bottlenecks could be).