High Pass Filtering for a Square or Clipped signal

[Ross21]

Hey all,

I had input a 'none clipping' signal into an IIR linear HPF filter and received a clipped signal. Not exactly what I had expected . Asking for some deep explanation.

The input file is a pure 100Hz square wave at -3 DbFS (so it doesn't clip).
However, when I run an HPF (100Hz) on it - it does clip.

I have attached a python plot showing this phenomena. It occurs with different filters (2nd or 3rd order).

More interesting is what should I do to avoid this in a multi-band cross over with real music signals.
My idea is to build a multi-band cross-over (say three bands) in which the mid-band runs an HPF and LPF (naturally). I was surprised to see how much this band clips if the input is a pre-recorded clipped audio signal (like square), which by itself doesn't clip {like a square wave a bit lower than 0 dBFS}.

Any idea how to design a multi-band limiter / HPF system which doesn't create such clipping ?

In one image you can see the input at 0 dBFS (blue) with its output (orange).
On the second image the input is -3 dBFS (blue) with its output (orange).
Both orange signals 'clip'.

[BertKoor]

Firstly, it doesn't (yet) clip but the output peak amplitude is higher. This is due to the abrupt level change in the original signal when the square goes from -1 to +1 and back to -1 again. If you calculate the power of the signal, it's probably of the same power as what was put in. The filter has smeared the phases of harmonics causing the peak to be higher.

In short, a square wave of this amplitude is not a realistic test signal. It won't clip until it's converted from floating point to integers.

[mystran]

Any idea how to design a multi-band limiter / HPF system which doesn't create such clipping ?

You turn down (or let the user turn down) the gain (and that's what limiters in general do). If that sounds like a joke, it's really not.

The worst-case peak magnitude of any filter is the sum of the absolute magnitudes of the impulse response samples, but really the only sensible way to deal with this is to design your complete signal flow in such a way that you always have enough headroom for whatever expansion you end up with (at least until you intentionally want to distort things, but that's obviously another thing).

[mystran]

The filter has smeared the phases of harmonics causing the peak to be higher.

You actually do not need any phase-smearing to get higher peaks. As long as you have an impulse response with both positive and negative samples (even if it is linear phase; note that this is necessarily the case for any high-pass filter; edit: well, assuming the sum of absolute values of the impulse response samples is higher than unity, but that's usually always the case if the filter has unity response anywhere), there exists some signal for which the peak level will increase.

[Ross21]

For the sake of this discussion let's assume that a clipped signal is "over 1.000 or under -1.000".

Yes, a limiter (or even clipper), or user intervention will do the job. My question should have been more focused such as:
* Is this phenomena related to IIR? Will it reproduce with FIR ?
* Any known method to eliminate it without a limiter/clipper/user? Like, maybe an all pass filter which will 'smear' these sharp peaks?
* If filter would run at very high sample rate (relative to signal's frequency) would that do the trick?
* Or maybe if it run at very low frequency rate (Vs. signal) that would eliminate it?

I'm after an ideal solution (if any...) in which the filter will output a 'none clipped' signal for a 'none clipped' input of any kind - without "external intervention" of other none linear algorithms, or user.

Its more of a theoretical question.

[imrae]

I don't see how oversampling would help. I think this is something that naturally arises when messing with the harmonic structure of a saturated signal.

It might be an interesting exercise to take an FFT and use an optimisation algorithm to modify the phases, minimising the peak level of the inverse-FFT-reconstructed signal.

Not much use in realtime as you'll get discontinuities at the window, but maybe you'll find an all-pass filter shape that helps?

[mtytel]

Yeah the square wave peaking at -1 to 1 is not a very good test for a few reasons.

You might be surprised to know that the fundamental of your square wave is *above* 0dBFS. That means if you had a perfect filter that just isolated the fundamental frequency, it would clip after filtering.
Here's a quick demonstration of what I mean with the fundamental and the 3rd harmonic. Notice how the fundamental clips but adding the harmonic brings it back: https://www.desmos.com/calculator/sxlgfshuom

Also if you square wave is peaking at -1 to 1 and you're also anti-aliasing it, you will most likely clip because you need to introduce a little wobble for the anti-aliasing which will bring it outside your bounds.

But back to your filter.
Your signal is now clipping because the IIR filter shifts the phase of each frequency differently.

If you want a filter that avoids phase shifting, you want a 'linear-phase' filter. You can make a linear-phase FIR filter and you can get an approximate linear-phase IIR filter. These linear-phase filters come at the cost of higher latency or higher order though.

[Ross21]

Good explanation, thanks.

[mystran]

* Is this phenomena related to IIR? Will it reproduce with FIR ?

It happens with both.

* Any known method to eliminate it without a limiter/clipper/user? Like, maybe an all pass filter which will 'smear' these sharp peaks?

Turn down the gain.

Seriously, pretty much anything you do (eg. all-pass filters) will only ever tend to make it worse (at least for some signals).

* If filter would run at very high sample rate (relative to signal's frequency) would that do the trick?

No. In fact, you'll probably get some more peak level increase from the oversampling filters and signals such as your digital square-wave actually have higher peak levels in their continuous-time reconstruction between the sampling points anyway.

I'm after an ideal solution (if any...) in which the filter will output a 'none clipped' signal for a 'none clipped' input of any kind - without "external intervention" of other none linear algorithms, or user.

For general filters there is none.

If the impulse response is strictly non-negative (ie. either positive or zero everywhere), then the sum of the absolute magnitudes is equal to the DC gain, but such filters are typically low-pass filters (or multi-tap comb filters, but those don't help with this application either) with rather poor frequency selectivity, so usually only useful for time-domain smoothing.

[mystran]

But back to your filter.
Your signal is now clipping because the IIR filter shifts the phase of each frequency differently.

WRONG.

The signal level increases because the sum of the absolute magnitudes of the filter impulse response samples is higher than unity. As long as this is the case, even if the filter is linear-phase, the worst-case peak level will be higher than the input level. As it turns out, this happens with pretty much every useful audio filter.

In fact, the absolute worst-case signal (resulting in the highest possible peak levels) is actually easy to construct too: you take the impulse response, time-reverse it and replace every sample with it's signum (ie. +1 or -1 depending on the sign). The resulting signal will cause the filter output to peak at the sum of the absolute magnitudes of the impulse response coefficients.

[mtytel]

The signal level increases because the sum of the absolute magnitudes of the filter impulse response samples is higher than unity. As long as this is the case, even if the filter is linear-phase, the worst-case peak level will be higher than the input level. As it turns out, this happens with pretty much every useful audio filter.

Yeah this is the best way to see if a given filter will ever increase the peak of any given incoming signal.

WRONG.

I was addressing the OP's confusion though maybe I wasn't clear.
If you think of your filter's input signal as the sum of a passband and a stopband, a linear phase filter's output will only clip if the original passband signal clips.
However a non-linear phase filter's output may clip even if the original passband signal doesn't clip.

I'll take another step further (though I may be WRONG). Since signals in music tend to have a high power-to-peak ratio, I would say that non-linear phase filters tend to increase the peak more than linear-phase filters.

[mystran]

WRONG.

I was addressing the OP's confusion though maybe I wasn't clear.
If you think of your filter's input signal as the sum of a passband and a stopband, a linear phase filter's output will only clip if the original passband signal clips.

Well, the individual bands can still "clip" even if you use filters that sum back to the original signal.

[Ross21]

Can you please explain this:

The signal level increases because the sum of the absolute magnitudes of the filter impulse response samples is higher than unity. As long as this is the case, even if the filter is linear-phase, the worst-case peak level will be higher than the input level. As it turns out, this happens with pretty much every useful audio filter.

Is this correct-->
Input: Impulse response (00...00100..00)
System: The filter
Expected output: Most sample will be '0' but some would be none '0' due to the impulse

Sum all 'absolute values' of the samples up, if the result is higher than one, then the filter's output might be higher than '1' (depending on input of course).
However if the above result is lower than '1' (or equal to ??) , the filter will neve output a result higher than '1' for any 'accepted' input ['accepted input' being "every sample must be between -1.0 and 1.0"]

Correct ?