Sub-"Minimum-Phase" Filters?
-
- KVRist
- Topic Starter
- 76 posts since 5 Nov, 2015
While experimenting with IIR oversampling filters, I found that I can take a supposedly minimum phase filter and run its IR through Voxengo Deconvolver with "minimum-phase transform" enabled and get an IR with even less phase shift.
How is this even possible if the original filter is already considered minimum phase?
Is it possible to directly design a filter like this so I don't have to use the MP transform?
Examples:
Here's a 36-Pole Butterworth LP @ 1/4 the sample rate. Before and after minimum-phase transform:
Here's two of the above filter cascaded, MP-transformed + 5 samples of PDC. IMO, that's really not that bad in terms of phase shift.
How is this even possible if the original filter is already considered minimum phase?
Is it possible to directly design a filter like this so I don't have to use the MP transform?
Examples:
Here's a 36-Pole Butterworth LP @ 1/4 the sample rate. Before and after minimum-phase transform:
Here's two of the above filter cascaded, MP-transformed + 5 samples of PDC. IMO, that's really not that bad in terms of phase shift.
-
- KVRAF
- 1607 posts since 12 Apr, 2002
I'm not really an expert in this but here's my guess.
The analog prototype 36pole LP has 36 degrees of freedom (36 complex pole positions under the conjugation restriction, or, for some filters, 36 real positions, or some mixture), however changing any of those will change the response (IIUC), and thus cannot be changed. This specific filter also has no zeros, but even if it had some, those would have been already in the minimum phase position, that's why such analog filters are usually referred to as minimum-phase. So you have only 36 degrees, all of which are "blocked", so effectively you have no degrees of freedom.
Once you get a digital IR, you have as many degrees of freedom as the length of your IR (and technically you may get more poles and zeros than your TF formally has, it's just some of them are cancelling each other). And those are different degrees of freedom than the ones you had in the analog prototype. So it's not fully surprising, that there is still some playroom left.
There might have been some minimal-phase transformation of the analog IR as such (I'm actually not deep enough in the theory to know how would it look), but even if it existed, you would end up with some analog IR which (IIUC) doesn't correspond to any differential system and thus is not implementable (how do you physically implement convolution in the real world???). This is completely different to the digital case, where any finite IR has a corresponding difference system implementing it.
The analog prototype 36pole LP has 36 degrees of freedom (36 complex pole positions under the conjugation restriction, or, for some filters, 36 real positions, or some mixture), however changing any of those will change the response (IIUC), and thus cannot be changed. This specific filter also has no zeros, but even if it had some, those would have been already in the minimum phase position, that's why such analog filters are usually referred to as minimum-phase. So you have only 36 degrees, all of which are "blocked", so effectively you have no degrees of freedom.
Once you get a digital IR, you have as many degrees of freedom as the length of your IR (and technically you may get more poles and zeros than your TF formally has, it's just some of them are cancelling each other). And those are different degrees of freedom than the ones you had in the analog prototype. So it's not fully surprising, that there is still some playroom left.
There might have been some minimal-phase transformation of the analog IR as such (I'm actually not deep enough in the theory to know how would it look), but even if it existed, you would end up with some analog IR which (IIUC) doesn't correspond to any differential system and thus is not implementable (how do you physically implement convolution in the real world???). This is completely different to the digital case, where any finite IR has a corresponding difference system implementing it.
- KVRAF
- 7888 posts since 12 Feb, 2006 from Helsinki, Finland
Mathematically "mininum-phase" means your poles and zeroes are inside the z-plane unit-circle (ie. root magnitude less than 1) or in case of the continuous-time version on the negative half-plane of the Laplace s-plane. If the poles respect this condition, the filter is causally stable and if the zeroes also respect it we call it minimum-phase and we can find a causally stable inverse simply by swapping the poles with the zeroes.
Now, for classic engineering types like LP/BP/HP/Notch you typically place the zeroes on the unit-circle (or imaginary axis and/or "point at infinity" in the case of Laplace), which strictly speaking means that the filter is not really "minimum-phase" in the strict sense of having a stable inverse, but we'll typically still call such edge-cases "minimum-phase" since we can't do any better with the same magnitude response.
Anyway, when you sample an IIR a couple of things happen. It needs to be truncated somewhere, so it's no longer an IIR, but rather a FIR and because it's no longer an IIR it no longer has any poles, but rather just a bunch of zeroes. On top of that, the whole "minimum-phase" vs. "strictly minimum-phase" thing above actually can become important, because methods like log-cepstrum (one of the more reliable ways to do minimum-phase conversion of long FIRs; no idea what Voxengo's Deconvolver uses, but log-cepstrum would be my first guess) are actually ill-defined where the zeroes are exactly on (or numerically speaking "too close to") the unit-circle (since log(0)=-inf). Practically speaking this is not a huge deal, one can permute the response slightly (eg. treat anything below some suitable threshold like -200dB as if it was at -200dB), but it can affect the resulting phase slightly and there's a few other reasons why pretty much any numerical method in general tends to be a bit approximation.
That said.. I suspect what you're seeing can probably be attributed mostly to truncation, where the response of the poles is represented by a finite number of zeroes and then these zeroes are reconfigured into a minimum-phase configuration.
Now, for classic engineering types like LP/BP/HP/Notch you typically place the zeroes on the unit-circle (or imaginary axis and/or "point at infinity" in the case of Laplace), which strictly speaking means that the filter is not really "minimum-phase" in the strict sense of having a stable inverse, but we'll typically still call such edge-cases "minimum-phase" since we can't do any better with the same magnitude response.
Anyway, when you sample an IIR a couple of things happen. It needs to be truncated somewhere, so it's no longer an IIR, but rather a FIR and because it's no longer an IIR it no longer has any poles, but rather just a bunch of zeroes. On top of that, the whole "minimum-phase" vs. "strictly minimum-phase" thing above actually can become important, because methods like log-cepstrum (one of the more reliable ways to do minimum-phase conversion of long FIRs; no idea what Voxengo's Deconvolver uses, but log-cepstrum would be my first guess) are actually ill-defined where the zeroes are exactly on (or numerically speaking "too close to") the unit-circle (since log(0)=-inf). Practically speaking this is not a huge deal, one can permute the response slightly (eg. treat anything below some suitable threshold like -200dB as if it was at -200dB), but it can affect the resulting phase slightly and there's a few other reasons why pretty much any numerical method in general tends to be a bit approximation.
That said.. I suspect what you're seeing can probably be attributed mostly to truncation, where the response of the poles is represented by a finite number of zeroes and then these zeroes are reconfigured into a minimum-phase configuration.
Last edited by mystran on Sat Jan 22, 2022 1:43 pm, edited 1 time in total.
- KVRAF
- 7888 posts since 12 Feb, 2006 from Helsinki, Finland
Also as a practical tip, you probably never want to use Butterworth for brickwall designs. Even if you want to avoid passband ripple, Chebychev type-2 is there for you and even with very small amount of stop-band ripple (eg. you can set it close to your noise floor) it'll still give you a steeper filter for (usually much) lower order compared to Butterworth types... plus the response is actually nicer 'cos the stop-band attenuation is more even across the whole stop-band.
-
- KVRAF
- 1607 posts since 12 Apr, 2002
I think this is not the definition but its practical translation to filters with rational transfer functions. A true definition is probably "a filter having the smallest phase delay among the filters with the same amplitude response", where the minimum might be or usually is taken among some implied set of filters, e.g. all filters with the same pole count (and where there might even be some variation among which phase response is considered to have the smallest phase delay, as the phase delay varies across the frequency range). I'm not sure if it 1:1 maps to the idea of flipping the zeros, especially if we don't restrict the filter set to the rational transfer functions of the same order.mystran wrote: ↑Sat Jan 22, 2022 1:32 pm Mathematically "mininum-phase" means your poles and zeroes are inside the z-plane unit-circle (ie. root magnitude less than 1) or in case of the continuous-time version on the negative half-plane of the Laplace s-plane. If the poles respect this condition, the filter is causally stable and if the zeroes also respect it we call it minimum-phase and we can find a causally stable inverse simply by swapping the poles with the zeroes.
- KVRAF
- 7888 posts since 12 Feb, 2006 from Helsinki, Finland
This is the Wikipedia definition:Z1202 wrote: ↑Sat Jan 22, 2022 1:56 pmI think this is not the definition but its practical translation to filters with rational transfer functions.mystran wrote: ↑Sat Jan 22, 2022 1:32 pm Mathematically "mininum-phase" means your poles and zeroes are inside the z-plane unit-circle (ie. root magnitude less than 1) or in case of the continuous-time version on the negative half-plane of the Laplace s-plane. If the poles respect this condition, the filter is causally stable and if the zeroes also respect it we call it minimum-phase and we can find a causally stable inverse simply by swapping the poles with the zeroes.
For rational transfer functions the conditions for the placement of poles and zeroes follow trivially from the above.https://en.wikipedia.org/wiki/Minimum_phase wrote: In control theory and signal processing, a linear, time-invariant system is said to be minimum-phase if the system and its inverse are causal and stable.
edit: That said, I'm obviously aware that a slightly relaxed definition is often used (namely that the inverse be causal and stable except for zeroes exactly on z-plane unit-circle / s-plane imaginary axis), hence why I personally usually like to use the term "strictly minimum-phase" where the existence of causal and stable inverse everywhere is of practical importance.
-
- KVRAF
- 1607 posts since 12 Apr, 2002
Even if this is the official definition, I believe some justification of such definition of the term "minimum-phase" would be in order. Wikipedia has a justification for rational transfer functions, which kinda makes sense (although leaves some questions still open), but IIUC (just briefly looked through the text) is missing a general justification for arbitrary systems.mystran wrote: ↑Sat Jan 22, 2022 2:35 pm This is the Wikipedia definition:https://en.wikipedia.org/wiki/Minimum_phase wrote: In control theory and signal processing, a linear, time-invariant system is said to be minimum-phase if the system and its inverse are causal and stable.
In fact, it would look to me way more appropriate if the definition would explicitly speak of the system's phase response function and then there would be a theorem that a system is minimum-phase if and only if the system and it's inverse are both causal and stable.
- KVRAF
- 7888 posts since 12 Feb, 2006 from Helsinki, Finland
I don't know about official definitions, that's just the most common one that I've seen.
That said, the point I was trying to make was not to argue definitions, but rather the opposite: to raise awareness of the fact that sometimes "minimum-phase" refers to minimizing the group delay (in which case it is acceptable to have zeroes on the z-tranform unit-circle or Laplace imaginary axis), where as in other situations you actually need a causal and stable inverse (or related properties like a finite log-spectrum) and it is therefore sometimes important to pay attention to which definition is expected.
Like I said, personally I prefer the term "strictly minimum-phase" (which as far as I can tell is also in common use with precisely this meaning) in the case where the more "strict" definition (no marginally stable zeroes) is important.
edit: For example https://inst.eecs.berkeley.edu/~ee225a/ ... lement.pdf gives a definition for minimum-phase as "stable, causal, all zeroes either inside or on the unit-circle" and then additionally states "The sequence is strictly minimum phase [emphasis from the original] if it is minimum phase and also has no zeros on the unit circle. An LTI system is said to be minimum phase if its impulse response is a minimum-phase sequence." and then goes on to state that a strictly minimum phase has a stable, causal inverse.
That said, the point I was trying to make was not to argue definitions, but rather the opposite: to raise awareness of the fact that sometimes "minimum-phase" refers to minimizing the group delay (in which case it is acceptable to have zeroes on the z-tranform unit-circle or Laplace imaginary axis), where as in other situations you actually need a causal and stable inverse (or related properties like a finite log-spectrum) and it is therefore sometimes important to pay attention to which definition is expected.
Like I said, personally I prefer the term "strictly minimum-phase" (which as far as I can tell is also in common use with precisely this meaning) in the case where the more "strict" definition (no marginally stable zeroes) is important.
edit: For example https://inst.eecs.berkeley.edu/~ee225a/ ... lement.pdf gives a definition for minimum-phase as "stable, causal, all zeroes either inside or on the unit-circle" and then additionally states "The sequence is strictly minimum phase [emphasis from the original] if it is minimum phase and also has no zeros on the unit circle. An LTI system is said to be minimum phase if its impulse response is a minimum-phase sequence." and then goes on to state that a strictly minimum phase has a stable, causal inverse.
- KVRist
- 91 posts since 24 Dec, 2015 from Bristol, UK
A (wrapped) phase plot of a delay of, say, 4 taps would look like a saw tooth. You have a delay of 4 taps in your impulse response. So I don't think the phase is as different as you think between the two. Try taking your delay out.
- KVRAF
- 7888 posts since 12 Feb, 2006 from Helsinki, Finland
Related to this... raw phase-plots like these are very hard to read visually, because all the oscillation tends to obscure whatever useful information there might be... and that information is really in the slope of the plot, which is further warped by the logarithmic frequency axis...
In order to judge these things visually, I'd generally recommend plotting group delay instead. By comparing two group delay plots, it'll be much more obvious if they are (more or less) the same up to bulk delay, or whether there's something more significant going on.
-
- KVRist
- Topic Starter
- 76 posts since 5 Nov, 2015
This is true, once you compensate for the unit delays, the difference is not that great.
BUT the MP-transformed version has a much shorter leading edge on the IR and thus less delay.
I'm aiming for absolute minimum delay, minimum phase-shift and zero pre-ringing. The 2nd image appears to be approaching or at the limit of what's possible with those three requirements.
I can reduce the phase-shift further with some allpass filters (3rd image), but then rippling starts to appear on the front side of the IR. This makes sense intuitively, since the IR will become more symmetrical as it transitions from minimum-phase to linear-phase.
---------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------
-
- KVRist
- Topic Starter
- 76 posts since 5 Nov, 2015
This only applies to IIR right? I'm no longer going to use an IIR directly.
At this point I just want to derive a minimum-phase FIR.
It appears so, at least in 2007:
viewtopic.php?p=2821548#p2821548
I'm trying to follow your instructions here:
viewtopic.php?p=8335873#p8335873
Instead of starting from an IR, I'm defining the magnitude directly.
https://www.desmos.com/calculator/5j15sm3jow
Where I'm stuck is: "take complex(!) exponent of each bin". Do you mind elaborating on that? I'm sure I did it totally wrong.
The real and imag values I'm getting before the final iDFT don't make sense to me.
- KVRAF
- 7888 posts since 12 Feb, 2006 from Helsinki, Finland
Like I tried to point out earlier already, it's probably an artifact of a numerical method, which gives you an IR which is practically, but not analytically the same.MrBeagleton wrote: ↑Mon Jan 31, 2022 9:01 pmThis is true, once you compensate for the unit delays, the difference is not that great.
BUT the MP-transformed version has a much shorter leading edge on the IR and thus less delay.
Some more Wikipedia: "The most general causal LTI transfer function can be uniquely factored into a series of an all-pass and a minimum phase system."I'm aiming for absolute minimum delay, minimum phase-shift and zero pre-ringing. The 2nd image appears to be approaching or at the limit of what's possible with those three requirements.
I can reduce the phase-shift further with some allpass filters (3rd image), but then rippling starts to appear on the front side of the IR. This makes sense intuitively, since the IR will become more symmetrical as it transitions from minimum-phase to linear-phase.
Your minimum-phase IR already has the minimum possible group delay for the given response. You can't really reduce the phase-shift as such, but rather you can add more phase-shift in order to make it more linear, which is essentially the same thing as adding more delay (since delay is just linear phase-shift) and as you've observed, this will also result in more and more pre-ringing.
So you have conflicting goals: you can either minimize the group delay, or you can maximize the linearity of the phase-shift. Improving one necessarily makes the other one worse, so you need to pick a trade-off.
- KVRAF
- 7888 posts since 12 Feb, 2006 from Helsinki, Finland
Well, sort of. The point is that Butterworth response is not really the best choice if your goal is to get a brickwall filter, because the magnitude response is monotonic (ie. it never goes all the way down to zero, except at Nyquist). For brickwall filters, even if you want the passband to be flat, it makes sense to let the response in the stop-band oscillate about zero.MrBeagleton wrote: ↑Mon Jan 31, 2022 9:34 pmThis only applies to IIR right? I'm no longer going to use an IIR directly.
At this point I just want to derive a minimum-phase FIR.
That was more or less a "quick outline" more than a "reference implementation" and I wrote it from memory, so perhaps I forgot some detail (eg. you probably at least need some 1/N scale factors with either your forward or inverse FFTs).I'm trying to follow your instructions here:
viewtopic.php?p=8335873#p8335873
exp(a+i*b) = exp(a)*(cos(b)+i*sin(b))Where I'm stuck is: "take complex(!) exponent of each bin". Do you mind elaborating on that? I'm sure I did it totally wrong.
Alternatively if you're writing C++ you can just use std::exp with std::complex<double> (or float, but really you should probably use doubles here) and it'll do the right thing.
-
- KVRist
- Topic Starter
- 76 posts since 5 Nov, 2015
It still makes sense to me as a relative layman, but I see your point. I'll go further and say I want zero pre ringing/rippling. The remaining two parameters I want to "minimize" in context of each other. But there's some wiggle room. Like I would allow slightly more delay if it reduced the phase-shift (after delay compensation), but not at the cost of pre-ringing.mystran wrote: ↑Mon Jan 31, 2022 9:42 pm Your minimum-phase IR already has the minimum possible group delay for the given response. You can't really reduce the phase-shift as such, but rather you can add more phase-shift in order to make it more linear, which is essentially the same thing as adding more delay (since delay is just linear phase-shift) and as you've observed, this will also result in more and more pre-ringing.
So you have conflicting goals: you can either minimize the group delay, or you can maximize the linearity of the phase-shift. Improving one necessarily makes the other one worse, so you need to pick a trade-off.
Voxengo Deconvolver seems to get me there. I'm hoping to confirm the log-cepstrum method does the same, since we're not 100% sure what Deconvolver uses. I just want to be able to do it myself and not rely on Deconvolver. I have other ideas that require minimum phase filters of arbitrary magnitude, beyond just oversampling filters.