Trying to understand oversampling

[matt42]

Hi JCJR,

The problem with what you trying to do is finding an interpolator that is cheap, attenuates the stop band and leaves the pass pand alone. Assuming you find one and also assuming you're using a FIR filter you'll need kernal size times upsampling factor multiplications to implement the filter. If we zero pad the number of multiplies is equal to kernal size. In other words if we found an interpolator it would need to attenuate enough that the FIR kernal size could be halved for x2 upsampling, or quartered for x4 and we'd still be loosing out the cost of the interpolation itself. Maybe this is possible for lower up sampling factors, but it'd be hard to imagine at say x8 or x16.

[Miles1981]

What is its amplitude? Infinity. What is its integral? Zero. What is zero times infinity? Zero. (Or, unknown. Doesn't matter.)

It's called a dirac, and it has an integral of 1.

[stratum]

The rest sounds good, but about the first sentence: Linear interpolation is equivalent to zero stuffing then running through a pretty crappy low pass filter (with a triangular-shaped impulse response) that droops at the high end and has poor stop-band rejection (aliases); then you say to run that through a better lowpass filter. Why not zero stuff and go right to the good filter?

Hi,

I had taken that idea from the well known line6 patent regarding how to obtain alias-free distortion.
It may be the case that patents in general aren't good educational material as they have other purposes. Probably back in 90's the devices they had produced did not have as much as processing power as they probably have today. Somewhere in the book 'dspguide' the fact that linear interpolation is a lowpass filter is stated, but I have never investigated whether it is a serious issue for specific purposes or not. For a guitar signal it does not look like one, since guitars are not wide bandwidth instruments anyway. I do not recall what was the exact value but the patent file had specified a rather low sampling rate for the ADC, and in spite of that they further state that preferred oversampling method involves a linear interpolation, and that makes sense to me, as the problem they are solving is not about an audio post production system requiring hifi quality audio.

As for an idea with intuitive appeal it makes sense to say 'connect the dots and filter and that's over sampling', as it would be much harder to explain why the idea of filling the missing samples with zeros is the theoretically correct way as that would require telling a bit more about the subject than simply saying that doing so would keep the frequency response flat. After the rather long thread about DFT that we recently had, I can see why such explanations aren't common in forums - not only it requires effort but the discussion also goes through 'bumpy roads', so to speak.

edit : I can see this thread has taken a few steps through that 'bumpy road' , already. Eventually it can converge to a good explanation I guess. Doing so would require explaining the sampling theorem and that would eventually explain why filling the missing samples with zeroes is not doing the signal anything as the signal itself is an impulse train with different amplitudes (it is also necessary to explain what is right and wrong about this last sentence).

[Aleksey Vaneev]

Somewhere in the book 'dspguide' the fact that linear interpolation is a lowpass filter is stated

It's a bit more complex than that. Linear and spline interpolators include both low-pass filtering AND fractional delay filter. In such system, the low-pass characteristic changes with the fractional delay position. Such simple interpolators have NO low-pass filtering at zero fractional offset and full low-pass filtering at 0.5 fractional position. For example, if you use splines for resizing images, the resulting image will have a wavy appearance: with sharpened and softened areas.

[earlevel]

I tried such things in the past, hoping that filling the gaps with something a little smoother than zeros might allow good performance with a quicker cheaper main brickwall filter.

Hi Jim,

Well, the problem as you know is that the cheap filters (smoothing) causes other issues (hi-freq droop, etc.), and a leisurely roll off into the stop band—characteristics of linear interpolation (the spectrum of which is a sinc squared function), for instance. Now, even if you don't care to correct for the droop in the pass band, mostly likely your prime motivation for the "main brickwall filter" is to sharpen up the transition band—and that's typically where that filter gets costly anyway. In the end, the linear interpolation hurts the passband response, and the only improvement is getting more net attenuation further into the stop band, where it matter less than at the beginning. So, it's almost always best to the urge to smooth things out. With zero stuffing you'll get no roll off of the aliases, but at least that means you're starting with a flat response.

[earlevel]

Ok, I get it work,
The filter cutoff must be at 0.25 instead 0.5
The problem is that i need a lot of taps to remove some aliasing (with a x^2 input signal) .
To get 4x os, the filter cutoff at 0.125.
I think this is not the right way to do it.

Misc tips, on the number of taps issue:

As mentioned, you can do better with two stages. Now, your first thought might be that it sounds like you're doubling the work. But it takes, basically, half the number of taps to do a 0.25 cutoff as it does a 0.125 cutoff with the same relative specs. So, you're next thought might be that ok, so you break even. But, you can do better than that, because both stages needn't have the same specs—you can let one allow aliasing through a bit as long as the other cleans it up.

Now, normally you'd want those cutoffs to be a bit less than 0.25 (or 0.125), or you'll let some aliasing through. But in multi-stage (multi-rate) processing, you can often go with 0.25 (and clean it up with another stage), and you'll end up with zero coefficients (0.5 gives every other coefficient zero, etc.), which also shortens the number of coefficients you really need to use.

Also, you can use an equiripple FIR design. If you allow a small fraction of a dB ripple in the passband, the filter gets a lot shorter.

[Aleksey Vaneev]

Also, you can use an equiripple FIR design. If you allow a small fraction of a dB ripple in the passband, the filter gets a lot shorter.

I do not think this is a rule. Equiripple design usually narrows the transition band, makes the transition from pass-band to stop-band shorter and steeper, but it does not offer a better stop band attenuation. Mainly filter length affects the stop-band attenuation, so if you make filter shorter you will always have a lower stop-band attenuation.

[earlevel]

Also, you can use an equiripple FIR design. If you allow a small fraction of a dB ripple in the passband, the filter gets a lot shorter.

I do not think this is a rule. Equiripple design usually narrows the transition band, makes the transition from pass-band to stop-band shorter and steeper, but it does not offer a better stop band attenuation. Mainly filter length affects the stop-band attenuation, so if you make filter shorter you will always have a lower stop-band attenuation.

I'm not sure if you're misunderstanding what I'm saying. But in either case, figure the specification you need—particularly transition band width, stop band attenuation. Design the shortest windowed-sinc filter you can to achieve that. Now do the same with equiripple, giving up a bit of pass band ripple. The latter will be shorter, if you're not doing something wrong, at least in any useful specification for this purpose I can think of.

[earlevel]

Somewhere in the book 'dspguide' the fact that linear interpolation is a lowpass filter is stated

It's a bit more complex than that. Linear and spline interpolators include both low-pass filtering AND fractional delay filter. In such system, the low-pass characteristic changes with the fractional delay position. Such simple interpolators have NO low-pass filtering at zero fractional offset and full low-pass filtering at 0.5 fractional position. For example, if you use splines for resizing images, the resulting image will have a wavy appearance: with sharpened and softened areas.

You are right about the delay issue, which is a problem is variable fractional delay system, but it doesn't come into play in this case.

Linear interpolation is identical with a convolution with a triangular kernel. In the case of upsampling by a constant ratio, there is no shifting. Say up sampling by 2x: The coefficients would be { 0.0, 0.5, 1.0, 0.5, 0.0 }. We can ignore that zeros, of course, and step through at the output sample spacing, equivalent to half-stepping the input. So, the output will either be an input sample, or average of two adjacent samples (0.5 * x[n-1] + 0.5 x[n]). You may say that mean that every other input sample is unfiltered, and the other are filtered, but if you want to look at it that way you also have to consider the modulation by the sample rate. In any case, the impulse response is still a triangular function—the kernel shape—and the frequency response is the sinc^2 function.

To be clear: I'm saying that you are correct in the point you bring up, and in the general case of linear interpolation you have to consider the position-dependent filtering problem. But that doesn't come into play in a constant delay, constant rate conversion scenario like constant sample rate conversion by an integer ratio. In this particular case, linear interpolation is exactly a crappy lowpass filter as stratum brought up.

[Aleksey Vaneev]

Also, you can use an equiripple FIR design. If you allow a small fraction of a dB ripple in the passband, the filter gets a lot shorter.

I do not think this is a rule. Equiripple design usually narrows the transition band, makes the transition from pass-band to stop-band shorter and steeper, but it does not offer a better stop band attenuation. Mainly filter length affects the stop-band attenuation, so if you make filter shorter you will always have a lower stop-band attenuation.

I'm not sure if you're misunderstanding what I'm saying. But in either case, figure the specification you need—particularly transition band width, stop band attenuation. Design the shortest windowed-sinc filter you can to achieve that. Now do the same with equiripple, giving up a bit of pass band ripple. The latter will be shorter, if you're not doing something wrong, at least in any useful specification for this purpose I can think of.

My point is that the equiripple filter won't be shorter for the given stop-band attenuation. With equiripple you can only adjust transition band's width: this means that with equiripple you can in practice increase the pass-band width, that's the only application of equiripple I can think of. This is useful for crappy telephony 4kHz limited frequency range, but for pro audio equiripple is close to useless.

[mystran]

The only sensible way to increase the attenuation of a windowed sinc design is to use a better window. You can get your attenuation down to numeric precision this way for pretty much any sensible filter length, the only reason you need longer filters is to get the transition width down (since it grows wider with more attenuation, obviously).

The problem I personally have with equiripple designs (or in fact any kind of design with pass-band ripple) is that if you put 10 plugins in series and every one of them causes 0.1dB ripple then suddenly you have 1dB ripple at the other end of the chain. It's most obvious if they all use the exact same filters, but even if they don't you can still get some unlucky peaks. This problem doesn't really exist with filters that have a flat pass-band, since even if they lost a little bit of high frequencies, one can easily correct this (eg. when mixing) with a simple high-shelf if it ever becomes a problem... where as good luck trying to correct the mess that you get with a bad mix of equiripple designs.

edit: If you get more attenuation from a windowed sinc by increasing the filter length, then your filter is probably too short for your chosen window.. or you are using a "sub-optimal" window with spectral decay and relying on the decay to get you the attenuation.. which is sometimes workable I suppose, but like .. in general if your filter is long enough, using a better window is the best way to get more attenuation

[earlevel]

My point is that the equiripple filter won't be shorter for the given stop-band attenuation. With equiripple you can only adjust transition band's width: this means that with equiripple you can in practice increase the pass-band width, that's the only application of equiripple I can think of. This is useful for crappy telephony 4kHz limited frequency range, but for pro audio equiripple is close to useless.

Maybe you're right. Anyway, they were some things for the OP to try.

However, I had this discussion with robert bristow-johnson (aka r b-j—IIRC, he prefers to lowercase his name, a la fred harris), who knows filters well, years ago on the music-dsp list. I had probably said something about not finding much benefit to equiripple designs—I had tried before and found only an insignificant improvement in length at best when developing products a couple of decades ago. Robert disagreed, saying they would be much shorter at a given spec. I went back and tried some filters, and he was right for all the ones I tried.

Anyway, I don't have time to check right now, and it's not important to me, just wanted to put the caveat out for others. Ultimately, it's pretty easy for anyone to check—make their windowed-sinc solution, then go find a Parks-McClellan calculator online and see if it will do better. Go with what works best. Personally, I almost always want to calculate my filters at run time, so I always use windowed sinc (with Kaiser window).

(For anyone who wants to check the msic-dsp mailing list thread I referred to, I did find it in my email: Late December 2010, "Interpolation for SRC, applications and methods". Least squares was discussed too, but I believe I checked those length against Park-McClellan—I could have a faulty memory, and maybe it was least squares.)

And a good point form mystran on the accumulated ripple. Anyway, just an idea (one that I've never used personally), the OP seemed distressed about the coefficient length...

[edit] PS—Did a quick search while waiting on a lengthy install: "Even though the Kaiser window method yields a larger filter order for the same specifications [compared with equiripple], the algorithm is less computationally expensive and less likely to have convergence issues when the design specifications are very stringent." http://www.mathworks.com/help/signal/ex ... esign.html

[Aleksey Vaneev]

[edit] PS—Did a quick search while waiting on a lengthy install: "Even though the Kaiser window method yields a larger filter order for the same specifications [compared with equiripple], the algorithm is less computationally expensive and less likely to have convergence issues when the design specifications are very stringent." http://www.mathworks.com/help/signal/ex ... esign.html

The problem in what "specification" means. Equiripple is substantially different "specification" to flat frequency response. So, you can't compare equiripple to Kaiser windowing without making such a wide assumption. Kaiser windowing is quite universal, and I use it in several instances. It is good in that you can easily trade between transition band width and stop-band attenuation for the given filter order. Indeed, if you compare Kaiser windowing and equiripple windowing at the same transition band width, the equiripple will provide a better stop-band attenuation. Or if you fix both the transition band width and stop-band attenuation, you can get a shorter filter order with equiripple. But in my opinion for pro audio it is not worth it. I do not think you can reduce filter order considerably without introducing a huge (audio-wise) ripple.

[S0lo]

Sorry, zero padding doesn't have notches or even a slope. Zero padding retains the original frequencies plus spectral mirroring above nyquist.

Do you mean original nyquist or upsampled nyquist?

If original, then if the original signal wasn't bandlimited under it's nyquist (i.e aliasing), Will the zero padding images cross original nyquist into audible range? if so, then filtering wouldn't remove that part of aliasing, would it?

[earlevel]

Or if you fix both the transition band width and stop-band attenuation, you can get a shorter filter order with equiripple.

Yes, that's precisely what I was saying, no more.

But in my opinion for pro audio it is not worth it.

Like I said, I've never used it. But if I absolutely needed those cycles to get the performance I needed (that would be far more likely in a 1-sample buffered dedicated DSP chip implementation, not likely in a mac/pc host-processor-based system because you can't run that close to the edge), especially in a stomp box where you won't be chaining a bunch of identical units (though it doesn't happen often with plugins either), I wouldn't hesitate at using that tool