Like I noted above, you can easily test this: feed it a band-limited saw-wave with slow(!) frequency sweep and look at the analyzer. If you see partials moving in different directions, you have aliasing (ie. the interpolation kernel is not good enough), if it's just a "general mess" then it's probably another issue. In the latter case, the kernel response is likely varying too much depending on the sub-sample position. This could happen if your kernel doesn't have enough taps (per branch) to maintain a good response when shifted (though I don't see this as a huge problem with any practical kernel length), or it could be because you don't have enough branches and/or your interpolation for "in between" values is poor and leads to noise (see below)... or it could even be a simple programming error, who knows.matt42 wrote:This is a big part of my question. Other than aliasing, due to poor bandlimiting, can a kernal cause distortions and how? Take linear interpolation; it's just a triangular kernel. Sure it doesn't bandlimit well and will filter the pass band, but would it add distortion to source signal (assuming source is band limited to target nyquist) if we use it to down sample to a target sample rate?mystran wrote:It can be aliasing or it can be noise from the kernel interpolation.
What I mean by varying response is that when you have something like a triangular kernel, when you sample it at the middle to get get [..,0,1,0,...] the response is completely flat at unity. Then when the center falls in the middle of two samples, you get [..,0,0.5,0.5,0,..] which places a zero at Nyquist (ie. the response is very gradual low-pass that goes all the way to zero). In between these two extremes you get something in between. If you then sample "randomly" at different points, you get noise as the response varies. This noise is at the high frequencies (where the variation happens), so such a triangular filter is only really a good interpolator at low frequencies (for which increasing the resolution or "oversampling" helps).
Ultimately though, it's impossible to say what your problem might be. You simply need to try varying one parameter at a time, look at an analyzer with a test signal and see what happens. It's worth noting that this type of stuff tends to be really sensitive to details, so it's relatively easy to also make small oversights in code that end up nudging some value somewhere slightly, resulting in seemingly "working" but slightly noisy situation. You could even have some half-sample alignment issue with your windowed sinc-design that could cause sub-optimal performance (been there, done that)... so unfortunately "it's complicated"
