Oversampling LPF question

[earlevel]

I don't understand how you think they are "exactly equivalent". Is it a language issue? You emphasize exact equivalence twice. Do you consider an encrypted file as exactly equivalent to the original file? It's not. The encrypted file contains the information of the original file, but is not equivalent to it. For one thing, the original file doesn't contain the information of the encrypted file, which must be true if they are equivalent. If A = B, then B = A.

Encryption is a poor example, as there is additional secret information (the key) in the mix. A better example would be lossless compression: even though the files (in the sense of byte streams) are not the same, the "signal" (ie. the logical file stored) is still the same.

Does a file become something else because you put it inside a zip? No. Does it become something else when you download it over the internet (so it's split and encapsulated into TCP or UDP packets)? No. For all intents and purposes it is the same file... unless you are concerned by the implementation of the codec or the networking protocol.

I gave a good example with AM radio, which you didn't address. The encryption example was not an analogy to digital, it was a question about whether you considered something repacked but containing the same material as "exactly equivalent"—your emphasis with italics.

And.. that's the point I'm trying to make: if we are not concerned about the actual dirty details of converting between digital and analog domain, but rather we are only interested in the band-limited signal that we can (theoretically) convert back and forth in a lossless fashion, then we can treat the signals as being equivalent.

In a discussion about the meaning of the sampled data, you said the samples and the continuous signal were "exactly equivalent". Twice in the same post. In a discussion about digital signal processing. Now you're saying they are the same once the digital data is encoded, and that's all we care about. Of course it's not all we care about, we do digital signal processing, this is forum about DSP.

Sorry, there's no way to undo what you've said, if you won't admit that it as wrong. This just looks like spin.

While we can't store every possible time-instant of a continuous signal in finite memory, we can (theoretically in the limit and given exact arithmetics) obtain the value of the band-limited signal at any arbitrary point in time from only the Nyquist samples. There is no mathematical difficulties here, only practical ones.

There are no mathematical difficulties if the samples represent. Which you've told me is this thread, is incorrect. I gave you the take from Crochiere and Rabiner, which you didn't address, but by extension think they are wrong. You seem to be trying to change from telling my my understanding of sampling theory is fundamentally wrong to treating it as if we were having a philosophical discussion and you're just expressing your view on life.

Sorry, I'm not trying to badger you on this. But you've said I'm wrong and won't give any mathematical reason.

Ofcourse in practice we can't do this perfectly, so it would be closer to the situation with a lossy codec. If we store a picture as a JPEG or a movie as an MPEG, then certainly there is some generation loss in practice... but from the logical point of view it's still the same picture or the same movie, just slightly degraded due to (what I previously called) "implementation details."

Not a good analogy. You can look at a displayed picture. You can't look at the numbers in the file and see the picture. But audio is far different on top of that, the eye is fine with discrete data.

ps. Also.. for the most part when I say "discrete-time" or "band-limited continuous-time" I am not concerned about actual "digital" or "analog" signals, but rather two signals that exist in the realm of pure mathematics. We can defined a bijection between the two domains and declare said bijection as an equivalence relation. All I'm trying to say is that whether or not this is helpful or counter-productive depends on the situation.

This all started with me making statement that you said were fundamentally wrong. Now you're saying it's philosophy or something. And you're contrasting that with "pure mathematics", but you don't present any math. You disagree with my math but don't tell me how. I present you similar mathematics from a text book, you don't address it.

PS—Let me retract the "wrong" part. At least, skimming the old part of the thread, probably Z1202 was closer to saying that. You've questioned the usefulness of telling it like it is (it's PCM, it's a pulse modulation—no, not viewpoint, it's what made the samples), for some reason, preferring a philosophical "the samples are the signal, they are one and the same" zen thing. But this is a discussion about sample rate conversion. Understand that the samples are impulses, and the frequency domain implications (sidebands) is of utmost usefulness in designing oversampling systems (and also non-linear processing). Viewing it as just the signal is not helpful in the context of this thread.

[Tone2 Synthesizers]

Come on guys. Isn't life too short for this

[earlevel]

(oops, inadvertent post deleted)

[BertKoor]

[reaction to last post on page 4]
The same is true btw for an analog transfer or recording: equivalent up to certain limits, but never identical.

[earlevel]

[reaction to last post on page 4]
The same is true btw for an analog transfer or recording: equivalent up to certain limits, but never identical.

But it's a different issue entirely. Are we talking about writing signal processing code inside the tape recorder, where it might have to know about magnetic particles? Inside a record player where it might have to understand wiggles in grooves? This is the DSP and PluginDevelopment forum, the topic of the thread Oversampling LPF question. Here it matters that digital audio is an impulse train.

No?

PS—Your comment helped me, in thinking about what peoples' reservations might be, thanks. *

[camsr]

The thing is, if you change the sampling rate, you change the band limit.

[earlevel]

Come on guys. Isn't life too short for this

Not my intent to get involved in pointless bickering. Though I don't think this is pointless.

The sample data is encoded with Pulse Code Modulation. It results in a particular kind of spectrum. Knowledge of that spectrum is important in understanding many aspects of signal process, and especially sample rate conversion. The thread topic is Oversampling LPF question, it's not like I'm forcing the subject matter.

But the replies to me question whether PCM is even a real thing, or think that it's actually a point of view. And say the point of view can be that the samples represent the amplitude of cardinal sines. But they say it's just "exactly equivalent" to the audio data.

The sampling theorem disagrees with both of those alternatives, but more importantly both of those alternatives have no frequencies beyond half the sample rate. So they are not helpful with understanding sample rate conversion.

It's all fine if you just want to apply a cookbook SRC approach, where you just follow a recipe. But apparently that wasn't sufficient for the OP, he started this thread. With knowledge of the relationships of the sidebands, choices are apparent, not rules in a procedure.

Maybe it would help to know that I've had direct discussions with the OP in that past, well before this thread started. He reads my website, where I've written articles on what I'm talking about here.

[Tone2 Synthesizers]

your website is really great. lots of useful info there. thanks for this!

[earlevel]

Thanks for saying!

(I need to be more active, have written articles i don't put up because it's already unwieldy to navigate, need to put up an index page that makes it easier to navigate to content of interest. Again, appreciate the encouragement.)

[mystran]

But they say it's just "exactly equivalent" to the audio data.

This is really the only point that I somewhat care about.

By Nyquist-Shannon band-limited continuous-time signal and it's discrete-time sampled represented can be converted back and forth at will, exactly (mathematically at least). Because they can be converted back and forth exactly, we cannot lose information in either direction, otherwise the conversion would not be exact. So from the information theoretical point of view, they are simply different representations of the same information, just like a signal and it's Fourier transform.. or a plain-text file and an entropy-coded version of the same. Nothing is gained, nothing is lost, hence equivalent.

As for replacing every sample with a sinc, that's a practically useful idea that leads to a practically useful resampling algorithm. If we treat the samples as impulses and for every time-instant take an inner-product with a sinc, we're doing the filtering as "gather." If we replace each impulse by a sinc and take a sum, then we're doing the same filter as a "scatter" which is simply a transpose of the gather-mode filter. So this idea that you keep rejecting is basically the same as the difference between a direct-form and transposed direct-form filter.. plus a somewhat philosophical argument about whether or not we should consider the ideal reconstruction kernel (or equivalently the band-limit) as a property of the signal or not.

I said practical, so what's the deal with this transposition stuff? Well.. if we're resampling from one rate to another (without assumption of any nice ratios between the two rates), then given a fixed sinc kernel, the gather-mode filter measures the bandlimit at the input rate and the scatter-mode filter measures it at the output rate. So with a single sinc-kernel, you can upsample with gather and downsample with scatter and (assuming our kernel is perfect), there will never be aliasing... and even though sinc reconstruction is no longer theoretically valid if we vary one of the sampling rates, if such variation is "slow" then we can practically still use sinc and because we can reference our sinc to either input or output rate at will, we don't need to keep redesigning any filters on the fly.

[earlevel]

But they say it's just "exactly equivalent" to the audio data.

This is really the only point that I somewhat care about.

By Nyquist-Shannon band-limited continuous-time signal and it's discrete-time sampled represented can be converted back and forth at will, exactly (mathematically at least). Because they can be converted back and forth exactly, we cannot lose information in either direction, otherwise the conversion would not be exact. So from the information theoretical point of view, they are simply different representations of the same information, just like a signal and it's Fourier transform.. or a plain-text file and an entropy-coded version of the same. Nothing is gained, nothing is lost, hence equivalent.

It's tough talking about this now, because you started out with "exactly equivalent", twice in the same post including italicized for emphasis. When I pointed out that was not possible, you changed to it being more of a philosophical view (don't recall the exact wording). I'm not trying to be hard nosed, I just don't have a feel for whether you're being loose with "equivalence".

But this is not about information theory—we know full well what information can be recovered from the digital signal. But information theory doesn't say we have to ignore what the processes have done to the original. Knowing that we can't substitute the encrypted text for plain text is not in conflict with knowing both harbor the same information.

If you take a sine wave audio signal and convert it to PCM data at a rate of Fc, which we'll say is ten times the frequency of the original sine wave, the digital signal will have sine wave components (strictly looking at positive frequencies, as negative are trivial for real signals) 0.1Fc, 0.9Fc, 1.1Fc, 1.9 Fc, 2.1Fc... These don't add information, but they put the signal in a form that lets in be kept as discrete data. And it useful to know and discuss the effects. Remember, I didn't come in here and tell you what you were saying is wrong, you commented to what I said, saying it's wrong or not very useful. I'm not saying you must take this into account and tell other people about it. I'm asking why won't you let me tell other people about it?

(But that's why I rarely visit kvr any more: Person seeking knowledge asks a question; various knowledgeable people reply; the rest of the thread is knowledgeable people bickering about not liking the way another knowledgeable person answered.)

As for replacing every sample with a sinc, that's a practically useful idea that leads to a practically useful resampling algorithm. If we treat the samples as impulses and for every time-instant take an inner-product with a sinc, we're doing the filtering as "gather." If we replace each impulse by a sinc and take a sum, then we're doing the same filter as a "scatter" which is simply a transpose of the gather-mode filter. So this idea that you keep rejecting is basically the same as the difference between a direct-form and transposed direct-form filter.. plus a somewhat philosophical argument about whether or not we should consider the ideal reconstruction kernel (or equivalently the band-limit) as a property of the signal or not.

Sinc is the impulse response of a lowpass filter, so convolving with it is lowpass filtering. What would you be filter?... Oh, right—the modulation products (sidebands) of the pulse modulation. My approach was to tell the person what frequencies were there and why, so they can make intelligent choices about the lowpass filters he implements. Without needing to look up and understand a cookbook recipe. (To be clear, I have no problem with you giving him the cookbook recipe.)

I said practical, so what's the deal with this transposition stuff? Well.. if we're resampling from one rate to another (without assumption of any nice ratios between the two rates), then given a fixed sinc kernel, the gather-mode filter measures the bandlimit at the input rate and the scatter-mode filter measures it at the output rate. So with a single sinc-kernel, you can upsample with gather and downsample with scatter and (assuming our kernel is perfect), there will never be aliasing... and even though sinc reconstruction is no longer theoretically valid if we vary one of the sampling rates, if such variation is "slow" then we can practically still use sinc and because we can reference our sinc to either input or output rate at will, we don't need to keep redesigning any filters on the fly.

Well, again we're talking about filter implementation. And you're following a recipe for sample rate conversion. That's fine. But you're telling me to not tell another guy how it really works, essentially. I showed him why he needs filters, I didn't tell him what filter to use and what parameters he needs.

[earlevel]

BTW, in case anyone lost sight of the OP's question that I replied to, starting all of this:

I have a conceptual question for the SRC experts here.

Upsampling by a high amount, say 64x, typically requires inserting 63 zero samples after every source sample and then running that through a large, steep LPF (usually FIR). That LPF processing basically interpolates the values for the 63 new sample points.

Now, given that you only have one actual sample point every 64 samples followed by 63 zeros, why wouldn't you get some droop (ripple) between those samples in the resultant LPF-filtered signal?

So, in other words, is ripple a side effect in oversampling?

(I understand 64x is an extreme example and is not typically done in one step - but the question is if ripple modulation is a consideration/factor in upsampling)

To the OP's question, you replied with a very reasonable response. Z1202 didn't address the question, but instead faulted details of your response. I responded to the OP's question, addressing neither of your posts. Both you and Z1202 commenced faulting details of my response. (Meanwhile, the OP thanked me for my detailed answer.) That's what's so typical of the way this forum operates. The original question and resulting help is overshadowed by people finding fault with other helpers' answers. It might feel different if I'd said something that was factually incorrect, but it's hard to come away from these threads and still feel clean.

I'll repeat that I admire you guys and your what you've. It still doesn't feel any better to take x amount of time to help someone, only to spend 8x the time mud wrestling about it.

(Sure someone will say I can bail out. But then I've put my knowledge up here semi-permanently only to accompanied by semi-permanent reminders from credible authorities that what I say can't be trusted.)

[mystran]

It's tough talking about this now, because you started out with "exactly equivalent", twice in the same post including italicized for emphasis. When I pointed out that was not possible, you changed to it being more of a philosophical view (don't recall the exact wording). I'm not trying to be hard nosed, I just don't have a feel for whether you're being loose with "equivalence".

I don't know if you want to call it loose, but I very much mean "equivalent" in the sense of information theory: no information is lost.

But this is not about information theory—we know full well what information can be recovered from the digital signal.

This is absolutely about information theory. Take (uniform) sampled values, treat them as scaling factors for Dirac deltas and band-limit (filter) in continuous time. This is reversible process (we can sample at the same time-instants again to obtain the original samples, no information is lost. Take the sampled signal and replace each with a continuous sinc, add those together and we end up with the same continuous-time signal again, so it's still reversible and no information is lost.

So our disagreement here is whether the "scaled dirac delta" is the "true nature" of a sampled signal and I'd argue that it's a matter of interpretation, because the samples are just values and you can also end up with the same results choosing "scaled sincs" for example. As long as we are only concerned with the Nyquist-bandwidth, it's all the same.

Note that if we argue about "diracs" vs. "sincs" then in both cases we have chosen a continuous-time extensions for our sampled discrete-time values. What I'm trying to say is that in some situations it's useful to use one and in other situations it's useful to use another, rather than insisting that one is more "true" than the other. Neither diracs nor sincs exist in the real world, it's all theoretical either way.

Well, again we're talking about filter implementation. And you're following a recipe for sample rate conversion. That's fine. But you're telling me to not tell another guy how it really works, essentially. I showed him why he needs filters, I didn't tell him what filter to use and what parameters he needs.

I'm about 100% confident that the "other guy" has not cared about this conversation for a long time.

[earlevel]

It's tough talking about this now, because you started out with "exactly equivalent", twice in the same post including italicized for emphasis. When I pointed out that was not possible, you changed to it being more of a philosophical view (don't recall the exact wording). I'm not trying to be hard nosed, I just don't have a feel for whether you're being loose with "equivalence".

I don't know if you want to call it loose, but I very much mean "equivalent" in the sense of information theory: no information is lost.

You keep saying this, yet not a person—certainly not me—has made any claim of loss of information. Information recover is virtually perfect. There is no point even discussing this, you seem to be just trying to justify your earlier claim that samples and the bandlimited audio are "exactly equivalent". "Exactly equivalent" is not synonymous with "contains", even in information theory.

This is all random, a distraction. I said the sample train contains frequency components in addition to the bandlimit audio frequencies. They do—it's freaking PCM, the audio signal is modulated. If there are no sidebands in the discrete-time domain, then we wouldn't have the issues of aliasing with non-linear processing and sample rate conversion. You didn't like my explanation to the OP of why they exist, you prefer them to be explained away as it they aren't really there. Even though they are real, and crucial to how I was trying to help the OP. I'm not going to get sucked further into your rationalization of why we can pretend those PCM sidebands don't exist. Let them not exist for your explanations, not mine.

But this is not about information theory—we know full well what information can be recovered from the digital signal.

This is absolutely about information theory. Take (uniform) sampled values, treat them as scaling factors for Dirac deltas and band-limit (filter) in continuous time. This is reversible process (we can sample at the same time-instants again to obtain the original samples, no information is lost. Take the sampled signal and replace each with a continuous sinc, add those together and we end up with the same continuous-time signal again, so it's still reversible and no information is lost.

So our disagreement here is whether the "scaled dirac delta" is the "true nature" of a sampled signal and I'd argue that it's a matter of interpretation, because the samples are just values and you can also end up with the same results choosing "scaled sincs" for example. As long as we are only concerned with the Nyquist-bandwidth, it's all the same.

Note that if we argue about "diracs" vs. "sincs" then in both cases we have chosen a continuous-time extensions for our sampled discrete-time values. What I'm trying to say is that in some situations it's useful to use one and in other situations it's useful to use another, rather than insisting that one is more "true" than the other. Neither diracs nor sincs exist in the real world, it's all theoretical either way.

This is more sidetrack stuff. It's not me who brought up diracs and sincs, I have no interest in responding on them, different topic. (I did bring up impulse trains, which are ideal in the digital sampled realm, but it's not an important point of what I posted. If you ant to take that up, start a thread and I'll join it.) I don't really understand your "samples are just samples" point. The mathematical ("exactly equivalent"—in the actual sense) model is PCM. You can't just take samples of and not have it result in the sidebands. That's the whole point of the sampling theorem.

Well, again we're talking about filter implementation. And you're following a recipe for sample rate conversion. That's fine. But you're telling me to not tell another guy how it really works, essentially. I showed him why he needs filters, I didn't tell him what filter to use and what parameters he needs.

I'm about 100% confident that the "other guy" has not cared about this conversation for a long time.

Well, you make my point, which I elaborated on the follow post. I responded to the OP within 38 hours of his post, in detail. He subsequently thanked me for the excellent post. Much of the rest of this thread has been other helpers faulting my comments, and how I explain things. For instance, one of your initial complaints was that my description (supposedly) wasn't best for generalizing to non-uniform sampling. Something that isn't even associated with the question and has no part in this discussion.

[mystran]

"Exactly equivalent" is not synonymous with "contains", even in information theory.

And THIS is where we disagree.

ps. That's fine though, because disagreement is good. The thing is, I actually like to argue these things mostly because I think it helps me formulate my thoughts in more exact terms. Clearly I have failed to describe my thoughts well enough which means that I still need to find a better way to formulate those arguments, but I don't think I can do it now, we can revisit the topic in a couple of years.