Oversampling LPF question

[mystran]

For most intuitive reasoning, the continuous time signal implied by the samples is probably the best, as this is what you're gonna hear at the DAC output (rather than any kind of a lollypop sequence, which actually has infinite power and cannot exist in reality).

I would like to point out (again) that the nice thing about the view point that treats the samples as values of a (unique) continuous band-limited signal is that it also works on non-uniform sampling grids (where you can still recover "perfectly" with Lagrange interpolation with a few "minor" gotchas, where as cardinal sine is only the limit for uniform grids).

[earlevel]

Steps are generally misleading. Lollypops are okay, if you want to think in terms of abstract repeating spectra. For most intuitive reasoning, the continuous time signal implied by the samples is probably the best, as this is what you're gonna hear at the DAC output (rather than any kind of a lollypop sequence, which actually has infinite power and cannot exist in reality). E.g. taking any wave editor with a decent interpolation display you can compare your original signal with the upsampled signal and see that there are no new ripples (whereas you may be surprised to see the ripples in your original signal already, if you're used to steps). With lollypops you'll maybe see the ripples only in the upsampled signal, but that doesn't mean that they were not there before, this is one of the misleading aspects of lollypops.

I guess the reason why most theoretical books will do more or less exclusively with lollypop idea, is because they are... theoretical (rather than intuitively practical), plus they might be focusing on general DSP rather than music DSP, or simply follow the common practice from general DSP. Steps are more intuitive but very misleading, frankly speaking I have yet to think of a theoretical DSP book which advocates the idea of the steps, though. The proper continuous time signal is intuitive and not misleading. YMMV

The reason that theoretical books show impulses is because that's the mathematical fact. While looking at samples and thinking in terms of the output of the DAC is abstract. (I'm not saying it can't be helpful.)

For instance, I've had people who've been so accustomed to this thinking and apparently learned that way, that they say that each sample is a sinc. Then they need to hedge and admit a windowed sinc because otherwise it's just as impossible as the impulse they are arguing against. Then, what is the window...and what is the resulting latency...(but not as bad as one guy who argued with my explanation of integer upsampling, who said we can't know what's between samples. When I asked why, then did zero insertion work, he replied, "serendipity" )

While the DAC output view works fine for "keeping your eye on the ball", it doesn't work as well for non-linear stuff, and sample rate conversion. Anyway, my point is that the books show it this way for the same reason they are precise with math. It explains everything.

Also, I have to add about "infinite power and cannot exist in reality". Well, we can't reproduce audio in reality either, we just move a speaker as a proxy for the sound source driven by a close-enough analog of the signal. And as far as impulses, we produce incredibly precise one all the time in electronics, and audio is way on the low end of the frequency spectrum. We it's not at all hard to get the quality we're after, we don't need infinite power and perfect impulses.

However, the samples in digital memory are theoretically perfect, or at least stand-ins for perfect, as close as we can measure and fit into the available bits. They must be perfect, since we discarded everything between them in order to fit in memory, and only do the math with those points. Unlike sampled analog, where we can have non-instantaneous pulses and the analog signal processing (simple gain, whatever) would apply to it all.

Not arguing here, just presenting some thoughts...

[mystran]

The reason that theoretical books show impulses is because that's the mathematical fact. While looking at samples and thinking in terms of the output of the DAC is abstract. (I'm not saying it can't be helpful.)

For instance, I've had people who've been so accustomed to this thinking and apparently learned that way, that they say that each sample is a sinc. Then they need to hedge and admit a windowed sinc because otherwise it's just as impossible as the impulse they are arguing against.

The samples themselves are neither impulses or cardinal sines. They are simply values of a band-limited signal at the sampling instants. If the sampled values happen to lie on a uniform grid, then one can reconstruct said band-limited signal efficiently by adding together scaled and shifted cardinal sines (where as in the non-uniform case case we need Lagrange-interpolation), but really the samples themselves are just values, nothing more, nothing less.

edit: The point I'm trying to make is that any mathematical formalism that reconstructs the underlying band-limited signal correctly is equally valid and trying to argue that the sampled values have some fundamental meaning beyond being sampled values is really just about as useful as trying to argue that chicken tastes better than fish.

[earlevel]

The samples themselves are neither impulses or cardinal sines. They are simply values of a band-limited signal at the sampling instants. If the sampled values happen to lie on a uniform grid, then one can reconstruct said band-limited signal efficiently by adding together scaled and shifted cardinal sines (where as in the non-uniform case case we need Lagrange-interpolation), but really the samples themselves are just values, nothing more, nothing less.

You say they are not impulses just values at a certain point in time. Well, of course, that's saying the obvious, they are values in memory. But they represent impulses. And it works out very well whether you consider the digital samples as an impulse train or whether you translate it into an actual analog impulse train and do analog signal processing on it from there. I'm not demanding that you buy into that, but I have a lot of DSP books on my side so when you tell me I'm wrong, it's hard for me to take you seriously.

For instance, open one of countless DSP books where they illustrate the repeating spectrum that's unavoidable in any real signal. Do the math to demodulate that spectrum with the spectrum of the audio it represents (that is, the part of the spectrum below half the sample rate). What do you get? A unit impulse train.

Or go the other way. Convolve the spectrum of an impulse train with the spectrum of your signal—what do you get? Endless repeating mirrored copies of the signal spectrum. Or, do it in the time domain by multiplying, and check the results against AM theory—you get the same thing.

Tip for the benefit of others, I'm sure you know this: The Fourier transform of an ideal impulse train of frequency Fs gives cosines at integer multiples of the fundamental, so 0Fs, 1Fs, 2Fs, to infinity...all at the the same amplitude. So, for 48 kHz, that's DC, 48k, 96k...AM yields plus and minus frequencies against each of the signal components in the frequency domain. So, DC duplicates the signal, and the others the mirrored images.

I've given my point of view, you've given yours, I'm fine with that. But if you want to discuss more, or continue to say I'm wrong, I guess I'd ask if you think the repeating spectrum is a useful fairy tale, wrong, or ? (BTW, it's pretty easy to show they really are there in the samples...)

[mystran]

The samples themselves are neither impulses or cardinal sines. They are simply values of a band-limited signal at the sampling instants. If the sampled values happen to lie on a uniform grid, then one can reconstruct said band-limited signal efficiently by adding together scaled and shifted cardinal sines (where as in the non-uniform case case we need Lagrange-interpolation), but really the samples themselves are just values, nothing more, nothing less.

You say they are not impulses just values at a certain point in time. Well, of course, that's saying the obvious, they are values in memory. But they represent impulses. And it works out very well whether you consider the digital samples as an impulse train or whether you translate it into an actual analog impulse train and do analog signal processing on it from there. I'm not demanding that you buy into that, but I have a lot of DSP books on my side so when you tell me I'm wrong, it's hard for me to take you seriously.

I'm not trying to argue that you are "wrong" as much as just to point out that "samples representing impulses" is a formalism that relies on the implicit assumption that the sampling grid is uniform, where as the "samples as values of a band-limited signal" is strictly more general. Most traditional DSP books assuming a uniform sampling grid and this works perfectly fine for many purposes, yet there are also plenty situations where you might want to consider other grids. Thinking in terms of an underlying band-limited continuous signal that we can reconstruct from the know sampled values gives us flexibility in terms of choosing whatever formalism happens to work at any given time.

Just to clarify, I'm not against any particular formalism as such. I'm just against the notion that one should pick one particular formalism and hold on to it at all cost. Sometimes one view point is useful, sometimes another view point is useful and what I'm trying to advocate for is learning about all of them, just so you can pick whichever is the most useful for any given situation. Also at the end of the day, as long as one ends up with the correctly reconstruct signal, it doesn't really matter which formalism was used to obtain it.

[earlevel]

I'm not trying to argue that you are "wrong" as much as just to point out that "samples representing impulses" is a formalism that relies on the implicit assumption that the sampling grid is uniform, where as the "samples as values of a band-limited signal" is strictly more general. Most traditional DSP books assuming a uniform sampling grid and this works perfectly fine for many purposes, yet there are also plenty situations where you might want to consider other grids. Thinking in terms of an underlying band-limited continuous signal that we can reconstruct from the know sampled values gives us flexibility in terms of choosing whatever formalism happens to work at any given time.

Just to clarify, I'm not against any particular formalism as such. I'm just against the notion that one should pick one particular formalism and hold on to it at all cost. Sometimes one view point is useful, sometimes another view point is useful and what I'm trying to advocate for is learning about all of them, just so you can pick whichever is the most useful for any given situation. Also at the end of the day, as long as one ends up with the correctly reconstruct signal, it doesn't really matter which formalism was used to obtain it.

Point well made, mystran. Thank you for clarifying.

[mystran]

Now, that said, there IS some ripple in an anti-aliasing LPF and that ripple is what remains after the attenuation of the filter. In fact, if you use a filter with too high of a cutoff frequency or poor attenuation that "clocking" ripple will be clearly visible on a scope (and even audible as digital whine if the sample rate is low).

I want to comment on this, since I guess you are thinking in terms of the time-domain ripple, which is usually known as "ringing" (as opposed to spectral ripple)?

One could argue that it isn't really an artefact as such, but rather simply the natural feature of a signal with a sharp cut-off point in the spectrum. It can be reduced by using a wider, more gradual transition region at the (rather obvious) cost of losing some high frequencies. Like most things in DSP, it's another time vs. frequency trade-off: sharp features in the spectrum cause ringing in the time-domain, while sharp-features in the time-domain will cause ripple in the spectral domain.

[Z1202]

The reason that theoretical books show impulses is because that's the mathematical fact. While looking at samples and thinking in terms of the output of the DAC is abstract. (I'm not saying it can't be helpful.)

I'm more than a bit puzzled by your claim about a "mathematical fact". The mathematical fact is that you _can_ analyse discrete-time signals in terms of lollypops, which is particularly convenient for reasoning about comparing the spectra of continuous and discrete-time signals. However there is no mathematical fact which tells you that discrete signals _are_ lollypops. You simply cannot compare signals of fundamentally different nature (or more precisely, it's up to you to propose a way to do so, which is consistent with the other things you're doing with the signals, primarily with ADC and DAC conversions), therefore you cannot claim anything there for an objective fact. It is just the human choice to tackle the problem one way or another. The way how DACs and ADCs work is a fact, because those are real devices. How we theoretically analyse them is our own choice, which is okay, as long as it's consistent with DACs and ADCs. The only facts there are whether our choice is consistent or not. (And, to my best knowlegde, DACs and ADCs do not _physically_ work in terms of lollypops, again, this is just a theoretical construct).

When we talk about aliasing, e.g. in a nonlinearity, we are actually comparing how a discrete-time nonlinearity work to how it would work in continuous time (which is our desired scenario), thus lollypop paradigm is _useful_ again, but it's not a fact, it's just a useful theoretical aid which we choose to employ to compare the spectra in contunuous and discrete time cases. However, other than in such cases (e.g. in the original question about ripples), the interpolated signal interpretation is more insightful, IMHO. Lollypops cause other kinds of confusion, like the original question. YMMV

Edit: I'd like to elaborate once more on the example of ripples. If you think of ripples created by upsampling in terms of lollypops it gets quite confusing. There "are no ripples" in the original signal, but there are ripples in the upsampled signal. But if you put both siginals through a DAC, both signals are 100% identical (if the upsampler and DAC are ideal). So, where do the ripples come from, if both signals are essentially identical? OTOH, in terms of interpolated signal, the ripples _are_ there from the very beginning, it's just that the "lollypop resolution" wasn't enough to see them. Easy and intuitive.

[Z1202]

I'm not demanding that you buy into that, but I have a lot of DSP books on my side so when you tell me I'm wrong, it's hard for me to take you seriously.

This might be the source of misunderstanding. I don't think anybody telling you that you're wrong or the books are wrong. At least what I'm trying to tell, is that you're overabsolutizing the book texts. What books tell you is how things _can_ be understood, and this understanding is consistent with what we're doing, and that _is_ a fact. And there are good reasons to understand things in the lollypop way, especially if the books are not specifically targetting music DSP. What I'm arguing against is a claim that lollypops are _the_ way to understand things (they certainly are _a_ way), especially in music DSP context. Or claiming that discrete signals _are_ lollypops.

[earlevel]

I'm not demanding that you buy into that, but I have a lot of DSP books on my side so when you tell me I'm wrong, it's hard for me to take you seriously.

This might be the source of misunderstanding. I don't think anybody telling you that you're wrong or the books are wrong. At least what I'm trying to tell, is that you're overabsolutizing the book texts. What books tell you is how things _can_ be understood, and this understanding is consistent with what we're doing, and that _is_ a fact. And there are good reasons to understand things in the lollypop way, especially if the books are not specifically targetting music DSP. What I'm arguing against is a claim that lollypops are _the_ way to understand things (they certainly are _a_ way), especially in music DSP context. Or claiming that discrete signals _are_ lollypops.

OK, we're misinterpreting each other's intentions, I suppose. I'm not absolutizing the text, a comment was made where it was questioned why the classic texts would take that approach, as if it were arbitrary. I was countering that, saying it made sense. Most importantly, I said their depiction of the spectrum (repeated images) is precisely what the theory suggests. BTW, I claim nothing about lollipops—that's just a drawing device to help the eye—I've consistently said impulses.

Also, you gave your point of view that ADCs and DACs were fact, and the theory describes them. My point of view, as an electrical engineer, is the opposite. (I worked as an engineer supporting high speed monolithic DACs and ADCs 40 years ago, btw, this is not a new topic to me.) That is, the theory describes what we get, and the converters are designed to get us there is some convenient and inexpensive manner. If a DAC uses s&h then corrects for the resulting aperture error, it's because that's a cheap way to arrive at an equivalent place that the theory says. So I wouldn't call such a DAC "the fact", personally.

But I think you misunderstood what I was calling a fact. I was referring to people who insist that samples are the embodiment of sinc. I now understand that's not exactly what mystran was saying, but I've certainly had enough people say it to me to know there are plenty of people who insist that. (I recently had a discussion with someone who was clearing mistaking Whittaker-Shannon—interpolation/DAC modeling—with the sampling theorem.) Read my comment about fact again—I contrasted that with "looking at samples and thinking in terms of the output of the DAC [e.g., sinc functions] is abstract." If given the choice between sincs and impulses, I'd say impulses are the fact—fully explaining the spectrum of a sample signal—and sinc interpolation is an abstraction, looking ahead to the DAC output. I guess I could have said truth instead of fact, as I think you view fact as something tangible (nothing wrong with that).

Anyway, we're clearly making suppositions about each others' comments and terminology at this point. I currently have no disagreement with anyone.

[Z1202]

But I think you misunderstood what I was calling a fact. I was referring to people who insist that samples are the embodiment of sinc. I now understand that's not exactly what mystran was saying, but I've certainly had enough people say it to me to know there are plenty of people who insist that. (I recently had a discussion with someone who was clearing mistaking Whittaker-Shannon—interpolation/DAC modeling—with the sampling theorem.) Read my comment about fact again—I contrasted that with "looking at samples and thinking in terms of the output of the DAC [e.g., sinc functions] is abstract." If given the choice between sincs and impulses, I'd say impulses are the fact—fully explaining the spectrum of a sample signal—and sinc interpolation is an abstraction, looking ahead to the DAC output. I guess I could have said truth instead of fact, as I think you view fact as something tangible (nothing wrong with that).

I'd say neither is a fact. They are different interpretations of what samples are. One interpretation is commonly accepted in theoretical textbooks but that doesn't make it a fact. Not that I say that this interpretation is wrong, but calling it a fact is a bit too far. Anyway, I'm not sure this discussion is leading anywhere further...

PS. One more comment on "fully explaining the spectrum of a sample signal": that's like a dog biting its tail. The spectrum of the sample signal is being interpreted in terms of "lollypops", which particularly results in periodic spectra. But that's just a convention. We could have said the "samples are sincs" (this is not exactly what I'm advocating but close enough) and the spectrum simply extends from -Fs/2 to +Fs/2 - that would be another convention, which might be too limiting in the context of generic DSP but works well in music DSP context. Neither is a truth or a fact, they are just conventions.

[mystran]

I would propose we settle this discussion by concluding that z=exp(s*T) by definition, which (as far as I can tell) should then be sufficient to derive just about anything else you might want to know about uniform sampling (eg. you can get the impulses by taking the inverse Laplace transform, or you can do interpolation by multiplying with a suitable linear phase-shift ... and so on).

ps. And just to clarify: no, I'm not being entirely serious here.

[mystran]

We could have said the "samples are sincs" (this is not exactly what I'm advocating but close enough) and the spectrum simply extends from -Fs/2 to +Fs/2 - that would be another convention, which might be too limiting in the context of generic DSP but works well in music DSP context.

<pedantic>
You would need complex sampling if you want to differentiate between positive and negative frequencies; with cardinal sines you only get half of it.

Additionally you don't really have to sample in the base band; any other range of frequencies works just as well as long as the total bandwidth doesn't exceed half the sampling-rate (for real-valued sampling) and bandpass sampling is another case where the "samples are sincs" viewpoint just doesn't work directly (rather you need at least some additional hand-waving about frequency-shifting back and forth if you want to work with cardinal sines).
</pedantic>

[Z1202]

You would need complex sampling if you want to differentiate between positive and negative frequencies; with cardinal sines you only get half of it.

Hmmm, I'm not sure. What if I multiply a cardinal sine with a complex value?

Additionally you don't really have to sample in the base band; any other range of frequencies works just as well as long as the total bandwidth doesn't exceed half the sampling-rate (for real-valued sampling) and bandpass sampling is another case where the "samples are sincs" viewpoint just doesn't work directly (rather you need at least some additional hand-waving about frequency-shifting back and forth if you want to work with cardinal sines).

That just confirms my point about convention How often do you need to reason about other bands in music DSP? And if you have to, there's nothing which should stop you to take a difference of two sincs (or whatever) as a kernel

And mind you, I'm not advocating dropping the lollypops altogether. I'm arguing their use as a go-to interpretation.

[earlevel]

But I think you misunderstood what I was calling a fact. I was referring to people who insist that samples are the embodiment of sinc. I now understand that's not exactly what mystran was saying, but I've certainly had enough people say it to me to know there are plenty of people who insist that. (I recently had a discussion with someone who was clearing mistaking Whittaker-Shannon—interpolation/DAC modeling—with the sampling theorem.) Read my comment about fact again—I contrasted that with "looking at samples and thinking in terms of the output of the DAC [e.g., sinc functions] is abstract." If given the choice between sincs and impulses, I'd say impulses are the fact—fully explaining the spectrum of a sample signal—and sinc interpolation is an abstraction, looking ahead to the DAC output. I guess I could have said truth instead of fact, as I think you view fact as something tangible (nothing wrong with that).

I'd say neither is a fact. They are different interpretations of what samples are. One interpretation is commonly accepted in theoretical textbooks but that doesn't make it a fact. Not that I say that this interpretation is wrong, but calling it a fact is a bit too far. Anyway, I'm not sure this discussion is leading anywhere further...

PS. One more comment on "fully explaining the spectrum of a sample signal": that's like a dog biting its tail. The spectrum of the sample signal is being interpreted in terms of "lollypops", which particularly results in periodic spectra. But that's just a convention. We could have said the "samples are sincs" (this is not exactly what I'm advocating but close enough) and the spectrum simply extends from -Fs/2 to +Fs/2 - that would be another convention, which might be too limiting in the context of generic DSP but works well in music DSP context. Neither is a truth or a fact, they are just conventions.

Do we really need “lollipops” and “dog biting its tail” as descriptions of what I’m advocating, and mathematical terms for yours?

I don’t think the sinc explanation is sufficient. It doesn’t explain aliasing well, and especially doesn’t explain zero insertion for upsampling well (why do we need to filter again if we’ve only added zero-scaled sincs?). It doesn’t even explain, in itself, the need to bandlimit the input. With impulses, all of those things are implicit, obvious consequences of the AM involved.

Sampling works in the analog domain too. It’s easy to build an analog impulse-based system that works and has the same behavior that I describe. Perfection will be limited by the perfection of the impulse, of course. But if you build such system based on sincs, you essentially need to do the impulse-equivalent sampling then impart the sincs on it—effectively modeling a DAC. I don’t see how that is an equivalent model to what we store in memory. (And it seems you need to make arbitrary choices about the sincs—we can’t have an infinitely long one, so we need to consider a window, and therefore lower the frequency a suitable amount to satisfy the Nyquist criterion.)

I don’t want to give the impression I’m digging in, and I think at this point I've said all I care to. I’m just addressing your comments, and since you have a problem with the impulse view as “a dog biting its tail”, I added some reasons why the sinc view of sampling has some shortcomings for me as well. If the impulse view is self-serving, and least it's fully served.

PS—I'm reminded that another term for "impulse sampling" is "ideal sampling"—at least I have that on my side