Sampling theory—"best" explanation

[earlevel]

Please check out my new series on sampling theory, and let me know what you think. The goal was to be brief, but thorough, and avoid abstract mathematical explanations.

http://www.earlevel.com/main/tag/sampli ... ?order=asc

About the series title: I’m not trying to be presumptuous with, “the best explanation you’ve ever heard”, but I think it’s unique in that it separates sampling origins from the digital aspects, making the mathematical basis more obvious. Over the years, I’ve seen everything from “each samples represents the amplitude of a sinc function” to “we don’t know what’s between the samples”. I thought about this for a long time in order to come up with this explanation.

As noted in the prologue, this as a test run at a script for a video presentation, which will have more detailed and animated support graphics and examples. So, the “best explanation” part is about the approach, not this embodiment. (In that sense, is it like Tenacious D’s “Tribute”—not the best song in the world, but a song about the best song in the world?)

I will conclude the series with another article of observations and other comments at a later date.

[camsr]

Digital samples are ideal impulses, while what comes out of a DAC is a lowpass filtered zero-order voltage hold?

[earlevel]

Digital samples are ideal impulses, while what comes out of a DAC is a lowpass filtered zero-order voltage hold?

Sort of. I mean, it depends on the hardware. Ideally, it's lowpass filtered ideal impulses. Or at least lowpass-filtered "good enough" impulses (has ideal frequency content in the passband, maybe not too good past that, but we don't care). But zero-order (steps) are easier, though cause droop in the passband—which can be fixed by corrective filtering. Modern DACs employ other techniques (fewer bits at a higher rate, including one-bit in a delta-sigma loop). But unless you're designing DAC hardware, you can pretend it's ideal, because they all do the best they can to emulate ideal impulses into a perfect lowpass filter.

[BertKoor]

I think what camsr really meant is, that you're required to have a shitload of dsp knowledge to understand that single sentence. So it will go straight over the head of your audience.

Nb: I haven't read your page yet.

Edit: after a skim over the page he's right. Sampling is dsp chapter one. Trying to explain that with concepts from chapter eleven won't land.

[PurpleSunray]

hm I kind of agree with camsr and BertKoor.
You start right off with DSP terms. If someone knows what impulses are or bandlimited is, he knows about DSP already.
Assume zero DSP knowlege if you want to address i.e. musicans that want to understand how their plugins works.
I would probably start with showing / explaining a simple sine wave. Then show what happens if you summ-up different sinewaves (a new waveform). Explain that you can create a rect-wave by stacking sine waves infinitly. You need to bring audience to a point where they understand what bandlimited actually means. Than explain what impulses are, ect. pp.
Don't assume any kind of prior DSP knowlege if it shall be an explanation for non-technical ppl.

[mystran]

Actual practical modern DAC is complex enough beast that I personally think talking about ZOH at all is not really helpful unless you are planning on writing a series on DAC design.

The theory behind an "old school" ZOH-DAC is that you can convolve the impulse train with a box-filter to get the ZOH waveform, then band-limit the result and finally deconvolve the box response... but if you want to avoid abstract mathematics I'm not sure if you want to go there.

[camsr]

It makes sense to me that samples are actually impulses and not continuous waveforms. It validates everything regarding resampling and why we can hear what is in-between samples (a continuous waveform). An integrator of some sort is a required and very important part of sampling. The integrator is what fills in the blanks.

The article being a beginner's intro, it does lack specifics. It's decent so far, having some DSP knowledge rattling away upstairs helps and so does calculus. So far it's skim, and some more important details are missing, like the discussion of ADC/DAC operation, but the theory is there. That would help to make it more understandable.

[earlevel]

Actual practical modern DAC is complex enough beast that I personally think talking about ZOH at all is not really helpful unless you are planning on writing a series on DAC design.

The theory behind an "old school" ZOH-DAC is that you can convolve the impulse train with a box-filter to get the ZOH waveform, then band-limit the result and finally deconvolve the box response... but if you want to avoid abstract mathematics I'm not sure if you want to go there.

Actually, I was reluctant to answer camera's question because I didn't go about explaining the inner working of DAs. I only touched on that point because people see steps varies places, including hardware diagrams, and I wanted to say why.

[earlevel]

It makes sense to me that samples are actually impulses and not continuous waveforms. It validates everything regarding resampling and why we can hear what is in-between samples (a continuous waveform). An integrator of some sort is a required and very important part of sampling. The integrator is what fills in the blanks.

The article being a beginner's intro, it does lack specifics. It's decent so far, having some DSP knowledge rattling away upstairs helps and so does calculus. So far it's skim, and some more important details are missing, like the discussion of ADC/DAC operation, but the theory is there. That would help to make it more understandable.

Hm...I don't think it's a beginner's intro (my blog is not for beginners). I think of it more as someone who knows DSP, but has never learned that it's about impulses and therefore modulated alias images. I've run into that a lot over the years. I was trying to convey a specific think here—why why samples represent impulses, and what that implies for the frequency spectrum.

About discussing ADC/DAC details: They can get that somewhere else, it's not what I was after. And the idea was brevity, not a book. And this is a roadmap for a video—they get long fast. The video will be fleshed out with animated explanations and examples, but I like to keep them short. For example, my video on dither:

https://youtu.be/zWpWIQw7HWU

Thanks for the feedback—my reply might sound defensive, but just stating what my target was, what I was after, and maybe it will help the feedback.

[Smashed Transistors]

Please, no more "steppy" representation for audio DAC outputs.

This is deeply misleading especially to "beginners". It does not represent how modern DAC work and what internal digital values mean.
It's awful to have to discuss with a "beginner" convinced that "analog is the daddy", because he thinks that the "steps" are the way audio DACs work.

The Dirac "pulses" with spots at the top are not much better.

The impulses we are talking about are band limited impulses. Smooth impulses. Ideally sinc impulses.

[earlevel]

If someone knows what impulses are or bandlimited is, he knows about DSP already.

This sentence worth discussing.

In my experience, lots of people who know DSP do not know know that sample represent impulses, and do not understand how the frequency spectrum is aliased. (I've had related arguments online for decades.) No?

Or, if they do, I think the vast majority don't understand why. I never learned why from any of the classic DSP text books on my shelf (I had to think about it). They explain the characteristics of digital samples, but not how we got there.

No?

I think that separating digital from sampling makes it easier to grok—but I understand it will be most beneficial to electrical engineers and long-time modular synthesists (amateur radio guys too), like myself, who have an intuitive grasp of amplitude modulation. To that end, my preference is to have an AM lab widget on the webpage, but I don't have that kind of time right now. The video will have animation and more detail on this point.

[earlevel]

The impulses we are talking about are band limited impulses. Smooth impulses. Ideally sinc impulses.

Only after processing through the lowpass filter.

[Smashed Transistors]

No @earlevel,

Steppy DAC + Low pass filters are audio DACs of the 70s.

the "modern" (since the 80s) audio DACs include multiple steps including intricate PWM, filtering and dithering. Compared to a classic analog low pass filter it allows to preserve phase.

[earlevel]

No @earlevel,

Steppy DAC + Low pass filters are audio DACs of the 70s.

the "modern" (since the 80s) audio DACs include multiple steps including intricate PWM, filtering and dithering. Compared to a classic analog low pass filter it allows to preserve phase.

I don't think you read my article. I think you are arguing what you think I might have said.

But back to your comment that impulses are smooth, ideally sinc. That can't be argued anywhere until after the lowpass filtering of the DAC, where you could say the analog wave is the summation thereof. The samples in the digital domain do not represent sinc.

[earlevel]

OK, forgot how sensitive people are to even mentioning steps in audio (I connected the dots for visibility in my dither video, and got blow-back from it). It was an aside in my article, anticipating confusion from people who’ve looking into it deep enough to see stairstep diagrams for DACs. Maybe I’ll pull the aside, maybe not.

I’m an electrical engineer, designed my own circuits with DACs, etc., over decades—I assure you I know all about how sigma-delta and other schemes work, so please stop telling me that modern DACs don’t work this way. Instead do a web search on how DACs work. Click the images tab. Diagrams with stair steps. Modern DACs? Here’s the first article I pulled up on one, the article has diagrams with stairsteps. This is why I thought I needed to touch on why samples don’t represent steps, even if you read that DACs use steps.