## Sampling theory—"best" explanation

72 posts
• Page

**2**of**5**• 1,**2**, 3, 4, 5- KVRist
- 441 posts since 4 Apr, 2010

FWIW, I reread the part in my article about DACs and steps, and I see why the comments about modern DACs. At one time I explained it with a disclaimer that that's how a canonical DAC works, details different in modern DACs but effectively implementing the same thing. It got more wordy than I wanted (if you mention that modern DACs don't work like that, you have to say why, and how they are the same, otherwise you've just explained something then said it really doesn't work that way!), so chopped it down to a canonical DAC explanation. Maybe I'll reword it, or otherwise make it clear I'm discussing the basic flow and how the S&H steps relate. Or might just add a word like "fundamentally".

Edit: Ah...lol...that part of the article is talking about the ADC anyway, not the DAC. Where are people seeing me describing DACs in the article?

Edit: Ah...lol...that part of the article is talking about the ADC anyway, not the DAC. Where are people seeing me describing DACs in the article?

Last edited by earlevel on Sat Aug 26, 2017 1:26 pm, edited 1 time in total.

My audio DSP blog: earlevel.com

- KVRAF
- 2230 posts since 17 Apr, 2005, from S.E. TN

I'll read the articles. Nigel writes good.

Sure its impulses but the zero order hold type works and so long as you gloss over the hidden complexity, the math explaining how IC chips operate-- Designing/building a zero order hold dac is real simple requiring very little math.

Just some kind of gated source of parallel bits (to avoid settling glitches when the word changes), some kind of resistor array, buffer, lowpass filter. The territory of old Radio Shack Forrest Mims high school level circuit books.

I don't recall ever building something with an actual DAC chip but would roll my own DACs out of matched resistors when needed. Was too cheap to buy dac chips back when they were expensive and too lazy to figure out how to use em. Maybe some day but probably not.

Just reinforcing Nigel's assertion that some people might be familiar with the practice and use of simple dacs without knowing any math except how to use ohms law, kirchoffs law and timing diagrams to build the voltage output equivalent of an etch-a-sketch toy.

Kinda like some fellas somehow manage to build rather high performance racing engines without knowing much math or physics. The tinker gene.

Sure its impulses but the zero order hold type works and so long as you gloss over the hidden complexity, the math explaining how IC chips operate-- Designing/building a zero order hold dac is real simple requiring very little math.

Just some kind of gated source of parallel bits (to avoid settling glitches when the word changes), some kind of resistor array, buffer, lowpass filter. The territory of old Radio Shack Forrest Mims high school level circuit books.

I don't recall ever building something with an actual DAC chip but would roll my own DACs out of matched resistors when needed. Was too cheap to buy dac chips back when they were expensive and too lazy to figure out how to use em. Maybe some day but probably not.

Just reinforcing Nigel's assertion that some people might be familiar with the practice and use of simple dacs without knowing any math except how to use ohms law, kirchoffs law and timing diagrams to build the voltage output equivalent of an etch-a-sketch toy.

Kinda like some fellas somehow manage to build rather high performance racing engines without knowing much math or physics. The tinker gene.

- KVRist
- 441 posts since 4 Apr, 2010

Thanks, Jim, I'm always interested to hear your take.

I've mostly reviewed ADCs/DACs to specify or review what's specified, in modern times. I't been a long time since I actually etched a circuit board and soldered a DAC (DIP days). I put a 12-bit DAC in a ceramic package on a board, multiplexed to drive my analog synth components (pre-MIDI, mono-voiced days).

A couple of years later, I was working in the VLSI chip biz. One of the chips, a real cash cow, was an 8-bit video-speed ADC (a "flash" converter—essentially a string of 256 resistors and comparators). $800 in singles, $500 in quantity. (IRRC, the Bell Labs Hal Alles synth used their 16-bit multipliers—I remember the 16-bit MAC chips dissipated 5 watts each, I think the multipliers were at least 3. I suppose you could heat your home computing an FFT in hardware—a task they did use those chips for.) Fun times.

I've mostly reviewed ADCs/DACs to specify or review what's specified, in modern times. I't been a long time since I actually etched a circuit board and soldered a DAC (DIP days). I put a 12-bit DAC in a ceramic package on a board, multiplexed to drive my analog synth components (pre-MIDI, mono-voiced days).

A couple of years later, I was working in the VLSI chip biz. One of the chips, a real cash cow, was an 8-bit video-speed ADC (a "flash" converter—essentially a string of 256 resistors and comparators). $800 in singles, $500 in quantity. (IRRC, the Bell Labs Hal Alles synth used their 16-bit multipliers—I remember the 16-bit MAC chips dissipated 5 watts each, I think the multipliers were at least 3. I suppose you could heat your home computing an FFT in hardware—a task they did use those chips for.) Fun times.

My audio DSP blog: earlevel.com

- KVRAF
- 4709 posts since 11 Feb, 2006, from Helsinki, Finland

earlevel wrote:Smashed Transistors wrote:No @earlevel,

Steppy DAC + Low pass filters are audio DACs of the 70s.

the "modern" (since the 80s) audio DACs include multiple steps including intricate PWM, filtering and dithering. Compared to a classic analog low pass filter it allows to preserve phase.

I don't think you read my article. I think you are arguing what you think I might have said.

But back to your comment that impulses are smooth, ideally sinc. That can't be argued anywhere until after the lowpass filtering of the DAC, where you could say the analog wave is the summation thereof. The samples in the digital domain do not represent sinc.

NO! This is exactly what is wrong with all these "intuitive understandings" of the sampling theory. Each digital sample is simply a value of some band-limited signal at some particular point in time. It is not an impulse, it is not a sinc, it is simply a value of some signal. From the mathematical point of view it is often convenient to treat it as an impulse or sinc or whatever, but fundamentally it's just a value (of some band-limited signal at a certain point in time).

Sampling theory then tells us that if the number of sample is sufficiently high, we can reconstruct the underlying (band-limited) signal exactly using Lagrange interpolation, which in the limit case with uniform sample placement gives us sinc-interpolation, but this doesn't change the fact that really it's just a sufficient number of values taken from a band-limited function at various points in time.

Now, the reason I'm making such a fuzz about this, is because the low-pass filtering of impulses interpretation and the sinc-impulses interpolation interpretations only work in the uniform case (which is arguably the most convenient to work with), but sampling theory as such is not limited to this special case. Samples don't stop being samples and Lagrange interpolation doesn't stop working and the band-limiting constraints still apply as long as you have enough samples on average. Things get awfully messy for sure, but the fundamental underlying theory holds.

So.. really.. it's just values of a band-limited signal at certain points in time. You can mathematically treat it in various other ways in various special cases, but it's not like there's any real impulses of any sort at any point, it's just values.

<- plugins | forum

- KVRAF
- 6775 posts since 16 Feb, 2005

Yeah the ADC captures values over time but that new signal is integrated at the DAC the create the new waveform. ZOH is like a really bad integrator! But it's also causal and error is limited the further from nyquist the signal is. I could think that the trapezoidal integrator would make a large improvement, but that also requires 2x oversampling to give the midpoint in between the original samples.

No idea how this relates to delta-sigma modulation. Not sure how it works either, but the conversion of PCM to a bitstream implies upsampling of some kind.

No idea how this relates to delta-sigma modulation. Not sure how it works either, but the conversion of PCM to a bitstream implies upsampling of some kind.

- KVRAF
- 11774 posts since 7 Dec, 2004

re mystran's post: I agree with that objection.

If there is any attempt made to "simplify the explanation" you won't be explaining anything at all. The only way to go from "infinitesimal samples" to anything else is to fundamentally change what you're explaining.

"Samples at a certain point in time" seems a redundant phrasing. "Infinitesimal" is also redundant because all that information is represented in the term "sample". For example if I sample the temperature outside my window it's currently approximately 28c; according to my thermometer.

I have sampled not the "current temperature" but instead the value of the reading provided from a specific thermometer at a specific infinitesimal point in time as I have interpreted and recorded it.

All that information is represented in the term "sample". I believe it is essential to start from the beginning by defining basic scientific principals and the meaning of the term "sample".

From that point on understanding should be intuitive and most of the resulting properties of discrete signals come naturally.

If there is any attempt made to "simplify the explanation" you won't be explaining anything at all. The only way to go from "infinitesimal samples" to anything else is to fundamentally change what you're explaining.

"Samples at a certain point in time" seems a redundant phrasing. "Infinitesimal" is also redundant because all that information is represented in the term "sample". For example if I sample the temperature outside my window it's currently approximately 28c; according to my thermometer.

I have sampled not the "current temperature" but instead the value of the reading provided from a specific thermometer at a specific infinitesimal point in time as I have interpreted and recorded it.

All that information is represented in the term "sample". I believe it is essential to start from the beginning by defining basic scientific principals and the meaning of the term "sample".

From that point on understanding should be intuitive and most of the resulting properties of discrete signals come naturally.

Free plug-ins for Windows, MacOS and Linux. Xhip Synthesizer v8.0 and Xhip Effects Bundle v6.7.

- KVRAF
- 11774 posts since 7 Dec, 2004

camsr wrote:Yeah the ADC captures values over time but that new signal is integrated at the DAC the create the new waveform.

That's very roughly thinking in the right terms but still 100% incorrect.

It isn't possible to integrate an infinitesimal sample value because it has no length. The length is "so small as to be unmeasurable".

https://en.wikipedia.org/wiki/Infinitesimal

The integral of an infinitesimal value is zero.

If we go with your interpretation we must replace the sample with a Dirac delta function for it to work.

This is arguably useful in many cases but is entirely abstract. The fact that the delta function has a defined integral is an axiom of the heuristic: "Dirac delta function".

The article wrote:Individual digital samples are impulses. Not bandlimited impulses, ideal ones.

So in other words the entire concept begins by adopting that heuristic right from the start, but it arrogantly defines a sample as an impulse without mentioning that this is in fact not true and not a statement of fact but rather the adoption of a useful heuristic to simplify things.

Free plug-ins for Windows, MacOS and Linux. Xhip Synthesizer v8.0 and Xhip Effects Bundle v6.7.

- KVRAF
- 6775 posts since 16 Feb, 2005

aciddose wrote:The integral of an infinitesimal value is zero.

If we consider the integral of a single sample, then the result is infinitesimal small. That's what mystran was hinting at, the sampling is a value all by itself with no knowledge of it's surrounding data. Once we have two samples, we can establish a boundary for the integral and the result of 0 is valid.

Your thermometer analogy doesn't take into account that the thermometer is "integrating" heat before it's sampled. That's why a single temperature sample would be valid, whereas an arbitrary value of something arbitrary isn't.

The big big important thing with audio would be the ADC anti-imaging filter. It is what provides continuity for the sampling process. Without that filter, the aliased images would completely invalidate the signal.

Last edited by camsr on Sat Aug 26, 2017 5:36 pm, edited 1 time in total.

- KVRAF
- 11774 posts since 7 Dec, 2004

It's all a bunch of senseless hand-waving.

Every value that can be measured is being integrated by the measurement device. This is what "analog" means!

In fact this explanation is the basis of the lowest level physics: a value is only actually a value after it has been "integrated" by being measured. It is created by the measurement itself.

So you haven't actually explained anything at all and are just going in logical circles. There is absolutely zero difference between the infinitesimal sample of the value of a thermometer interpreted and recorded by an individual vs. a discrete sample point taken from an ADC.

Every value that can be measured is being integrated by the measurement device. This is what "analog" means!

In fact this explanation is the basis of the lowest level physics: a value is only actually a value after it has been "integrated" by being measured. It is created by the measurement itself.

So you haven't actually explained anything at all and are just going in logical circles. There is absolutely zero difference between the infinitesimal sample of the value of a thermometer interpreted and recorded by an individual vs. a discrete sample point taken from an ADC.

Free plug-ins for Windows, MacOS and Linux. Xhip Synthesizer v8.0 and Xhip Effects Bundle v6.7.

- KVRAF
- 6775 posts since 16 Feb, 2005

Right, and I get what you are saying. If we keep going on this train of thought we might end up arguing the theory of relativity and such, and I'm not qualified to answer such questions.

- KVRAF
- 11774 posts since 7 Dec, 2004

earlevel:

There is nothing "wrong" with adopting such a heuristic to simplify an explanation. I use the Dirac delta function to explain concepts in sampling myself.

It should start by defining "Dirac delta function" and explaining that it is being adopted as a foundation for the explanation. If additional explanation about the assumption of a uniform sequence of such functions is provided I believe it should address mystran's objection as well as my own.

I also believe that the fact that a sample is defined as an infinitesimal discrete value as measured by a measurement device should be very explicit right from the beginning of the article.

https://en.wikipedia.org/wiki/Sample

https://en.wikipedia.org/wiki/Sample_%28signal%29

There is nothing "wrong" with adopting such a heuristic to simplify an explanation. I use the Dirac delta function to explain concepts in sampling myself.

It should start by defining "Dirac delta function" and explaining that it is being adopted as a foundation for the explanation. If additional explanation about the assumption of a uniform sequence of such functions is provided I believe it should address mystran's objection as well as my own.

I also believe that the fact that a sample is defined as an infinitesimal discrete value as measured by a measurement device should be very explicit right from the beginning of the article.

https://en.wikipedia.org/wiki/Sample

https://en.wikipedia.org/wiki/Sample_%28signal%29

Free plug-ins for Windows, MacOS and Linux. Xhip Synthesizer v8.0 and Xhip Effects Bundle v6.7.

- KVRist
- 441 posts since 4 Apr, 2010

mystran wrote:NO! This is exactly what is wrong with all these "intuitive understandings" of the sampling theory. Each digital sample is simply a value of some band-limited signal at some particular point in time. It is not an impulse, it is not a sinc, it is simply a value of some signal. From the mathematical point of view it is often convenient to treat it as an impulse or sinc or whatever, but fundamentally it's just a value (of some band-limited signal at a certain point in time).

No—disagree just as strongly.

You're not just saying I'm wrong, you're saying a lot of guys on my bookshelf are wrong (starting with Rabiner and Gold), you're saying JOS is wrong—just people I can think of off the top of my head. I'm not going to do a bunch of searching to back this up, but here's JOS saying how a DAC works:

"The reconstruction of a sound from its samples can thus be interpreted as follows: convert the sample stream into a weighted impulse train, and pass that signal through an ideal lowpass filter which cuts off at half the sampling rate. These are the fundamental steps of digital to analog conversion (DAC). In practice, neither the impulses nor the lowpass filter are ideal, but they are usually close enough to ideal that one cannot hear any difference."

ref: http://ccrma.stanford.edu/~jos/mdft/Rec ... _Math.html

Any DSP textbook that shows a repeating, aliased spectrum is saying precisely that samples represent impulses. As I pointed out in my article, it's the essence of PCM.

My audio DSP blog: earlevel.com

- KVRist
- 441 posts since 4 Apr, 2010

camsr wrote:Yeah the ADC captures values over time but that new signal is integrated at the DAC the create the new waveform. ZOH is like a really bad integrator!

I think you're misinterpreting what ADC does. There is no attempt at ZOH, strictly speaking. It's just S&H long enough to get the measurement—the level can go to hell for the rest of the sample period after that, it's unused. In general, the more precise you want a measurement, the longer it takes. The hold is just so it sits still long enough to measure the "instant" it's holding on to.

My audio DSP blog: earlevel.com

- KVRist
- 441 posts since 4 Apr, 2010

aciddose wrote:earlevel:

There is nothing "wrong" with adopting such a heuristic to simplify an explanation. I use the Dirac delta function to explain concepts in sampling myself.

It should start by defining "Dirac delta function" and explaining that it is being adopted as a foundation for the explanation. If additional explanation about the assumption of a uniform sequence of such functions is provided I believe it should address mystran's objection as well as my own.

I also believe that the fact that a sample is defined as an infinitesimal discrete value as measured by a measurement device should be very explicit right from the beginning of the article.

https://en.wikipedia.org/wiki/Sample

https://en.wikipedia.org/wiki/Sample_%28signal%29

When you say, 'It should start by defining "Dirac delta function"', I'm not sure if you mean my article should, or?...not exactly following you here. In any case, I intentionally avoided mentioning Dirac deltas, though passing mention has been in my many video script rewrites. I thought long and hard about how to explain it, and decided that for those who can grasp AM and that a pulse train is equivalent to an infinite series cosine waves at integer multiples of the sampling frequency, the spectrum becomes obvious.

Sure, that's not everyone, but then why should I give the same explanation everyone else does? If people didn't figure it out from what's out there, why should I repeat?

My audio DSP blog: earlevel.com