Sampling theory—"best" explanation

[JCJR]

Hmmm, envisioning a simple "roll yer own" R2R zero-hold DAC at 44.1 kHz samplerate, followed by maybe an 8th order analog butterworth lowpass filter tuned to 15 or 20 Khz Fc, whatever-- Maybe an elliptical filter would do better but more rilnging--

Each S&H "freeze frame" of the zero-order hold ought to have duration of about 22.68 uSec. Which is a "fairly short pulse" for analog audio work.

If we feed JUST ONE fairly sharp 22.68 us pulse into the analog lowpass filter and record filter output via storage scope, would the capture show a waveform resembling a sinc? I suspect so but dunno and I don't recall ever being imaginative enough to make such a test.

Maybe as the pulse duration is made shorter and the lowpass filter is made better, the resemblance to sinc would continue to improve? Though the filter output amplitude would get smaller as the filter is improved and the pulse is made narrower?

[aciddose]

Yes. The filter would get closer to a minimum phase version of sinc() the more it approximated a "brick wall" filter. It is not possible to construct a linear phase analog filter, although it is possible to approximate one with other caveats.

"Just one fairly sharp pulse" though is the definition of the Dirac delta function. There is no "ideal" function as such a function is impossible but the delta function is considered the limit to improving approximations. (An infinite amplitude impulse with zero length.)

Again it isn't possible to produce a linear phase analog delta but it is possible to approximate one. You'd end up with something more like a minimum phase version in any real circuit.

[camsr]

Yeah the ADC captures values over time but that new signal is integrated at the DAC the create the new waveform. ZOH is like a really bad integrator!

I think you're misinterpreting what ADC does. There is no attempt at ZOH, strictly speaking. It's just S&H long enough to get the measurement—the level can go to hell for the rest of the sample period after that, it's unused. In general, the more precise you want a measurement, the longer it takes. The hold is just so it sits still long enough to measure the "instant" it's holding on to.

I was refering to the DAC, where a ZOH is a really bad integrator

[earlevel]

Yeah the ADC captures values over time but that new signal is integrated at the DAC the create the new waveform. ZOH is like a really bad integrator!

I think you're misinterpreting what ADC does. There is no attempt at ZOH, strictly speaking. It's just S&H long enough to get the measurement—the level can go to hell for the rest of the sample period after that, it's unused. In general, the more precise you want a measurement, the longer it takes. The hold is just so it sits still long enough to measure the "instant" it's holding on to.

I was refering to the DAC, where a ZOH is a really bad integrator

OIC. OK, reluctant to talk about DAC hardware anymore, since my article doesn't and this thread started off on the wrong track by erroneously going down that path...but I'll bite off just a little...

The problem is that trapezoidal integration is wrong too. Meanwhile, ZOH is very easy to achieve, and trivial to correct for, especially in an oversampling DAC. Maybe first order hold is used in some converters, I don't know—but my point is that there is little motivation.

interestingly, wikipedia's first sentences for "zero-order hold" and "first-order hold" (emphasis added):

zero-order: "The zero-order hold (ZOH) is a mathematical model of the practical signal reconstruction done by a conventional digital-to-analog converter (DAC)."

first-order: "The first-order hold (FOH) is a mathematical model of the practical reconstruction of sampled signals that could be done by a conventional digital-to-analog converter (DAC) and an analog circuit called an integrator."

[aciddose]

When you say, 'It should start by defining "Dirac delta function"', I'm not sure if you mean my article should, or?...not exactly following you here. In any case, I intentionally avoided mentioning Dirac deltas, though passing mention has been in my many video script rewrites. I thought long and hard about how to explain it, and decided that for those who can grasp AM and that a pulse train is equivalent to an infinite series cosine waves at integer multiples of the sampling frequency, the spectrum becomes obvious.

Sure, that's not everyone, but then why should I give the same explanation everyone else does? If people didn't figure it out from what's out there, why should I repeat?

Well, it's more about acknowledging that you are still repeating just a slightly varied version of the same explanation.

Most people who get into DSP probably have no trouble understanding the explanation that starts right from the beginning by defining some axioms including "sample = a Dirac delta where its integral = value of the sample". In my experience I've found that in fact what really throws me off is where an article or lecture never gets into explaining these fundamental concepts and explicitly defining the axioms it uses to form the heuristic it is founded on.

So I'm not suggesting your article should go into a huge depth about these subjects but that it should simply mention them in passing:

• "we'll start with a set of axioms based upon the Dirac delta function which you might want to look up to gain a further understanding but we won't go into detail here." ...
• "Our axioms add some additional terms. We define the sample as being an impulse which has an amplitude equal to the sampled value and we define its integral as the same value."

That will go over 99% the heads out there, but so will the other parts of your explanation. The simple fact is that a huge majority of people are simply not intelligent enough to understand much at all. For those who are capable of developing a true understanding however I believe mentioning these subjects even briefly will open up a path for those individuals to study further to get the sort of understanding they're looking for.

I also believe it is essential to point out which assumptions/axioms are being made and what sort of heuristic is being utilized while making that explicitly clear. "This isn't truth, but it is as close as you'll probably want to sit through a video for and it will hopefully be a great starting point to open that door if you've had trouble with it."

[camsr]

It's why I said your article was mostly theory. A sample provides a value to be integrated thereby making it into a continuous signal. I might be considering the dirac delta as an integrator which is probably confusing.. but aciddose is right to say that the infintesimal (or invalid) integral of a sample is nonsense (DAC side). Mystran was also right to say that the sampling is an (almost) instantaneous snapshot of the signal (ADC side). ADC/DAC are both different concepts but the fundamentals are the same.

The way I understand an "old-school" DAC to work is that the bitdepth controls directly an equivalent number of transistors to output a voltage and/or current. In the time between samples, the transistor output is basically constant, and this is where the ZOH comes from. The anti-alias filter then smooths and bandlimits the output. But that's the old-school DAC, and I haven't seen one of those... even cheap consumer equipment has delta-sigma conversion.

I'm not sure what a delta-sigma DAC does with the electronics, but there is probably a ZOH somewhere, it's ill-effect minimized at the same time.

[aciddose]

Delta-sigma is simply a pulse generator followed by an integrator and comparator in a feedback loop. So it's nothing but a "1-bit old-school DAC" in essence.

Actual converters are more complex than that but it's essentially a sort of PWM at a much higher frequency which approximates an ideal reconstruction better and more cheaply than a ZOH+filter could.

It's much like earlevel mentioned; It is possible to overcome the issues with ZOH by oversampling; the problem though is the amount of oversampling required and the expense of producing accurate filters vs. the resulting levels of noise.

Delta-sigma converters are used because they overcome the limitations of a "classical" ZOH DAC and provide much better results.

[earlevel]

It's why I said your article was mostly theory.

Yes, that's the idea. When you're writing DSP code, that's what you're working with. Then you let your computer hand it off to the converter of your choice, and it has physical constraints that it solves to get as close to the ideal as it can. But like it says at the bottom of my posts, "My audio DSP blog..."

[PurpleSunray]

If someone knows what impulses are or bandlimited is, he knows about DSP already.

This sentence worth discussing.

In my experience, lots of people who know DSP do not know know that sample represent impulses, and do not understand how the frequency spectrum is aliased. (I've had related arguments online for decades.) No?

Or, if they do, I think the vast majority don't understand why. I never learned why from any of the classic DSP text books on my shelf (I had to think about it). They explain the characteristics of digital samples, but not how we got there.

No?

Still don't get what's the target audience.
If you want to address ppl with DSP knowlege, that want to better understand sampling, you need to be fck deep and exact and correct. Or you have just kicked of an endless discussion with them about if you'r correct (just like here ).
If you want to address ppl with no DSP, you need to switch to simple mode. When I switch from dev to musican mode, I stop reading your articel after

Individual digital samples are impulses.

what?? digital samples are what I move arround on my DAW. captured it from analog synth, so they are digital now, no?
wtf u mean with this are impluse???

[stratum]

It wasn't such a confusing article - I don't know why it created so much discussion regarding its suitability to the so called "target audience". If somebody didn't work with equations related to DSP for about 20 years, usually he cannot read a typical DSP text effortlessly, that doesn't mean he would necessarily have problem with technical terms. Assumptions presented in the thread regarding the 'typical/target audience' are wrong.

[matt42]

Redacted

[stratum]

I'll tell you what would really be misleading to beginners: Ignore the rest of the literature, do your own thing, present your ideas without adequately highlighting the key differences/disagreements that you have with respect to the rest of the literature and rely on the reader's ability to follow equations to understand your position. That's a lot more confusing than technical terms and slight inaccuracies that may cause endless discussions. This article does not even come close in terms of its potential to create confusion. It just reiterates information available elsewhere using a different presentation style involving discrete-time analog circuits (those BBD chips mentioned) and uses amplitude modulation involved in that case as a precursor to explain the meaning of the digital representation. Whatever meaning that can be assigned to a representation can cause endless debates just by itself. Was intentionality in the head or wasn't? (I'm not sure the question has been answered or kept as useful material to write articles in the subject of the philosophy of language). What we can conclude from that is that all meanings that you can assign to such a representation are equally valid: Whether it's impulses, staircases or lines connecting dots (so that you can do trapezoidal integration) is the meaning you assign. It's a choice. There are endless options. Failing to note this is what's wrong with all explanations of sampling and discretization tutorials/blogs I have seen, not technical terms or slight inaccuracies.

[Oden]

"Individual digital samples are impulses".

Erm... ok. I don't like it when people confuse the mathematical world with the real world. Even though the mathematical models work well, they don't tell us what the reality of the situation is. Besides that the models describe things that can't even principle exist or at least be observed ie. infinite sine waves, perfect circles etc.

For this reason it's probably best to leave the philosophical declarations out of the explanation. The samples for sure aren't ideal impulses, such thing hasn't even been demonstrated to exist, and in my opinion can't even exist due to the contradictory nature. Besides, I am pretty sure that the samples reside in the hard drive anyway. But each to their own.

So, in the mathematical theorem, there are ideal impulses. Whether the theorem talks about the real world or real world "samples" is a other question entirely.

[PurpleSunray]

It wasn't such a confusing article - I don't know why it created so much discussion regarding its suitability to the so called "target audience". If somebody didn't work with equations related to DSP for about 20 years, usually he cannot read a typical DSP text effortlessly, that doesn't mean he would necessarily have problem with technical terms. Assumptions presented in the thread regarding the 'typical/target audience' are wrong.

I'm the only one talking about "target audience", didn't created much discussion. The discussion was/is about the tech details.
Nope, that wasn't a confusing article, it's just nowhere the "best". Neither for beginners, nor for ppl that do already know what bandlimited is.
It's a blog post / brain dump.
It if it shall be the "best", think about if it shall the best for noob or pro, cuz itsn't same. That's all I'm saying.

I'll tell you what would really be misleading to beginners: <walloftext>.

If it's for beginners beginners you start to explain basics, litterature and and your different interepration of it comes when they know what you'r writng about.

[stratum]

I'm the only one talking about "target audience", didn't created much discussion.

Don't take it personally, you weren't the only one with that opinion.

It's a blog post / brain dump

Well yes it's a blog and not claimed to be anything else.