Sampling theory—"best" explanation

JCJR
 KVRAF
 2345 posts since 17 Apr, 2005 from S.E. TN
Re: Sampling theory—"best" explanation
Hmmm, envisioning a simple "roll yer own" R2R zerohold DAC at 44.1 kHz samplerate, followed by maybe an 8th order analog butterworth lowpass filter tuned to 15 or 20 Khz Fc, whatever Maybe an elliptical filter would do better but more rilnging
Each S&H "freeze frame" of the zeroorder hold ought to have duration of about 22.68 uSec. Which is a "fairly short pulse" for analog audio work.
If we feed JUST ONE fairly sharp 22.68 us pulse into the analog lowpass filter and record filter output via storage scope, would the capture show a waveform resembling a sinc? I suspect so but dunno and I don't recall ever being imaginative enough to make such a test.
Maybe as the pulse duration is made shorter and the lowpass filter is made better, the resemblance to sinc would continue to improve? Though the filter output amplitude would get smaller as the filter is improved and the pulse is made narrower?
Each S&H "freeze frame" of the zeroorder hold ought to have duration of about 22.68 uSec. Which is a "fairly short pulse" for analog audio work.
If we feed JUST ONE fairly sharp 22.68 us pulse into the analog lowpass filter and record filter output via storage scope, would the capture show a waveform resembling a sinc? I suspect so but dunno and I don't recall ever being imaginative enough to make such a test.
Maybe as the pulse duration is made shorter and the lowpass filter is made better, the resemblance to sinc would continue to improve? Though the filter output amplitude would get smaller as the filter is improved and the pulse is made narrower?

aciddose
 KVRAF
 11941 posts since 7 Dec, 2004
Re: Sampling theory—"best" explanation
Yes. The filter would get closer to a minimum phase version of sinc() the more it approximated a "brick wall" filter. It is not possible to construct a linear phase analog filter, although it is possible to approximate one with other caveats.
"Just one fairly sharp pulse" though is the definition of the Dirac delta function. There is no "ideal" function as such a function is impossible but the delta function is considered the limit to improving approximations. (An infinite amplitude impulse with zero length.)
Again it isn't possible to produce a linear phase analog delta but it is possible to approximate one. You'd end up with something more like a minimum phase version in any real circuit.
"Just one fairly sharp pulse" though is the definition of the Dirac delta function. There is no "ideal" function as such a function is impossible but the delta function is considered the limit to improving approximations. (An infinite amplitude impulse with zero length.)
Again it isn't possible to produce a linear phase analog delta but it is possible to approximate one. You'd end up with something more like a minimum phase version in any real circuit.
Last edited by aciddose on Sat Aug 26, 2017 6:34 pm, edited 1 time in total.
Free plugins for Windows, MacOS and Linux. Xhip Synthesizer v8.0 and Xhip Effects Bundle v6.7.

camsr
 KVRAF
 6859 posts since 17 Feb, 2005
Re: Sampling theory—"best" explanation
I was refering to the DAC, where a ZOH is a really bad integratorearlevel wrote:I think you're misinterpreting what ADC does. There is no attempt at ZOH, strictly speaking. It's just S&H long enough to get the measurement—the level can go to hell for the rest of the sample period after that, it's unused. In general, the more precise you want a measurement, the longer it takes. The hold is just so it sits still long enough to measure the "instant" it's holding on to.camsr wrote:Yeah the ADC captures values over time but that new signal is integrated at the DAC the create the new waveform. ZOH is like a really bad integrator!

earlevel
 KVRist
 474 posts since 4 Apr, 2010
Re: Sampling theory—"best" explanation
OIC. OK, reluctant to talk about DAC hardware anymore, since my article doesn't and this thread started off on the wrong track by erroneously going down that path...but I'll bite off just a little...camsr wrote:I was refering to the DAC, where a ZOH is a really bad integratorearlevel wrote:I think you're misinterpreting what ADC does. There is no attempt at ZOH, strictly speaking. It's just S&H long enough to get the measurement—the level can go to hell for the rest of the sample period after that, it's unused. In general, the more precise you want a measurement, the longer it takes. The hold is just so it sits still long enough to measure the "instant" it's holding on to.camsr wrote:Yeah the ADC captures values over time but that new signal is integrated at the DAC the create the new waveform. ZOH is like a really bad integrator!
The problem is that trapezoidal integration is wrong too. Meanwhile, ZOH is very easy to achieve, and trivial to correct for, especially in an oversampling DAC. Maybe first order hold is used in some converters, I don't know—but my point is that there is little motivation.
interestingly, wikipedia's first sentences for "zeroorder hold" and "firstorder hold" (emphasis added):
zeroorder: "The zeroorder hold (ZOH) is a mathematical model of the practical signal reconstruction done by a conventional digitaltoanalog converter (DAC)."
firstorder: "The firstorder hold (FOH) is a mathematical model of the practical reconstruction of sampled signals that could be done by a conventional digitaltoanalog converter (DAC) and an analog circuit called an integrator."
My audio DSP blog: earlevel.com

aciddose
 KVRAF
 11941 posts since 7 Dec, 2004
Re: Sampling theory—"best" explanation
Well, it's more about acknowledging that you are still repeating just a slightly varied version of the same explanation.earlevel wrote:When you say, 'It should start by defining "Dirac delta function"', I'm not sure if you mean my article should, or?...not exactly following you here. In any case, I intentionally avoided mentioning Dirac deltas, though passing mention has been in my many video script rewrites. I thought long and hard about how to explain it, and decided that for those who can grasp AM and that a pulse train is equivalent to an infinite series cosine waves at integer multiples of the sampling frequency, the spectrum becomes obvious.
Sure, that's not everyone, but then why should I give the same explanation everyone else does? If people didn't figure it out from what's out there, why should I repeat?
Most people who get into DSP probably have no trouble understanding the explanation that starts right from the beginning by defining some axioms including "sample = a Dirac delta where its integral = value of the sample". In my experience I've found that in fact what really throws me off is where an article or lecture never gets into explaining these fundamental concepts and explicitly defining the axioms it uses to form the heuristic it is founded on.
So I'm not suggesting your article should go into a huge depth about these subjects but that it should simply mention them in passing:
 "we'll start with a set of axioms based upon the Dirac delta function which you might want to look up to gain a further understanding but we won't go into detail here." ...
 "Our axioms add some additional terms. We define the sample as being an impulse which has an amplitude equal to the sampled value and we define its integral as the same value."
I also believe it is essential to point out which assumptions/axioms are being made and what sort of heuristic is being utilized while making that explicitly clear. "This isn't truth, but it is as close as you'll probably want to sit through a video for and it will hopefully be a great starting point to open that door if you've had trouble with it."
Free plugins for Windows, MacOS and Linux. Xhip Synthesizer v8.0 and Xhip Effects Bundle v6.7.

camsr
 KVRAF
 6859 posts since 17 Feb, 2005
Re: Sampling theory—"best" explanation
It's why I said your article was mostly theory. A sample provides a value to be integrated thereby making it into a continuous signal. I might be considering the dirac delta as an integrator which is probably confusing.. but aciddose is right to say that the infintesimal (or invalid) integral of a sample is nonsense (DAC side). Mystran was also right to say that the sampling is an (almost) instantaneous snapshot of the signal (ADC side). ADC/DAC are both different concepts but the fundamentals are the same.
The way I understand an "oldschool" DAC to work is that the bitdepth controls directly an equivalent number of transistors to output a voltage and/or current. In the time between samples, the transistor output is basically constant, and this is where the ZOH comes from. The antialias filter then smooths and bandlimits the output. But that's the oldschool DAC, and I haven't seen one of those... even cheap consumer equipment has deltasigma conversion.
I'm not sure what a deltasigma DAC does with the electronics, but there is probably a ZOH somewhere, it's illeffect minimized at the same time.
The way I understand an "oldschool" DAC to work is that the bitdepth controls directly an equivalent number of transistors to output a voltage and/or current. In the time between samples, the transistor output is basically constant, and this is where the ZOH comes from. The antialias filter then smooths and bandlimits the output. But that's the oldschool DAC, and I haven't seen one of those... even cheap consumer equipment has deltasigma conversion.
I'm not sure what a deltasigma DAC does with the electronics, but there is probably a ZOH somewhere, it's illeffect minimized at the same time.

aciddose
 KVRAF
 11941 posts since 7 Dec, 2004
Re: Sampling theory—"best" explanation
Deltasigma is simply a pulse generator followed by an integrator and comparator in a feedback loop. So it's nothing but a "1bit oldschool DAC" in essence.
Actual converters are more complex than that but it's essentially a sort of PWM at a much higher frequency which approximates an ideal reconstruction better and more cheaply than a ZOH+filter could.
It's much like earlevel mentioned; It is possible to overcome the issues with ZOH by oversampling; the problem though is the amount of oversampling required and the expense of producing accurate filters vs. the resulting levels of noise.
Deltasigma converters are used because they overcome the limitations of a "classical" ZOH DAC and provide much better results.
Actual converters are more complex than that but it's essentially a sort of PWM at a much higher frequency which approximates an ideal reconstruction better and more cheaply than a ZOH+filter could.
It's much like earlevel mentioned; It is possible to overcome the issues with ZOH by oversampling; the problem though is the amount of oversampling required and the expense of producing accurate filters vs. the resulting levels of noise.
Deltasigma converters are used because they overcome the limitations of a "classical" ZOH DAC and provide much better results.
Free plugins for Windows, MacOS and Linux. Xhip Synthesizer v8.0 and Xhip Effects Bundle v6.7.

earlevel
 KVRist
 474 posts since 4 Apr, 2010
Re: Sampling theory—"best" explanation
Yes, that's the idea. When you're writing DSP code, that's what you're working with. Then you let your computer hand it off to the converter of your choice, and it has physical constraints that it solves to get as close to the ideal as it can. But like it says at the bottom of my posts, "My audio DSP blog..."camsr wrote:It's why I said your article was mostly theory.
My audio DSP blog: earlevel.com

PurpleSunray
 KVRian
 813 posts since 13 Mar, 2012
Re: Sampling theory—"best" explanation
Still don't get what's the target audience.earlevel wrote:This sentence worth discussing.PurpleSunray wrote:If someone knows what impulses are or bandlimited is, he knows about DSP already.
In my experience, lots of people who know DSP do not know know that sample represent impulses, and do not understand how the frequency spectrum is aliased. (I've had related arguments online for decades.) No?
Or, if they do, I think the vast majority don't understand why. I never learned why from any of the classic DSP text books on my shelf (I had to think about it). They explain the characteristics of digital samples, but not how we got there.
No?
If you want to address ppl with DSP knowlege, that want to better understand sampling, you need to be fck deep and exact and correct. Or you have just kicked of an endless discussion with them about if you'r correct (just like here ).
If you want to address ppl with no DSP, you need to switch to simple mode. When I switch from dev to musican mode, I stop reading your articel after
what?? digital samples are what I move arround on my DAW. captured it from analog synth, so they are digital now, no?Individual digital samples are impulses.
wtf u mean with this are impluse???
~~ ॐ http://soundcloud.com/mfr ॐ ~~

stratum
 KVRAF
 1843 posts since 29 May, 2012
Re: Sampling theory—"best" explanation
It wasn't such a confusing article  I don't know why it created so much discussion regarding its suitability to the so called "target audience". If somebody didn't work with equations related to DSP for about 20 years, usually he cannot read a typical DSP text effortlessly, that doesn't mean he would necessarily have problem with technical terms. Assumptions presented in the thread regarding the 'typical/target audience' are wrong.
~stratum~

matt42
 KVRian
 1057 posts since 9 Jan, 2006

stratum
 KVRAF
 1843 posts since 29 May, 2012
Re: Sampling theory—"best" explanation
I'll tell you what would really be misleading to beginners: Ignore the rest of the literature, do your own thing, present your ideas without adequately highlighting the key differences/disagreements that you have with respect to the rest of the literature and rely on the reader's ability to follow equations to understand your position. That's a lot more confusing than technical terms and slight inaccuracies that may cause endless discussions. This article does not even come close in terms of its potential to create confusion. It just reiterates information available elsewhere using a different presentation style involving discretetime analog circuits (those BBD chips mentioned) and uses amplitude modulation involved in that case as a precursor to explain the meaning of the digital representation. Whatever meaning that can be assigned to a representation can cause endless debates just by itself. Was intentionality in the head or wasn't? (I'm not sure the question has been answered or kept as useful material to write articles in the subject of the philosophy of language). What we can conclude from that is that all meanings that you can assign to such a representation are equally valid: Whether it's impulses, staircases or lines connecting dots (so that you can do trapezoidal integration) is the meaning you assign. It's a choice. There are endless options. Failing to note this is what's wrong with all explanations of sampling and discretization tutorials/blogs I have seen, not technical terms or slight inaccuracies.
Last edited by stratum on Sun Aug 27, 2017 8:09 am, edited 3 times in total.
~stratum~

Oden
 KVRist
 219 posts since 30 Oct, 2010
Re: Sampling theory—"best" explanation
"Individual digital samples are impulses".
Erm... ok. I don't like it when people confuse the mathematical world with the real world. Even though the mathematical models work well, they don't tell us what the reality of the situation is. Besides that the models describe things that can't even principle exist or at least be observed ie. infinite sine waves, perfect circles etc.
For this reason it's probably best to leave the philosophical declarations out of the explanation. The samples for sure aren't ideal impulses, such thing hasn't even been demonstrated to exist, and in my opinion can't even exist due to the contradictory nature. Besides, I am pretty sure that the samples reside in the hard drive anyway. But each to their own.
So, in the mathematical theorem, there are ideal impulses. Whether the theorem talks about the real world or real world "samples" is a other question entirely.
Erm... ok. I don't like it when people confuse the mathematical world with the real world. Even though the mathematical models work well, they don't tell us what the reality of the situation is. Besides that the models describe things that can't even principle exist or at least be observed ie. infinite sine waves, perfect circles etc.
For this reason it's probably best to leave the philosophical declarations out of the explanation. The samples for sure aren't ideal impulses, such thing hasn't even been demonstrated to exist, and in my opinion can't even exist due to the contradictory nature. Besides, I am pretty sure that the samples reside in the hard drive anyway. But each to their own.
So, in the mathematical theorem, there are ideal impulses. Whether the theorem talks about the real world or real world "samples" is a other question entirely.

PurpleSunray
 KVRian
 813 posts since 13 Mar, 2012
Re: Sampling theory—"best" explanation
I'm the only one talking about "target audience", didn't created much discussion. The discussion was/is about the tech details.stratum wrote:It wasn't such a confusing article  I don't know why it created so much discussion regarding its suitability to the so called "target audience". If somebody didn't work with equations related to DSP for about 20 years, usually he cannot read a typical DSP text effortlessly, that doesn't mean he would necessarily have problem with technical terms. Assumptions presented in the thread regarding the 'typical/target audience' are wrong.
Nope, that wasn't a confusing article, it's just nowhere the "best". Neither for beginners, nor for ppl that do already know what bandlimited is.
It's a blog post / brain dump.
It if it shall be the "best", think about if it shall the best for noob or pro, cuz itsn't same. That's all I'm saying.
If it's for beginners beginners you start to explain basics, litterature and and your different interepration of it comes when they know what you'r writng about.stratum wrote: I'll tell you what would really be misleading to beginners: <walloftext>.
~~ ॐ http://soundcloud.com/mfr ॐ ~~

stratum
 KVRAF
 1843 posts since 29 May, 2012
Re: Sampling theory—"best" explanation
Don't take it personally, you weren't the only one with that opinion.I'm the only one talking about "target audience", didn't created much discussion.
Well yes it's a blog and not claimed to be anything else.It's a blog post / brain dump
~stratum~