Sampling theory—"best" explanation

[earlevel]

Still don't get what's the target audience.

Whoever wants to read it.

The opening gets to the points pretty quickly—at the very beginning, it just right on terms like bandlimited impulses and modulated audio. The first article fills a screen and a half on a 24" monitor—the reader can glance at it in seconds, noting the diagrams, and figure out pretty quickly if it's worth reading. Someone who isn't at all in the target audience won't find their way to my website.

As for which DSPers, why limit? A beginner would do well to catch the story of how this evolved along communications, and many will probably get the idea of what "impulse" means by looking at the diagrams. (The biggest stretch is probably AM—ideally, I wanted an interactive widget to explore AM, but I don't have the time—the video will elaborate.) And for long-time DSPers, it would be news for many that samples are impulses and why that means there are always aliased images in the digital domain. No disrespect meant—a good mechanic may not fully comprehend exactly how a carburetor works, or how fuel injection accomplishes the same thing.

I'm not sure what people expect here. I don't recall generally seeing instructions at the start of technical articles, or otherwise, about exactly who should be reading them.

[earlevel]

Individual digital samples are impulses.

what?? digital samples are what I move arround on my DAW. captured it from analog synth, so they are digital now, no?
wtf u mean with this are impluse???

First, let me say that I truly did shoot for not wasting words in this series. So if you feel I should have elaborated, maybe so, but it was intentional. I wanted to pare it down as much as I could—the video will benefit from more elaborate visuals, and I don't like 30-60 minute technical videos. As I said, the articles are a vehicle for me to finish that video and quit doing rewrites.

Look at the PAM diagram on the second article. That's it. The value of a digital sample represents a point in time (point as in "infinitesimal", to borrow from this thread). It is not shorthand for a level that fills between two sampling points, it is not shorthand for a sinc waveform of that height and centered on that point.

[earlevel]

Whether it's impulses, staircases or lines connecting dots (so that you can do trapezoidal integration) is the meaning you assign. It's a choice.

Sorry, I didn't follow most of your post. but this...No...you're confusing things here.

I make it clear that the "Individual digital samples" represent impulses (Part 1), "digital samples represent impulses" (Part 2), "once in the digital domain, samples represent perfect impulses" (Part 3).

A sample value, the digital value you run through a lowpass filter, or whatever, in your DSP algorithm, represents an impulse. It's the essence of the collective article. You are making a profound mistake if you think of the sample as connecting in any way. There is an endless stream of zero between them. If you want to make absolutely no changes to the spectrum but up the sample rate to 10x, you would put nine zeros after each sample.

[earlevel]

"Individual digital samples are impulses".

Erm... ok. I don't like it when people confuse the mathematical world with the real world. Even though the mathematical models work well, they don't tell us what the reality of the situation is. Besides that the models describe things that can't even principle exist or at least be observed ie. infinite sine waves, perfect circles etc.

For this reason it's probably best to leave the philosophical declarations out of the explanation. The samples for sure aren't ideal impulses, such thing hasn't even been demonstrated to exist, and in my opinion can't even exist due to the contradictory nature. Besides, I am pretty sure that the samples reside in the hard drive anyway. But each to their own.

So, in the mathematical theorem, there are ideal impulses. Whether the theorem talks about the real world or real world "samples" is a other question entirely.

See my reply to stratum. I'm not confusing that mathematical world with the real world. It's a DSP blog, not an "all things digital audio" blog (see the upper right corner of the blog, see my signature line below this post), and DSP is about the mathematical world.

[aciddose]

People have a hard time understanding concepts like an axiom or a heuristic.

Using these terms immediately might encourage people to look them up. In fact in a web article I'd simply link these terms to the corresponding wikipedia article!

The same for: a sample is an infinitesimal on the time axis.

An infinitesimal is a mathematical concept that has been abandoned in the past but more recently has been extensively developed. This is an advanced subject that most people (even mathematicians) may not be aware of.

If people start to learn about these words (vocabulary is a great sign of a developed intellect) they will be able to understand these very complex topics in a more intuitive way.

So I believe the solution is to offer up these terms in the introduction to the article. Point out the article uses a specific heuristic based upon the Dirac delta function with some additional axioms: The amplitude of a sample is equal to its value with its integral equal to the same value. This is the same as the sinc function.

Now things start to suddenly make a lot of sense! Why do people say "a sample is thought of as a sinc function": they're using a heuristic without explaining it, most likely because people are so f**king stupid they don't even know what a heuristic is.

Heuristic: a process or method enabling a person to discover or learn something for themselves. <-- CLICK HERE!

Our educational systems are an outright failure. They do not teach foundational concepts in critical thinking.

[stratum]

Sorry, I didn't follow most of your post. but this...No...you're confusing things here.

I make it clear that the "Individual digital samples" represent impulses (Part 1), "digital samples represent impulses" (Part 2), "once in the digital domain, samples represent perfect impulses" (Part 3).

A sample value, the digital value you run through a lowpass filter, or whatever, in your DSP algorithm, represents an impulse. It's the essence of the collective article. You are making a profound mistake if you think of the sample as connecting in any way. There is an endless stream of zero between them. If you want to make absolutely no changes to the spectrum but up the sample rate to 10x, you would put nine zeros after each sample.

Of course, there is no disagreement here. The sampling theory and the reconstruction of the sampled signal to obtain the analog signal back motivates this interpretation. That's the theory and by that theory that's the meaning assigned to the digital representation. But it's not the only possible interpretation, you know people are free to assume whatever they want if in the end it's going to be useful to solve some problem, and after it is solved, somebody can rewrite the solution in a way compatible with the 'correct' assumptions, if that is a formal text. That is going to obscure the reasoning that was behind the solution, though. Have a look at https://www.native-instruments.com/file ... _1.1.1.pdf chapter 3 to see what I mean (the first two pages will suffice). If you are making a virtual analog filter, it's better you choose the meaning of the digital representation to be virtually analog, of course. But you cannot write that sentence in a formal text exactly this way without creating endless discussions, so that's the way the book is.

[earlevel]

Sorry, I didn't follow most of your post. but this...No...you're confusing things here.

I make it clear that the "Individual digital samples" represent impulses (Part 1), "digital samples represent impulses" (Part 2), "once in the digital domain, samples represent perfect impulses" (Part 3).

A sample value, the digital value you run through a lowpass filter, or whatever, in your DSP algorithm, represents an impulse. It's the essence of the collective article. You are making a profound mistake if you think of the sample as connecting in any way. There is an endless stream of zero between them. If you want to make absolutely no changes to the spectrum but up the sample rate to 10x, you would put nine zeros after each sample.

Of course, there is no disagreement here. The sampling theory and the reconstruction of the sampled signal to obtain the analog signal back motivates this interpretation. That's the theory and by that theory that's the meaning assigned to the digital representation. But it's not the only possible interpretation, you know people are free to assume whatever they want if in the end it's going to be useful to solve some problem, and after it is solved, somebody can rewrite the solution in a way compatible with the 'correct' assumptions, if that is a formal text. That is going to obscure the reasoning that was behind the solution, though. Have a look at https://www.native-instruments.com/file ... _1.1.1.pdf chapter 3 to see what I mean (the first two pages will suffice).

Of course people are free to use the interpretation they want. But there are interpretations that work for some things and fail for others ("digital samples represent sinc functions"), ones that mostly fail but might let you get by (you can repeat samples, essentially "digital sample represent ZOH levels", or linear interpolation is good enough, essentially "samples are connected with a straight line"), and one that works ("digital samples represent impulses"). I'm teaching the one that works (and is the actual basis, not a model). Maybe you're being too general for me to get your exact point. But, basically, the purpose of the article is to teach the truth. I don't care if people don't want the truth, not my problem.

[stratum]

But, basically, the purpose of the article is to teach the truth. I don't care if people don't want the truth, not my problem.

Rabiner (you had recalled his name in the thread) would say we teach people how to solve problems. But you are right if you post something at KVR you can be sure it will be taken out of context in numerous ways.

[earlevel]

Rabiner (you had recalled his name in the thread) would say we teach people how to solve problems. But you are right if you post something at KVR you can be sure it will be taken out of context in numerous ways.

That's fine on its own...but I have some comments about what you might be getting at with this...

Rabiner didn't say anything about Dirac deltas in either book of his I have, nor likened samples to sinc functions (samples are represented as impulses throughout). But I'm guessing that you're not saying I should have included things like that in my article, or even muddied the waters by saying my explanation is just one of many possible. I suppose you're saying that I'm being too restrictive on this matter in this forum discussion. But I think you may have missed some context. I wasn't saying that "thou shalt not speak of samples in any other form than impulses". I was defending that they are impulses, mainly to two commenters who said in no uncertain terms that I'm wrong for saying they are impulses. Since that's precisely what digital samples represent, I have to say as much.

OK, one of those commenters might have been talking about what they become, coming out of the DAC—though my article didn't touch on DAC hardware so that wasn't clear to me. The other ("NO!...") was clear and forceful in his view that samples are not impulses mathematically at all. He didn't say, "I find it useful to view...", he said I was plain wrong. I pounded the table right back that he was wrong that I was wrong—that's it. (I want to make it clear that I highly respect this person, and read his words on filters and other topics with keen interest. I find it not that uncommon that even smart and well-studied people don't get that samples are impulses, mathematically, and that was a motivation for the article from the beginning.)

[earlevel]

Our educational systems are an outright failure. They do not teach foundational concepts in critical thinking.

Whew, truth. A few moments on Facebook sure backs that up...

I learned the word "heuristic" from 2001: A Space Odyssey (first the book, then the movie, back in '68-'69), as a kid. I watched it many times back then, going to the theater by myself...

[stratum]

Rabiner didn't say anything about Dirac deltas in either book of his I have,

Yes that was a rather general comment about his attitude with respect to education.

I think your article is fine as it is and forums as discussion mediums aren't perfect. Nor formal books are often great educational material. Things could be explained better in more informal ways. A blog is a way of doing that. This is like asking, why somebody should write a blog, when there are many formal books about the subject matter anyway, if the purpose wasn't to use a different style. That different style could be useful for other things, but muddying waters in that particular article to include what I have told about the digital representation may not necessarily be a good idea given that your objective is to explain something else. It's a part of the overall picture, though, and failing to notice it often creates confusion about the very subject matter you are trying to explain. You are saying this is the truth, other people were saying it's not exactly so and there are small inaccuracies in the blog (it's a blog after all) and yet people are free to bend the truth to suit their own purposes as engineering practice. To me it looks like the so called "truth" is rooted in certain assumptions with respect to a given theory with a specific purpose.

[earlevel]

Hmmm, envisioning a simple "roll yer own" R2R zero-hold DAC at 44.1 kHz samplerate, followed by maybe an 8th order analog butterworth lowpass filter tuned to 15 or 20 Khz Fc, whatever-- Maybe an elliptical filter would do better but more rilnging--

Each S&H "freeze frame" of the zero-order hold ought to have duration of about 22.68 uSec. Which is a "fairly short pulse" for analog audio work.

If we feed JUST ONE fairly sharp 22.68 us pulse into the analog lowpass filter and record filter output via storage scope, would the capture show a waveform resembling a sinc? I suspect so but dunno and I don't recall ever being imaginative enough to make such a test.

Maybe as the pulse duration is made shorter and the lowpass filter is made better, the resemblance to sinc would continue to improve? Though the filter output amplitude would get smaller as the filter is improved and the pulse is made narrower?

Yes, the shorter duration of the pulse, the resemblance to sinc improves (same with the lowpass, separately). You can see the effects of ZOH by plotting the FFT of a 1 followed by zeros (255 zeros for 256-point FFT, for instance), and again with 1,1,0... As you might expect, you get a gain of 6dB at DC, but the response droops (in the shape of sinc) and you're down around 3 dB (3.9 comes to mind?) at half the sample rate. That's well-behaved and not hard to fix in a DAC, though, and works out better than trying to solve by going for impulses. You can search on "frequency response of ZOH" if you want to see diagrams.

[PurpleSunray]

First, let me say that I truly did shoot for not wasting words in this series. So if you feel I should have elaborated, maybe so, but it was intentional. I wanted to pare it down as much as I could—the video will benefit from more elaborate visuals, and I don't like 30-60 minute technical videos. As I said, the articles are a vehicle for me to finish that video and quit doing rewrites.

Ok, than I just got it wrong.
If it is supposted to be blog post for dsp geeks, like stratum says, this is fine. Maybe even add some math forumlas to better explain it, those ppl will understand it if you'r not going too complex.

From the title "Sampling theory—"best" explanation" I was expecting something different.
That's why I brought up that "target audience" topic, because I couldn't see in which direction you'r aiming. "best" for whom?
Best for KVR DSP forum members? You just posted it to them, have counted 1 that has learned something new, all others discuss with you about the details, since they already understood sampling theorem.
Best for KVR Production Techniques forum members? No, not rly, too much technical terms.

I would love having such a "From 0 to Sampling theorem" guide, simply because I don't know any good one.
Something you can throw after a bunch of musicans and tell them "read / watch this" and shut the f**k up .... if they run their psydo-science crackerbarrel agian and talk about why digitial is sooooooo much worse than analoge, cuz digital waveform is all angular! no round corners, too much noize!

[earlevel]

From the title "Sampling theory—"best" explanation" I was expecting something different.
That's why I brought up that "target audience" topic, because I couldn't see in which direction you'r aiming. "best" for whom?
Best for KVR DSP forum members? You just posted it to them, have counted 1 that has learned something new, all others discuss with you about details, since they already understood sampling theorem.
Best for KVR Production Techniques forum members? No, not rly, too much technical terms.

Good questions.

The goal was that at least some people would feel, "wow, that's the best explanation I've even heard" (again, this is a reference to the video the article roadmaps). That is, after they've been through wikipedia and other technical explanation after explanation. Not the title the video will use on youtube, just a statement of what I'm shooting for—brief, useful, fills in gaps with an intuitive explanation. Not a great title, just couldn't think of a great one, and didn't want it to read, effectively, "here's yet another explanation of digital sampling".

There are a zillion discussions of dither on the web. From my video on dither, I sometimes get, "Thanks—I finally get what it does now", and that's what I'm shooting for.

As for technical level, you can't target both people who know nothing and those you know it all. But the former can't use my help and the latter don't need it—my target it always somewhere in between. People who can benefit from a little different presentation than the usual that available already in quantity. I'm not too much of a hand holder—if someone doesn't get a term, they'll either be motivated to find out more about it elsewhere or they aren't interested enough. When I give source code, I don't supply project files for Xcode and VS—if you don't know, it's time to figure it out. I'm not selling anything, and the purpose isn't a demonstration of knowledge, so I don't really see the point of reiterating the state of all knowledge or thoughts on a subject. LOL, I guess I covered everything

[PurpleSunray]

ahahah, yeah just move on like that
I was just looking for someone to write that "for dummies" guide. Sick of drawing sine waves, explaining noob stuff to the [n] idiot that thinks digital is just analog, but with square waves