What is the "perfect" digital sound synthesis technique?

[Architeuthis]

What is the "perfect" digital sound synthesis technique? Here's what I mean by perfect, could fulfill one or more of these requirements:
-does not require anti-aliasing
-algorithms are very simple, no complex math to reduce unwanted noise
-despite simplicity, still flexible enough to create a variety of tones.

The only example I can think of is additive synthesis.

The bigger/alternative question is this: If analog synths never existed, if no sound was ever created from electricity, what would digital synthesizers be? What technique would be popular? What synthesis technique takes most advantage of being in a digital environment?

*Assume that aliasing would be recognized and unwanted in an alternate reality where analog synths never existed.

[aciddose]

What is the "perfect" digital sound synthesis technique?

There isn't one. You assume additive doesn't require anti-aliasing, it in fact does.

The only time it would not produce aliasing is... well, never. That isn't how signals work. It is impossible for a component in additive to have zero harmonics and due to this it is impossible to synthesize a full band (spectrum) without producing aliased harmonics.

The ideal synthesis method is actually PCM playback at a variable rate.

1. Does not produce folded aliases
2. Can produce a full band, although the band's width is set by the playback pitch
3. Requires minimal processing, in fact zero

Memory is the constraint here. See the first digital synthesizers, they played back PCM.

Later synthesizers started to work with methods like additive and phase modulation.

You could argue that PCM playback at variable rate doesn't actually amount to synthesis at all.


Another option: Just yell really loud.

[arkmabat]

Organic oscillations. Combining digital with organic fish:

http://www.synthgear.com/2009/audio-gea ... lator-fish

[Architeuthis]

The ideal synthesis method is actually PCM playback at a variable rate.

1. Does not produce folded aliases
2. Can produce a full band, although the band's width is set by the playback pitch
3. Requires minimal processing, in fact zero

Could you expand on this? Got any article links? Would like to know more. Why can't modern computers integrate variable rate playback of sound (in a cohesive DAW environment)?

[aciddose]

Check out how the Amiga computers (Paula chip) played back samples. Many older samples did this, I think the fairlights worked this way too.

By adjusting the sample rate you simply shift the position of all the harmonics you output. There is never any digital resampling so you never need to process the data and there is never any opportunity for aliases to "reflect" off nyquist, as the pitch is adjusted on the output (DAC), not "virtually" and re-sampled to a digital signal before output as in software.

No, modern computers use a fixed sample rate. You have to re-sample the signal at a variable rate and apply a filter before sampling back to digital samples. Yes, we do this although it isn't perfect unlike outputting the samples directly into the analog domain.

[aciddose]

Understanding this requires a fundamental understanding of the sampling theorem. Remember that aliases are the harmonics of the signal you store in samples. When we output the signal from a DAC, we filter those harmonics away above 20khz because we can't hear them and they are only "images" reflected upward of the main signal.

This is always the result of the samples we take as they are infinitely thin points with an infinite number of harmonics at the sample rate.

So a 1hz tone sampled at 20k has a harmonic at 39999hz. A 19999hz tone has a harmonic at 20001hz. They are not just direct harmonics but "reflected" upward.

If we were to take this sampled signal and re-sample it, the harmonic at 20001hz would be reflected off nyquist (20k) and become a tone at 19999hz. All the reflections line up with the original harmonics and so no problem, we get the same result we started with.

Now if we shift the pitch upward to 1.2x, say we have the same tone at 19999hz. When we shift the pitch it becomes 19999 * 1.2, or 23998.8hz. Our sample rate remains 20k, and so that harmonic reflects down to 16001.2hz.

The mirror image of 23998.8hz is 20001 * 1.2, or 24001.2hz.

This mirror harmonic (known as an alias) is reflected off 20k to 15998.8hz.

So the result of re-sampling digitally is two tones,
15998.8hz and 16001.2hz.

If we'd taken the output directly to the analog domain without resampling, we'd have 23998.8hz and 24001.2hz. Those would just be filtered away by our analog filters.

[aciddose]

Note also however that this method also has issues.

The output from a naive DAC is just a flat pulse, not an infinitely thin pulse. So it has its harmonic amplitude falling off at 1/n. It is "zero-order hold" of the sampled signal.

https://en.wikipedia.org/wiki/Zero-order_hold

So the spectrum we get remains imperfect without being filtered. The DAC itself can over-sample for us by outputting a thin pulse. Say it runs at 16x the sample rate and so outputs a pulse only for 1/16th the sample period. Now the harmonics will look different and the slope the amplitude falls off will be different. In fact it will be zoomed in 16x the same shape as zero-order hold's spectrum.

It will be much flatter but not perfect. The pulse must also be 16x the amplitude if we use the same analog filter, which results in noise issues.

Say we have noise floor at -100db for the zero-order hold output. This is remarkable, so it's a good example just to show how bad the situation could become.

If we over-sample using the DAC at 16x, the noise floor will be raised by 24db to -76db.

Another concern is bandwidth. The DAC could simply increase the voltage level it outputs at, however this will multiple the amount of current required to supply the DAC and filter by up to 16x also.

At some point the DAC won't be able to handle that much current and will limit the slew rate, reducing bandwidth.

We may end up with a design that outputs 4x the voltage, uses 4x the current with 1/4th the bandwidth (still well above 20k) and only +12db more noise. -88db noise floor would be absolutely amazing for a device like an old-school sampler.

To adjust amplitude we may also use over-sampling. To get each bit of amplitude requires 2x over-sample. So if we want envelopes with 8-bit accuracy we'd need 256 x 16 = 4096x.

In order to decrease the amplitude of the output we can make the pulse more narrow. This is called PWM.

This is all well and good, however at 4096x a reasonable "top" sample rate such as 40000 * 16 = 640000hz, we have 2.62ghz Far beyond the capability of such high current electronics.

So trade-offs must be made across the board.

[Architeuthis]

Since variable rate sound generation is impractical, does that leave us with additive synthesis as the next best thing? What are some other techniques that produce little aliasing in a fixed sample rate environment?

[aciddose]

There are none. Additive is no better than variable rate PCM and is incredibly expensive.

Variable rate PCM is entirely practical with reasonable trade-offs made. (see: most early digital synthesizers and samplers like the fairlight, PPG and so on.)

Additive remains for the most part impractical even with modern uber-powered CPUs.

When you do a comparison factoring in cost, variable rate PCM is the winner by far.

With reasonable trade-offs, phase and frequency modulation can be made practical. (see: DX7 and others.)

After that, re-sampling filters again beat additive by a long stretch. (see: D-50, M1 and others.)

Finally additive is added to the list although with heavy trade-offs. (I'm not aware of any early models, although I know very limited software became available on early computers.)

Now subtractive synthesizers, again with trade-offs. (see first software synthesizers after about 1995, and VST and similar toward 2000.)

None of these are "perfect", they all have their issues and additional costs must be factored in to make them practical.

It just so happens that this is the same order these digital instruments became widely available. Variable rate PCM would have been more popular if memory were cheaper. Due to the high cost of memory and quickly reducing cost of processing, other techniques took the spot-light such as digital filters applied to PCM at a fixed rate.

Finally, now that we have an amazing amount of processing power we can do additive and subtractive at reasonable quality.

Still though, fixed rate PCM remains popular due to incredible availability and low cost of memory. Instead, the additional expense of processing (once you need to use a digital filter for anti-aliasing, adding volume and other modulation is relatively less expensive) is what severely limits it.

Variable rate PCM is no longer available as it requires specialized hardware. All the signal processing must be analog, with the exception of volume modulation. Although PWM is possible, since filters and other processing must be analog anyway a VCA becomes trivial compared to PWM considering the huge expense of running the clock at a higher frequency for high accuracy.

[JCJR]

Well, capable additive needs THE OPTION of independent multi-stage amplitude and pitch envelopes for each harmonic.

And if the harmonics are only available in a conventional harmonic series related to the note pitch, it is difficult/impossible to get noisy components and enharmonic components which are important for many sounds. Without noisy/clicky/enharmonic features, additive synthesis tends to sound too sterile, plasticky artificial.

So an additive system that can do multiple pitch/amplitude envelopes, plus flexible noise and enharmonic elements, doesn't exactly sound like it describes a simple synthesizer to create.

In addition, additive synths tend to be beastly complex and tedious to create sounds, edit patches. Maybe there are some simple to edit flexible additive synths somewhere. The ones I've used required a lot of time to make patches, and even with lots of parameters, still didn't have enough parameters to make realistic sounds.

Additive needs cunning user interface to make it as easy possible to edit, avoid as much possible the user getting lost in all the parameters. A lot of design and labor on GUI as well.

[MadBrain]

What is the "perfect" digital sound synthesis technique? Here's what I mean by perfect, could fulfill one or more of these requirements:
-does not require anti-aliasing
-algorithms are very simple, no complex math to reduce unwanted noise
-despite simplicity, still flexible enough to create a variety of tones.

The only example I can think of is additive synthesis.

The bigger/alternative question is this: If analog synths never existed, if no sound was ever created from electricity, what would digital synthesizers be? What technique would be popular? What synthesis technique takes most advantage of being in a digital environment?

*Assume that aliasing would be recognized and unwanted in an alternate reality where analog synths never existed.

Sampling.

Yes I know, a boring answer, but if you look at what people actually spend their money on and what kind of physical keyboards get built, it's still clearly in the lead. And yes while it totally does alias unless you implement some costly and complex algorithms, in practical everyday usage the improvement you get from proper anti-alias over simple linear interpolation is kinda marginal and it's hard to hear the difference unless you're specifically doing a comparison or making it alias on purpose. And for software synthesis, sampling way faster than anything else (almost everything on 386s and 486s is sampling and you didn't really see real time filters and software FM until the pentium and p2). We're starting to run into problems with sampling because it's so overused, a lot of samples have become cheesey and the overall result tends to be kinda static since every note is the same. Still a "natural winner" though.

Additive synthesis has its uses (for instance in Wallander Instruments) but it makes it very easy to make static narrow-band kinds of sounds, sounding like mp3 artifacts or noise in a tube. Practically none of the sounds you can make out of directly editing additive synthesis data sounds particularly good out of the box.

Wavetable synthesis is somewhat similar to sampling but more flexible (you can sweep though the wavetable whichever way you like), although it can suffer from some of the problems of additive synthesis if not done properly (repetition of wavetable material making it have the "noise in the tube" sound).

FM synthesis is pretty versatile and it's one of my favorite techniques. Although it tends to have this jangly FM sound no matter what you do.

Phase Distortion tends to be similar to FM in some ways. Actually it's very similar to a FM synth with the feedback cranked up to the max all the time when you get into the actual maths.

Physical modeling suffers from the large investment in research you have to do. Imho we just don't understand acoustic instruments all that well. Sure if you look at papers you'll see some quite impressive math, but when you listen to the results you have to admit that there are often glaring flaws - in particular, the "noise in a tube" sound is totally pervasive and has sunk a lot of synths, such as the Yamaha VP1. IMHO the "make it follow the equations then try to make it sound good" approach is totally wrong - they should make it sound good first, and then make it sound more like the real thing!

If you're looking for an algo that's easy to anti-alias, I think DSF is a good candidate. It's not well known at all and the implementation I've heard had weaknesses, but I guess it has some potential and it's naturally alias-free.

For other synthesis methods, they tend to be underdeveloped I guess, which is where you come in! ...right? :3

[Architeuthis]

So let's go deeper, get to the heart of the problem.

Correct me if I'm wrong. Harmonics are created as a result of the velocity of change of amplitude. A sine wave has the least velocity because each sample has the least change in amplitude. Square waves have high velocity changes in amplitude because square waves go from -1 to +1 at the speed of infinity. Triangle waves have aliasing because the switch in direction (going positive or going negative) is also at a velocity of infinity. With sine waves, before they change direction, the rate of change of amplitude slows down.

Can someone tell me what I'm trying to describe so I can use the right terminology? I've never heard velocity being used to describe sound. Phase Modulation also reminds me of what I'm trying to say, since phase modulation depends on the rate of change... or something.

Edit: What I'm trying to get at is: By limiting the rate of change, you limit the number of harmonics created thereby removing aliasing. A low pass filter is something that limits the rate of change.

[aciddose]

Sine waves in theory have the least harmonic (none) because the velocity never changes. The wave is a 2d rotation in time, a circle. A perfect circle has no edges and therefore no points, points being (simplified view) harmonics.

(Think of the time axis like X, left and right. The sine is a point rotated around the X axis through Y and Z. If you were to "squish" X down to zero width, you'd see the sine is a circle around the dot at X.)

Digital sine waves however are made up of samples just like any other digital signal. It is impossible for a digital signal to have no harmonic.

Even worse, the waveforms we could actually use in a practical additive synthesizer are merely approximations, not a true rotation. Even any rotation we could compute (reasonably easy, actually) suffers from denormalization due to quantization error.

It is possible to compute a rotation using a precomputed matrix (coefficients depending upon frequency) and handle frequent renormalization, although this can be a bit expensive. A trade-off must be made between maximum allowed harmonic amplitude (essentially the additive "noise floor") and processing cost. As denormalization is allowed to occur the amplitude of each partial will be increasingly erroneous and renormalization will create sudden spikes which lead to the same issue: harmonics.

(In this case denormalization refers to the amplitude/velocity/other changing from exactly 1.0)

You are correct, by limiting the rate of change you limit the harmonic content. The only digital signal with no harmonic is the null signal, a constant level with no change.

[Architeuthis]

For the purposes of this thread can we say that sine waves do not produce perceivable aliasing? I've never heard aliasing due to generating a sine wave... In other words, does anyone ever anti-alias a sine wave in their VST product?

[aciddose]

It wouldn't really make sense to do so as increasing the accuracy of the approximation may be cheaper than computing 2x the signal followed by an anti-aliasing filter.

So in other words oversampling would be more than twice the cost, while decreasing the level of harmonics by improving the sine() approximation might be far less expensive.

Yet, no, we can't ever say additive is alias free or perfect.

We can say that a single sine in isolation most likely won't have any perceptible aliasing, given it was generated with a good enough approximation.

When it comes to computing tons of these together however those harmonics (and aliases) at -40db, -60db or whatever result in what is essentially a noise floor.

Computing a highly accurate sine is just as expensive as computing a highly accurate ramp or pulse or any other waveform using an anti-aliasing filter. At this point subtractive (pcm, wavetable, etc) beats additive in cost and has the same aliasing concerns.