How to mimic acoustic instruments with synthesizers - a cry for help!

[pekadan]

Hi guys,

Please help me out with some tips and suggestions! For a university course in sound synthesis, I'm trying to come up with a good and interesting way to mimic acoustic instruments using some method of synthesis.

I started out with a pure subtractive approach, trying to match the timbre of an instrument with a suitable analog oscillator, then applying filters to reduce harmonics that don't fit the timbre, and obviously using envelopes and LFO to modulate amplitude and filter parameters to get the right sort of "movement". Works ok for some instruments, but I think it's quite an uninteresting old way of doing it (every analog synth since the 70's has had patches mimicking acoustic instruments in some similar ways).

My next approach was to use subtractive synthesis, but with a synth (Melda MPowerSynth) that offers additive oscillators. I generated FFT-diagrams of acoustic sounds by playing sampled instruments through an FFT-analyser, and then copied the sound by manually drawing the harmonics in the oscillator editor of the synth. This worked ok, but I couldn't really get the timbres to sound convincing, and the change in frequency over time still had to be simulated with filters.

My last approach was to use an additive synthesizer with morphing capabilities (experimented with Camel Audio Alchemy and discoDSP Vertigo). My idea was to get the frequency changes better simulated by morphing the sound from the attack stage to the loop- or sustain stage. I analysed each acoustic instrument as described above, but now I took two "samples" from the FFT-analyser - one of the attack and one at the time of loop or sustain. I drew the attack harmonics in the editor for oscillator A and the loop/sustain harmonics in the editor for B. Then I used an envelope to create the movement from A to B and combined it with a suitable amplitude envelope (plus LFO for vibrato), hoping this would give a somewhat convincing sound of playing a note of the acoustic instrument.

The result was terribly disappointing though. I couldn't get the timbres of the additive oscillators to sound right by copying the harmonics (I drew each n-th harmonic, up to about the 16th overtone, with a strength relative to the other harmonics) and the morphing would typically create some weird "mix" that didn't sound at all like the normal acoustic transition of the sound when playing a note.

To be clear, my objective was not to try to simulate any acoustic instrument in perfect detail. I would be satisfied if one could hear (or at least guess) what acoustic instrument my patch tried to mimic in the first place. But my results were patches that could all be named "synthetic nightmare", with no resemblence at all to any acoustic instruments.

What am I doing wrong? Are my understandings or ideas all wrong, or am I just not going about it correctly? Any ideas of how to better accomplish what I'm trying to do? All input is greatly appreciated! (I'm already late handing in my report).

[BertKoor]

FFT analysis is great for sustained notes, but imho fails at capturing the properties of the attack transient. Your research so far comes to the same conclusion. Find out why it fails. I think it is because the transient part is too complex, it's spectrum changes too fast over time.

Why not just use samples of those? That's what most ROMplers and workstation synths do

I'm already late handing in my report

Why not just be honest and report the results of your attempts so far?

I have not failed. I've just found 10,000 ways that won't work.

[fmr]

The attack transients of many (I would say the majority) of acoustic instruments have a strong noise component.

So, you have many inharmonic transients, and I mean MANY. 16 overtones is way too short, and we can't stick with harmonics (as i said, ther vast majority are inharmonic partials).

I think a good read on this will be the Jean Claude Risset "An Introductory Catalogue of Computer Synthesized Sounds". You can hear examples of the instruments here: https://www.youtube.com/watch?v=zXDaBA3aggI


I also advise you to study the "Amsterdam Catalog of Csound Computer Instruments" - http://www.codemist.co.uk/AmsterdamCatalog/

Another book that helped me a lot was " A Synthesist's Guide to Acoustic Instrruments" by Howard Massey.

[fluffy_little_something]

This should probably be in the sound design forum...

Anyway, depends on which instruments you are talking about. Some are more difficult to emulate than others, obviously.

[pekadan]

FFT analysis is great for sustained notes, but imho fails at capturing the properties of the attack transient. Your research so far comes to the same conclusion. Find out why it fails. I think it is because the transient part is too complex, it's spectrum changes too fast over time.

Why not just use samples of those? That's what most ROMplers and workstation synths do

Thanks!

That makes perfect sense and would be quite an easy thing to accomplish with a synth like e.g. Alchemy (I also need to produce a piece of music using patches I've created). For the purpose of this task, I think I can even generate the attack sound using whatever analog waveform works best and write in the report about the use of samples for the attack part.

I'm already late handing in my report

Why not just be honest and report the results of your attempts so far?

I have not failed. I've just found 10,000 ways that won't work.

You're right. Very good suggestion! And what a very suitable quote too, speaking of complex transients!

[pekadan]

The attack transients of many (I would say the majority) of acoustic instruments have a strong noise component.

So, you have many inharmonic transients, and I mean MANY. 16 overtones is way too short, and we can't stick with harmonics (as i said, ther vast majority are inharmonic partials).

I think a good read on this will be the Jean Claude Risset "An Introductory Catalogue of Computer Synthesized Sounds". ...

I also advise you to study the "Amsterdam Catalog of Csound Computer Instruments" - http://www.codemist.co.uk/AmsterdamCatalog/

Another book that helped me a lot was " A Synthesist's Guide to Acoustic Instrruments" by Howard Massey.

You da man fmr! I suppose inharmonic contents is a reason one treats noise separately from sine waves in additive synthesis. I'm a lot wiser already, but I'm surely going to check out the books you recommend. I hope I can find them online and start reading already today. Otherwise amazon, here I come.

[pekadan]

This should probably be in the sound design forum...

Ah, true, should I maybe ask a mod to move it?

[himalaya]

This is a deep and complex subject, and since you are late with your report, I fear not much can be done to help.

Subtractive synthesis can be a very good source of acoustic sounds if coupled with a flexible, modular signal path that offers wave-shaping, comb + formant filters. For example, you can create a very nice 'nylon' quitar pluck with a pulse waveform, comb filter and a LPF with a suitable filter envelope shape (to create a 'pluck' response).

Then, using a triangle wave through a waveshaper (with intensity assigned to a controller) into a comb and a LPF can give all manner of flute sounds. However, for this to be believable, you would need to have a separate source of noise going into one, or better, several comb filters (plus LPF or BPF) in order to simulate the breath noise. White/pink noise through a LPF or BPF alone is not enough.

Moving on from a modular subtractive engine, I'd recommend FM, above additive. I found FM to be much quicker to use and quicker to obtain great harmonic control. In fact, I have been making some acoustic sounds in a synth just in the past few days, an engine which can be described as 'analog subtractive' but one which also has very flexible FM capabilities. Here, I was able to get the basic tone just with the subtractive part of the synth, only adding overtones and more 'body' via FM. The FM is really only between 2 oscillators. The result is what sounds like a mix between a flute and a duduk or some other woodwind instrument. An imaginary acoustic sound, but to me it definitely sounds 'real'. I've uploaded a quick preview for you to have a listen:

https://soundcloud.com/electric-himalay ... ind-thingy

[pekadan]

This is a deep and complex subject, and since you are late with your report, I fear not much can be done to help.

Subtractive synthesis can be a very good source of acoustic sounds if coupled with a flexible, modular signal path that offers wave-shaping, comb + formant filters. For example, you can create a very nice 'nylon' quitar pluck with a pulse waveform, comb filter and a LPF with a suitable filter envelope shape (to create a 'pluck' response).

Then, using a triangle wave through a waveshaper (with intensity assigned to a controller) into a comb and a LPF can give all manner of flute sounds. However, for this to be believable, you would need to have a separate source of noise going into one, or better, several comb filters (plus LPF or BPF) in order to simulate the breath noise. White/pink noise through a LPF or BPF alone is not enough.

Moving on from a modular subtractive engine, I'd recommend FM, above additive. I found FM to be much quicker to use and quicker to obtain great harmonic control. In fact, I have been making some acoustic sounds in a synth just in the past few days, an engine which can be described as 'analog subtractive' but one which also has very flexible FM capabilities. Here, I was able to get the basic tone just with the subtractive part of the synth, only adding overtones and more 'body' via FM. The FM is really only between 2 oscillators. The result is what sounds like a mix between a flute and a duduk or some other woodwind instrument. An imaginary acoustic sound, but to me it definitely sounds 'real'. I've uploaded a quick preview for you to have a listen:

https://soundcloud.com/electric-himalay ... ind-thingy

Fantastic stuff! Thanks a lot for the input!

Even if it's a little late for me to "go back to the old drawing board", I'm very interested in learning more for the future. Where do you pick this sort of knowledge up? Learning by doing or can you recommend some resources?

[phreaque]

one of innovative tricks to emulate acoustic instruments, is to implement modulation with very short delays.
Try using a phaser with big amount of feedback and no lfo cycling.
You can use also a multi delay, not synced to BPM with very short times (like 9, 14, 26 ms)
Such concepts are essential to give the sound its organic, environmental characteristics.

[himalaya]

Where do you pick this sort of knowledge up? Learning by doing or can you recommend some resources?

You can pick up ideas from various sources, the Sound on Sound magazine run a series of articles a long time ago, which also touched upon imitative synthesis, however, at least in my case, it's mostly about exploration, perseverance and just sheer stubbornness ( as in, 'I will get this to sound like a flute', hehe).

It's also weird how it goes. For example, I know how to make a flute, or an electric piano, or brass etc, but sometimes sitting with a new synth it just does not happen. I will go through the same motions as I've always done, and it will not give me the sound I'm after. So it's here where deep familiarity with a synth pays off. If you choose a flexible synth and stay with it, you will be rewarded with great sounds.

It's about knowing the sound of your synth, knowing how all the various waveforms sound when modulated by different filters, when hard-sync is applied, when FM is added....you just need to have a mental memory map of how all of this sounds. So, when you hear an acoustic sound, say a clarinet, you will have an inkling of what waveform to start with: Would it be a sawtooth, a triangle or perhaps a square/pulse? This is crucial. Otherwise you may spend a lot of time meandering aimlessly looking at various synthesis engines and never get anywhere.

Having said that, there are synthesis techniques which are not easy to learn the sound of. For example, FM. Of course, we can learn how two operators sound at various ratios, mod intensities, etc. But when it comes to more complex algorithms it is really difficult to know exactly how a specific combination of operators will sound. So, there is also a need for exploration, just simply allowing yourself to spend a lot of time searching for various 'timbres'. And as you do so, you need to learn to save your half-edited sounds when you stumble upon some interesting timbre. For example, say you were looking for a flute sound, but in the process you have actually managed to get something that sounds like a clarinet, then what you need to do is, save it, naming it accordingly so that you can come back to it later, and develop it fully into a clarinet. Happy accidents do happen. Pay attention to them.

Then, when you want to emulate a sound you need to listen, and listen very carefully. For example, if I were to emulate a flute, I'd choose a suitable music track and copy it into my DAW. Then, I'd select a note which is long enough to be sustained and then loop it in the DAW. This single note would be my point of reference in order so that I can start building something similar in my chosen synth. It's crucial to get this right. Just simply making a sound from memory is not enough. That is, we know how a flute sounds, but doing it from memory may lead to the wrong result. Often, it's one little detail that can make or break the sound. For example, in the above audio demo, the comb filter is tuned very carefully and every tiny change of the frequency alters the noise too much, and it becomes too artificial, it looses it's 'breathiness', and to be honest, that 'breath' noise is still not right, so I need to go back and try to improve it.

So that's why we need a solid point of reference at all times during the synthesis sessions.

Overtime, you will discover tricks and shortcuts which may be specific to the synth you have chosen to work with. And of course, do try to read about synthesis as much as you can. Definitely try to source the Sound On Sound Synth Secrets articles.

[pekadan]

one of innovative tricks to emulate acoustic instruments, is to implement modulation with very short delays.
Try using a phaser with big amount of feedback and no lfo cycling.
You can use also a multi delay, not synced to BPM with very short times (like 9, 14, 26 ms)
Such concepts are essential to give the sound its organic, environmental characteristics.

Interesting! I'll give this a try for sure. Thanks!

[donkey tugger]

And of course, do try to read about synthesis as much as you can. Definitely try to source the Sound On Sound Synth Secrets articles.

Blimey, that takes me back. I remember those being really good the first time round, and I'm not into synthesisers..

I had a look and they've got a lot of them uploaded here now;

https://www.soundonsound.com/search/art ... ion%3A6971

[resynthesis]

Nobody mentioned physical modelling yet? I'm sure if you followed up on @fmr's suggestion you'd come across PM material pretty quickly as Risset used and wrote about it pretty extensively I believe

[pekadan]

Where do you pick this sort of knowledge up? Learning by doing or can you recommend some resources?

You can pick up ideas from various sources, the Sound on Sound magazine run a series of articles a long time ago, which also touched upon imitative synthesis, however, at least in my case, it's mostly about exploration, perseverance and just sheer stubbornness ( as in, 'I will get this to sound like a flute', hehe).

It's also weird how it goes. For example, I know how to make a flute, or an electric piano, or brass etc, but sometimes sitting with a new synth it just does not happen. I will go through the same motions as I've always done, and it will not give me the sound I'm after. So it's here where deep familiarity with a synth pays off. If you choose a flexible synth and stay with it, you will be rewarded with great sounds.

It's about knowing the sound of your synth, knowing how all the various waveforms sound when modulated by different filters, when hard-sync is applied, when FM is added....you just need to have a mental memory map of how all of this sounds. So, when you hear an acoustic sound, say a clarinet, you will have an inkling of what waveform to start with: Would it be a sawtooth, a triangle or perhaps a square/pulse? This is crucial. Otherwise you may spend a lot of time meandering aimlessly looking at various synthesis engines and never get anywhere.

Having said that, there are synthesis techniques which are not easy to learn the sound of. For example, FM. Of course, we can learn how two operators sound at various ratios, mod intensities, etc. But when it comes to more complex algorithms it is really difficult to know exactly how a specific combination of operators will sound. So, there is also a need for exploration, just simply allowing yourself to spend a lot of time searching for various 'timbres'. And as you do so, you need to learn to save your half-edited sounds when you stumble upon some interesting timbre. For example, say you were looking for a flute sound, but in the process you have actually managed to get something that sounds like a clarinet, then what you need to do is, save it, naming it accordingly so that you can come back to it later, and develop it fully into a clarinet. Happy accidents do happen. Pay attention to them.

Then, when you want to emulate a sound you need to listen, and listen very carefully. For example, if I were to emulate a flute, I'd choose a suitable music track and copy it into my DAW. Then, I'd select a note which is long enough to be sustained and then loop it in the DAW. This single note would be my point of reference in order so that I can start building something similar in my chosen synth. It's crucial to get this right. Just simply making a sound from memory is not enough. That is, we know how a flute sounds, but doing it from memory may lead to the wrong result. Often, it's one little detail that can make or break the sound. For example, in the above audio demo, the comb filter is tuned very carefully and every tiny change of the frequency alters the noise too much, and it becomes too artificial, it looses it's 'breathiness', and to be honest, that 'breath' noise is still not right, so I need to go back and try to improve it.

So that's why we need a solid point of reference at all times during the synthesis sessions.

Overtime, you will discover tricks and shortcuts which may be specific to the synth you have chosen to work with. And of course, do try to read about synthesis as much as you can. Definitely try to source the Sound On Sound Synth Secrets articles.

Wow, that's very interesting and useful info! It takes time and effort to get good at this, that's for sure, but I find your suggestions both inspiring and motivating. Thanks a million!

The sound you demoed previously is amazing. Not only does it sound like a real acoustic instrument as far as timbre goes, but the way you're able to play it too, with articulation, sounds very "real". Incredible. My first thought was to ask you what synth you've used to get a sound like this, but reading your last post I realize that's not a very relevant question. (but when you release an expansion pack of patches for that synth that includes sounds like this gorgeous "flutuk", that's when I have to know, ok!? ).

I'm curious to know what your approach to the attack of your woodwind instrument was. As previously said, it's often difficult to mimic the attack of the acoustic instrument with its complex inharmonic transients. Did you put special effort into getting the attack part to sound realistic? If so, will you share some thoughts?

Now I'm off to start browsing the Sound on Sound articles about synthesis. Sounds like a goldmine!

Again, thanks!