Real-time pitched audio-to-midi technology... behind times?

[Juanjo]

GCBS4 is the newest from my audio triggered synths, all the rest (string sculptor, tripp lead and phazosc) are in need of an update.
Hope to finish soon, then building some new stuff with the new Chris Kerry osc's, that should be some fun synths??

The man himself to the rescue
Are those CK oscs the ones you used in SaffronVA? They DO sound good...

[MaliceX]

What about that Vielklang VST thing? I do know that thing uses audio pitch tracking to do what it does; harmonize. But I was thinking, maybe it's also capable of pitch recognition without adding stuff..

[Juanjo]

What about that Vielklang VST thing? I do know that thing uses audio pitch tracking to do what it does; harmonize. But I was thinking, maybe it's also capable of pitch recognition without adding stuff..

It does audio-to-midi but not real time, it doesn't have audio input... it acts as an instrument where you have to load the audio files to be treated, similar to melodyne in that sense...

[FrettedSynth]

GCBS4 is the newest from my audio triggered synths, all the rest (string sculptor, tripp lead and phazosc) are in need of an update.
Hope to finish soon, then building some new stuff with the new Chris Kerry osc's, that should be some fun synths??

The man himself to the rescue
Are those CK oscs the ones you used in SaffronVA? They DO sound good...

Yes the osc's used in SafFronVA are by CK he did a great job.
Can't wait to use the morphing osc in a audio triggered synths Plus there are others, the hard sync osc should be fun to play with a guitar synth?

Fretted Synth

[Lagrange]

BTW, the fact that GCBS (:love: 3) does such a good job detecting notes, does this mean I can use MIDI out from one of the synths to trigger something else? If not (will try it in a bit)...

...all the rest (string sculptor, tripp lead and phazosc) are in need of an update.

Can you please add MIDI out to the updates?

Thanks

@Juanjo, I was taking a closer look at some recorded MIDI after detection and noticed when I zoomed way out low and behold it resembled specrtal oscilloscope analysis. It seems that the if the conversion is being made somewhat direct from Hz to notes that filtering and gating could theroretically be applyed the same way. Basically I tried (after the fact)filtering notes like I would audio and so far it seems to work almost like audio filtering. Anyhow what came to mind was, maybe by looking at it from this point of view there is a way to set up maybe several mics that would correspond to a specific filter setup. I'm going to do more testing and see what happens.

I will post some pics to show what I mean in a bit..

L

[Juanjo]

Lagrange, that's interesting but I'm not sure if I'm following you all the way... are you talking about plain filtering/gating + FFT filtering/gating + midi note filtering/gating? That would be a pita to set up for different environments but it might give decent results... keep us posted...

Just for the sake of completeness, I also tried the Digital Ear standalone software mentioned in this thread:
http://www.kvraudio.com/forum/viewtopic ... highlight=

Didn't do it for me either, but ymmv...

...all the rest (string sculptor, tripp lead and phazosc) are in need of an update.

Can you please add MIDI out to the updates?

I actually asked him through PM, apparently MIDI quantisation is a problem so that's why he went for direct pitch/envelope tracking for his synths... but hey, I also mentioned the option of doing an audio-controled sampler (SF2?) and apparently this might be a distinct possibility in the future

Now imagine this workflow... since Highlife is free and can autosample very reliably and efficiently, you could just autosample ANY patch from ANY synth/sampler and save it as SF2, ready to be loaded in the (vaporware as of now, I know ) audio-controlled SF2 player... if that player had the responsiveness of, say, GCBS4 that would be a good enough option for a lot of my needs...

[bermudagold]

thats basically what i tried to do with vokator juanjo. I'd sample a patch i wanted, load the sample in the sampler of vokator, and reverse vocode sing it. But, i dont know if it was the nature of FFT , or just NI implementation, but even with max bands in vokator, you would totally lose the characteristics of the patch you sampled. Would be cool if frettedsynth could achieve what ur suggesting without the bad sound of the vocoder method. I too dont know why more developers havent attacked this. Since so many instruments are hard to play realistically on a keyboard without their natural interface, you think more folks would be interested. Still seems to me like the most natural and expressive way to play virtual instruments,....but like the threads i showed you are from like two years ago when i was searching and we havent really advanced since. For what its worth vielklang actually does a good job with its midi output sometimes better than melodyne. But its not real time and doesnt create a midi file so you have to waste a track and cpu clock cycles routing the midi to ur instrument track

[Juanjo]

thats basically what i tried to do with vokator juanjo. I'd sample a patch i wanted, load the sample in the sampler of vokator, and reverse vocode sing it. But, i dont know if it was the nature of FFT , or just NI implementation, but even with max bands in vokator, you would totally lose the characteristics of the patch you sampled. Would be cool if frettedsynth could achieve what ur suggesting without the bad sound of the vocoder method. I too dont know why more developers havent attacked this. Since so many instruments are hard to play realistically on a keyboard without their natural interface, you think more folks would be interested. Still seems to me like the most natural and expressive way to play virtual instruments,....but like the threads i showed you are from like two years ago when i was searching and we havent really advanced since. For what its worth vielklang actually does a good job with its midi output sometimes better than melodyne. But its not real time and doesnt create a midi file so you have to waste a track and cpu clock cycles routing the midi to ur instrument track

Well I haven't used vokator but if I'm not mistaken it fft-modulates the loaded samples with the input signal like a vocoder, no? that would be the reason why the sounds didn't come out clean, that's expected behavior... what I'm talking here is different: take for example GCBS4 from frettedsynth and simply replace its oscillator with a sf2 player... the input signal would just control the sound through pitch-tracking and envelope-tracking, but it shouldn't fft-modulate it in any way... of course that's the broad theory and I'm sure that it's not so easy in practice but right now I cannot see why not, after all a SF2 player is just another kind of oscillator, right?

[Juanjo]

Hey, any Bidule owners reading this?
I just read in Bidule's online help that it offers a "Spectral to MIDI" module, but the current beta has expired so I can't demo it until they release a new version. Does anybody have any experience with this module? Are there any interesting audio-to-midi schematics available for Bidule?
I've been all week on the verge of stretching my budget beyond recognition to buy it but I'm hesitant because I haven't been able to demo it... but decent audio-to-midi would seal the deal for me.

[bermudagold]

i would love that ...the fretted synth sampler that is....but let us know if you get good results with bidule

[highkoo]

ive used 'TS Audio to Midi' just to get a few simple notes- by singing into the PC mic. ive only used it for the most basic possible thing, and i dont think it does anything realtime, but i wasnt disappointed with what it did.

[Lagrange]

Now imagine this workflow... since Highlife is free and can autosample very reliably and efficiently, you could just autosample ANY patch from ANY synth/sampler and save it as SF2, ready to be loaded in the (vaporware as of now, I know ) audio-controlled SF2 player... if that player had the responsiveness of, say, GCBS4 that would be a good enough option for a lot of my needs...

Very true but isnt that the same as having a MIDI out without the synth in the middle?

Hey, any Bidule owners reading this?

Wheres shamann when you need him? Not a seasoned vet but I know it can work wonders in this dept. because of the routing capabilities! Would still probably be pita to setup for individual voices/sounds/instruments. At the end of the day you can just save out the project as a template right?

ive used 'TS Audio to Midi' just to get a few simple notes- by singing into the PC mic. ive only used it for the most basic possible thing, and i dont think it does anything realtime, but i wasnt disappointed with what it did.

Listen guys all of these old apps mentioned are so win98 it's ridiculous to expect there going to give reliable A2M. I agree with the OP, looks like the tech is behind the times.

L

[Juanjo]

Now imagine this workflow... since Highlife is free and can autosample very reliably and efficiently, you could just autosample ANY patch from ANY synth/sampler and save it as SF2, ready to be loaded in the (vaporware as of now, I know ) audio-controlled SF2 player... if that player had the responsiveness of, say, GCBS4 that would be a good enough option for a lot of my needs...

Very true but isnt that the same as having a MIDI out without the synth in the middle?

Well, apparently not... from what I've gathered so far MIDI has some problems such as:

- Quantisation... MIDI parameters (CCs, velocity, etc) are usually 7bit and thus the range of possible values is only 128... this introduces stepping in the midi signal which oftentimes is audible

- When to trigger a new note and when to apply pitchbend? imho this is one of the things that frettedsynth's synths do waaaay better than any audio-to-midi: since both his pitch-tracking and envelope-tracking are so accurate his synths do pitchbends, glides, vibratos, etc so naturally that it's scary, even across a full octave or more... now try to do a fast one-octave glide with your voice through any audio-to-midi software: if you are lucky you'll just get several notes in a row incrasing/decreasing in pitch, not a single one pitchbending a full octave up or down...

I don't know, I'm sure there are even more technical reasons to bypass midi altogether... to me the audio-triggered sampler option looks more and more like the way to go for controlling audio with the human voice, perhaps it's different for a guitar since its output is more predictable than the voice...

[CinningBao]

I'm a very happy Bidule user, and I spent a long time looking for Audio-To-Midi tools - Bidule was my answer.
Along with the Spectral2Midi bidule there is also Spectral2LoudestFreq/Amp which, in combination with threshold driven MidiNoteCreators and few filters, can be tuned to Midify various audio signals.

If only I could convince the AlienSolo man to provide a MidiOut..

[Juanjo]

I'm a very happy Bidule user, and I spent a long time looking for Audio-To-Midi tools - Bidule was my answer.
Along with the Spectral2Midi bidule there is also Spectral2LoudestFreq/Amp which, in combination with threshold driven MidiNoteCreators and few filters, can be tuned to Midify various audio signals.

That sounds very low level but very interesting as well, thanks! I think there's some plogue in my future

If only I could convince the AlienSolo man to provide a MidiOut..

Hey this is the audio-controlled synth by Clone Ensemble, right? Yeah his harmonisator is awesome as well, I'll check out the AlienSolo demo, although I'm afraid that the reason why it might track pitch so well is BECAUSE he's bypassing midi altogether...
If you don't mind a couple of questions:
- Have you tried FrettedSynth's GCBS4?
- If so, how would you compare its pitch tracking with AlienSolo's?