Real-time pitched audio-to-midi technology... behind times?

[Juanjo]

If you don't mind a couple of questions:
- Have you tried FrettedSynth's GCBS4?
- If so, how would you compare its pitch tracking with AlienSolo's?

I just did a quick test so I'll answer my own question: imho AlienSolo's pitch tracking is miles ahead of any audio-to-midi approach, and right in the same league as FrettedSynth's stuff... I marginally prefer GCBS4's tracking though, but a nice alternative nonetheless.

[deastman]

I've had some success using my Nord Modular G2 to track the pitch of an incoming vocal signal and using that to control the frequency of an oscillator. The actual purpose of the patch was to route the oscillator and vocal into a vocoder, so I could have a vocoder which tracks the pitch being sung.

I don't have it in front of me, but basically what I did was:
1. Gate the audio
2. Blast the hell out of it with amplifiers and distortion to turn it into a square wave
3. Filter out everything above and below the fundamentals for the note range I'm singing
4. Send that into the G2's pitch tracking module.
5. (optionally) Quantize the results

I haven't tried sending that back out as MIDI data, but I think it should work fairly well. One caveat is that I have to tweak most of these settings each time to get optimal results.

[Lagrange]

imho AlienSolo's pitch tracking is miles ahead of any audio-to-midi

The fact that it's tracking for monophonic bass would probably mean that its an easier process no? I love AS and use it often to track otherwise impossible basslines..

L

[ouroboros]

AlienSolo does seem effective - never tried the distort to square wave, thought I have had more success limiting the freq ranges. It's funny that the fluid expression I hoped for is so elusive with the conversion, I suppose it makes sense.

kinda OT/
audio to midi has become fools' gold to me. There are some really cool effects that I hope to use it for, but truth be told, the primary utility was to get the songs out of my head and into midi quickly. After frustratingly fiddling for a while, I can only conclude that if I knew how to play an instrument well again, my interest in audio->midi would wane significantly.

I realize that the op was not describing my dilemma, so no critique intended. Just confession. Why? Because it's the interweb and I'm anonymous

[Juanjo]

imho AlienSolo's pitch tracking is miles ahead of any audio-to-midi

The fact that it's tracking for monophonic bass would probably mean that its an easier process no? I love AS and use it often to track otherwise impossible basslines..

L

I wouldn't say easier, but yeah certainly more focused... my ideal a2m software would have different tracking algorithms for different incoming audio signals (i.e. voice, guitar...) and for different controlling purposes (i.e. mono bass, mono lead, pad, wind intrument, poly string instrument, poly piano/keys...). Hey daydreaming is free

@ ouroboros: actually that's more or less my own experience... pretty excited before researching, pretty frustrated and not that interested afterwards

in any case I'm still looking into Bidule (not only for a2m but for a lot of other things, looks very powerful) and trying to set up Usine Free to produce some sound... this one also looks mighty deep and apparently has some a2m facilities as well...

[Stargard]

if you want to sing your instruments (I am a decent singer), nothing beats Digital Ear.
and I tried them all.
the real-time version is fantastic and if you spend some time setting everything up (yes there are MANY options), you will get a.....fantastic result.
by the way, the pitching works up until 12 halftones, so you HAVE to set your synth to that pitchbend range (12 halftones, not 2 like set up in most synths!). this is very important.
just listen to the examples on their digital-ear.com site. awesome.

if you can't sing, but want to whistle your melodies, you can set up a different algo.
the result is perfect.

when it comes to rhytmic material, TS Audio's beat algo is the best.
it works in real-time too and its really, really good for "beat boxing".

the output in Digital Ear is only as good as the input. this is no Autotune software.
this thing is my Holy Grail so far and I haven't heard anything that comes even remotely close to it.

[bermudagold]

this is funny....this is the same discovery and frustration process i went through 2 yrs ago. Yeah i tried alien solo back then right after vokator. It tracks well like fretted synth but ur limited to its synth sound, which i didnt really like. No offense to clone ensemble guy, his plugs are innovative and brilliant. For bass Lagrange, I had better results using delaydots phat which adds artificial harmonics. But unlike waves maxbass,renaissance bass,or newb, allows you to mute the original signal and just leave the harmonics added. Gave me much more realistic and better sounding sung basslines than alien solo. But like juanjo said, the reason frettedsynth and alien solo work so well is because they bypass midi altogether i believe. When i was asking these questions in the threads from 2 yrs ago i linked to, everyone said the limitation was midi, and thats why so many people have been waiting for "OSC" open sound control. But, the industry hasnt pushed it. Keep in mind midi is very old tech.

As far as digital ear, back when i tried it, it either wasnt good or the demo didnt allow you to export a midi file. Has it changed a lot in the last two yrs? I'll have to reinvestigate it. Still sad we are not closer to my dream workflow from when i first thought of this

[bermudagold]

by the way, the pitching works up until 12 halftones, so you HAVE to set your synth to that pitchbend range (12 halftones, not 2 like set up in most synths!). this is very important.

Thanks starguard, i never noticed this. Cant wait to see if this makes a difference. Also, for u melodyne folks, I just noticed that this very thing starguard is pointing out can be set to 24 per note in melodyne. Also, melodyne does have realtime midi send juanjo, but u will have to do it physically using ur interface, or use hubi or midiox for loopback. Some new stuff to try, kinda excited..lol...but i know not to get my hopes up.

By the way, i understand what u mean starguard about input. Even when i first tried this stuff back then i noticed it is really sensitive to which octave u sing in, how you tongue notes(can really mess up velocity interpretation, how u breathe and phrase, whether you hum, whistle, open mouth closed mouth, which vowel u use. Definitely gotta experiment a lot

[Juanjo]

Also, melodyne does have realtime midi send juanjo, but u will have to do it physically using ur interface, or use hubi or midiox for loopback.

mmm yeah, but does it have realtime audio input? I honestly don't know about the standalone versions, but I briefly tested the plugin and it behaves like a sampler i.e. you load wave files, process them to recognize the midi events, and only then you can output midi... of course this workflow surely fits a lot of applications but not the one I had in mind, which is "full realtime" from voice through tracking all the way to midi output into a separate VSTi...
But hey in any case Melodyne is currently way out of reach price-wise for me

[Lagrange]

Yes the plugin version of Melodyne has limited MIDI features unlike the the cre8 or studio versions. Heres a feature comparison chart that show the difs. One thing that caught my in the Studio version.

• handles polyphonic audio

I'm wondering if this is applicable to MIDI output.. AFAIK Melodyne only allows for real time parameter manipulation from external MIDI controller (ie. format, pitch).. Its too bad you could not transcribe MIDI notes directly from audio although as bermudagold said MIDI is the problem right?

in any case Melodyne is currently way out of reach price-wise for me

Tell me about it.
699? Probably be better off buying a dedicated hardware box that would probably do the job much better!

L

[bermudagold]

1)Went to my friend studio and re-tried melodyne, analysing with pitch bend set to the max of 24 halftones. Tried both the percussive/polyphonic and melodic algos at both smooth and crisp settings. I notice some difference between the first two algos, but little to no difference between smooth and crisp. However, setting the pitch bend in the vst instrument to anything above 6 is totally squirrely and doesnt sound good. The more expressive the instrument the higher you can go. But, with sample based vsti, doesnt sound good above maybe 4 as there are just not enough multisamples i guess. This works better with physical models and purely synthesis instruments.

2) I have re-realised that realtime is the thing that makes this work. Because,(especially since each vsti responds differently), it is much easier to adjust the way you are singing to get optimum results than to go back and forth between the analysis app and ur sequencer, tweaking the algo and testing the new midi file over and over in an iterative fashion. Then having to clean up the errors in the piano roll. That is so time consuming and not fun. Thats probably just as much time as taking the general idea of the notes from the first analysis, and knowing the note layout of the piano, learning to play with my fingers what i just sung. Thats what i do now, dont know which method is slower.

3)Since realtime is king i want to retry digital ear. So starguard, how are you using digital ear in realtime with ur host?

[bermudagold]

@Lagrange- Yeah melodyne's converter has a polyphonic algo. And, it does transcribe, It creates a best guess sheet music score at the top of the screen, and on the left of your analysed wave is kind of a tuner which shows how each part of the wave overlaps, blurs, or bends across multiple notes.

For realtime, i'm assuming you would have to record directly into melodyne and use their "realtime midi send" to another app with the vsti either physically through ur interface or virtually through midi yoke or similar.

It can be a rewire slave, but not a master i think. And it has this melodyne bridge vst thing. I dont know if you would be able to pass midi in realtime using either of those as i've never used rewire or the melodyne bridge before, but that seems like a better method than what i suggested above

[Jord Peck]

Hello! New to these forums. My name is Jord and professionally I am a one man band. Been working with a purely acoustic setup the past two years and am trying to bring the dieing art into the 21st century. If you are interested you can find out a little more on that here: http://electro-music.com/forum/viewtopic.php?t=21555

I've read the thread but I've only just begun learning about these electronic possibilities, so much of it sounds like a different language to me! This audio controlled synth dealy is only one of the problems I'm working on to come up with the most versatile and efficient one man band possible but it is also one of the most important! I believe you all are about to save my butt (thanks in advance).

Application: I want to run several mics through my G5 to control several soft synths/samplers with my voice in a live performance environment.

I understand there can be problems with audio interference and I will be playing acoustic drums with my feet and other parts of my body/head of my guitar etc. by way of chords and pulleys hooked up to strange little gadgets I've invented so I suppose that could put me at a disadvantage as the mics will be near them but because I trigger them by way of cords I can distance them somewhat...

I'd like to go the soft synth/sampler rout because they will be easier for me to control with my rig (my hands, feet, elbows and nearly every part of my body are going to be very busy!)and I may need to switch instruments on the individual mics from song to song but if there is a hardware solution (as far as vocal controlled synths)that would be better suited let me know.

I'm trying to find a balance between versatility, efficiency, and reliability and I know I'm probably asking a lot of this limited technology (and of you guys, THANK YOU THANK YOU THANK YOU!!!) but what are my options?

JP

[runagate]

Here's my addition to this debate (this time around, it's been rehashed coninutally for years, often by me, here on KVR as it's still an elusive dream...)

Audio-to-mid could be instantly improved by having a keyboard key or drum pad to strike to tell the software that this is where you'd like a new note-on message to occur. Otherwise you could set a pitchbend glide/resolution setting manually, or even control it live with a modwheel to tell it how choosy to be in real time.

Can someone please implement this and tell me if I'm crazy???

Lol, that's it.

I'll ask Olivier at Usine if his audio-to-midi could be made to be audio-to-OSC as an experiment, to further the cause of development, since Usine uses OSC already anyways.

- runagate

[Lagrange]

Audio-to-mid could be instantly improved by having a keyboard key or drum pad to strike to tell the software that this is where you'd like a new note-on message to occur. Otherwise you could set a pitchbend glide/resolution setting manually, or even control it live with a modwheel to tell it how choosy to be in real time.

Man it sounds like a solution brought from frustration with poor detection in the 1st place. So I'd have to say with all the advancements in spectral FFT processing can't a learn finger print feature be added to the start of detection (maybe stolen from noise detection)? Runagate, think about it, it really shouldnt be so hard for an algo to detect the difference between silence and the start of a sound that triggers a NOTE ON event! I think no one has really looked at this hard enough for real change to come to the technology..

At any rate I've gotten better result by breaking down the signal into partials and odd or even harmonics before running thru the Audio to MIDI ap. Obviously this is not possible live. The closest I've come live is to use..


sp3ctr3

You just have to create a bandpass with the LP HP freqs and mess with the Harmonic Threshold and Edges knobs..

Give it a try, you may be surprised by the outcome..

L