VST Midi humanizer plugin

[10bd01]

From the way I read your initial statements, it seemed to me that you were arguing this exact thing. You did state that "All of those inconsistencies are created through the intention of the player, and their intention is not to play random." I conflated that to say players are intentionally and consciously putting inconsistencies into their performance to derive a certain effect. I also inferred this based on your statement "It's unquantized according to how they express a specific music in a particular way". To me, the words "intention" and "express" imply a conscious effort. My argument was that those inconsistencies are not conscious nor are they are intentional but a result of many complex factors such as the distance from other performers, tiredness/alertness, even the distances between keys on a piano, or strings on a guitar.

To clarify, my meaning was that the inconsistencies - though not intentional or conscious themselves - are the results of intentional action and happen within that context. Is "through" the perfect word for this? Probably not, so I can understand you reading it that way. The inconsistencies IMO are effects of the intention and circumstance, neither of which are random. The fact that they're the result of - or happen in the context of - intentional action doesn't make them intentional themselves but the intention does influence how they unfold and what form they take.

I'm glad you're finding value in the randomization. For me, I'd rather make those adjustments more intentionally, not so the results sound intentional, but to ensure they sound natural and not random. Meaning, spending the time manually editing rather than letting a plugin do it. Unfortunately this means more time.

[Ah_Dziz]

The jsplug one is nice. There still isn't a plugin out there that has all the functionality I'd like. I've made versions in reaktor and plogue that do everything I want. If you just need a bit of simple time and velocity randomness, then there are lots of things for it.

[courtjestr]

I'm glad you're finding value in the randomization. For me, I'd rather make those adjustments more intentionally, not so the results sound intentional, but to ensure they sound natural and not random. Meaning, spending the time manually editing rather than letting a plugin do it. Unfortunately this means more time.

Interesting. I am not even sure how I would do that in a way that would lead to a more natural sound. Just manually manipulating start and stop points, velocities, pitches and timbres does not seem to me any more effective than introducing randomization. Just because a human does it does not make it any more human, unless the human through their manual manipulations is able to emulate the natural processes that go on when a human plays an instrument. If you can do that, then I doff my hat to you.

One interesting method I read about recently involves a person playing the rhythm of a passage without worrying about the actual notes and then laying the melody or chords on top of the recorded passage, keeping the velocities and the timing but adjusting the pitch. I could see that being effective though it still would probably not be accurate to a human playing the actual instrument. True humanization of course results from humans actually playing the instruments. Very few of us are virtuoso enough in all the instruments we might want to include in a song nor do we have the funds to hire musicians and so we need to resort to some sort of substitute, poor as it may be.

[markmann]

There are two things going on when humans play music. One is that even the best ones can't play exactly on the beat, so there will be tiny unintentional variations from a click track even if they try to play exactly in time with it. The other thing is that musicians intentionally introduce variations from some ideal click as 10bd01 is saying.

If you have played in live groups that don't use a click track, you will be familiar with this. Also, great orchestras and great conductors explicitly vary tempos. They explicitly push and pull tempos in certain areas of a musical piece to achieve certain musical effects. It can be like calculus - musicians understand how to play with the change in the rate of change in very complex ways. They often discuss it in terms of "feel" and they sometimes do it without being conscious they are doing it because it feels natural in certain musical situations. But even though they might not be making a conscious effort to do it, it is still intentional.

Look at DAW recordings of great drummers who are playing to a click track. You can see them making things "breath" by pushing or pulling certain parts of a song. They might be slightly behind during the intro, then very tight to the clock in the verse, then very much ahead while playing a fill, and then pushing consistently ahead during a solo. A great drummer can play to a click and make the whole band groove organically because of this.

Also, try taking a recording of a good band playing without a click track and adjust the DAW tempo throughout so the DAW click is in sync with the recording. You will see very interesting patterns of tempo change that correlate with musical changes. That's a macro level of intent. But I believe this kind of intent is also happening at a micro level even within the space of one or two beats. When you get a great band that has played together a long time, all the players start to feel and play these intentional variations in complementary ways. That's why different bands from the pre-click era groove differently. This intentional variation is a big part of what creates a unique band groove identity.

[courtjestr]

@markmann, I understand all of this and and agree with all of this. However, the majority of humanization plugins I have seen are only trying to address your first point, the inability for humans to play exactly in time. For that, they resort to randomness within certain constraints. This is to remove the robotic, static feeling the comes from everything being quantized perfectly, and the same phrase being repeated exactly, ad infinitum. I would not expect to use the plugins to produce music that sounds like musicians who have played together for years or an orchestra lead by a virtuoso conductor.

[10bd01]

There are two things going on when humans play music.
...
This intentional variation is a big part of what creates a unique band groove identity.

This post was a great description of what IMO a humanization plug would be aiming for. And if we REALLY want to get into the nitty gritty you'd/it'd be able to incorporate fingerings and kit setups. Human musicians are likely prone to inconsistencies based on fingering and the specific difficulties and physical challenges that different fingerings present. Professionals have worked on removing these kinks but there's likely certain fingerings that have system-wide inconsistency effects of a specific type that could be replicated with adequate measurement and implementation. Kits as well - I'm sure the positioning of the kit produces certain inconsistencies or physical challenges that are reflected in the players playing in a somewhat systemic manner.

I don't really think "randomization" should really be a part of any humanization plugin. In an ideal scenario you have the types of unquantized performance that markmann described - the overarching intention that pushes the entire ensemble in line with, against, before, after the beat in both macro and micro forms. "Randomization" is essentially a mistake, and those are the types of things that often would be fixed in some part of post-production anyway. Maybe there could be a "randomization" knob if you want to introduce what would amount to performance "noise" or mistakes, but IMO that's all it would be, and isn't necessary in a humanization plugin unless just to produce "performance noise".

The humanization plugin IMO would produce a specific type of human performance(s) without mistakes - meaning, without randomization but with a nuanced replication of the effects that result from the variables of real human performance. I think it would be helpful for it to know whether or not the music is in a major or minor key, fast or slow, with this beat or that and with that information pull a style template from a database of human musical performance expressions (?) that match it's qualities and which are based off of data pulled from live midi performance data. So it would hear the music was hard rock and pull from it's database of hard rock live performance midi data and extrapolate an appropriate application of humanizaiton through the AI/Machine Learner. Or something like that. IMO it would have to account for style in some way in order for the specific type of humanization it produces to appropriately match the musical expression, like humans would. I don't think the Rolling Stones are "imperfectly human" in the same way that John Tesh is, and unless you wanted John Tesh performing your rock and roll album these things would need to be fleshed out.

But it's all there - money on the table. Don't have a drummer? Hire Sugarfoot to perform your next beats. Looking for a violinist for your next film score? Hire the virtual Joshua Bell to perform it - based on 1,000 hours of live performance as interpreted by AI/machine learning, bringing your midi to life. Etc. etc. Or buy the basic version and get three versions each of: Rock Drummer, Rock Bassist, Pop Keyboardist, etc. etc.

Everybody's a-sleeping on it, though.

[markmann]

@markmann, I understand all of this and and agree with all of this. However, the majority of humanization plugins I have seen are only trying to address your first point, the inability for humans to play exactly in time. For that, they resort to randomness within certain constraints. This is to remove the robotic, static feeling the comes from everything being quantized perfectly, and the same phrase being repeated exactly, ad infinitum. I would not expect to use the plugins to produce music that sounds like musicians who have played together for years or an orchestra lead by a virtuoso conductor.

I agree that those randomization plugins can remove some of the robotic feeling and I sometimes use them. Rast's Naturalizer plugin can be helpful doing this. But usually when something sounds robotic and I apply "humanization" plugins (like the built-in function in Studio One), the result doesn't feel musical. I often just go back to the robotic track I think what we need is more sophisticated humanization algorithms based on AI analysis of real musicians playing certain types of feels. The Toontrack EZdrummer and EZkeys browser patterns are a step in this direction. They are unquantized files of great musicians playing to a click track.

Until we have more complex humanization plugins, I think the most musical way to reduce robotic feeling is to program in micro changes that feel "right" as 10bd01 has suggested. It's very tedious but does help.

Also, programming slight tempo changes can really help. Sometimes I'll start pushing the tempo up by .5 bpm per measure in the last 3 bars of a verse approaching a chorus, which then plays at 2 bpm faster than the verse, etc. Even in the early days of click tracks and electronic music the great musicians understood this trick. Peter Gabriel did this kind of thing quite a bit. I believe his tune Shock The Monkey has clearly visible tempo changes for the different parts of a song if you try to sync a click with it.

[10bd01]

the inability for humans to play exactly in time.

The point is - they are not playing exactly in time for a reason, it's not "random". There's a reason for it, markmann articulated the reason very well. I don't think humans even want to play exactly in time, so it's a weird thing to consider - humanization is naturally not-in-time, randomization is artificially not-in-time. IMO the solution you are using is just another version of a robotic static feeling - it's not an "exact" robotic static feeling, it's a "dispersed" robotic static feeling. It's almost like smearing or blurring the beat. It just sounds less "static and robotic" because it's less clear compared to it's counterpart - the robotic static quantized feeling. Neither of these has anything to do with human performance whatsoever. So to me it's answering the problem with the problem - let's make music sound less robotic; okay, let's use a random number generator (a robot) to do it!

[markmann]

The humanization plugin IMO would produce a specific type of human performance(s) without mistakes - meaning, without randomization but with a nuanced replication of the effects that result from the variables of real human performance. I think it would be helpful for it to know whether or not the music is in a major or minor key, fast or slow, with this beat or that and with that information pull a style template from a database of human musical performance expressions (?) that match it's qualities and which are based off of data pulled from live midi performance data. So it would hear the music was hard rock and pull from it's database of hard rock live performance midi data and extrapolate an appropriate application of humanizaiton through the AI/Machine Learner. Or something like that. IMO it would have to account for style in some way in order for the specific type of humanization it produces to appropriately match the musical expression, like humans would. I don't think the Rolling Stones are "imperfectly human" in the same way that John Tesh is, and unless you wanted John Tesh performing your rock and roll album these things would need to be fleshed out.

But it's all there - money on the table. Don't have a drummer? Hire Sugarfoot to perform your next beats. Looking for a violinist for your next film score? Hire the virtual Joshua Bell to perform it - based on 1,000 hours of live performance as interpreted by AI/machine learning, bringing your midi to life. Etc. etc. Or buy the basic version and get three versions each of: Rock Drummer, Rock Bassist, Pop Keyboardist, etc. etc.

Everybody's a-sleeping on it, though.

Yes. You're describing exactly what I would like to see. And yes, everybody is sleeping on it, but I think it's coming. In the meantime, I sometimes play a half-assed tabletop drum part into the DAW and then adjust the click to my human performance. Then I'll manually play some other instrument parts. I'm not sure I need a humanization plugin if I already have a human "feel" algorithm running in my brain

And this gets into another interesting question: do people who grew up listening mainly to click-based music hear human-generated intentional tempo variations as "wrong" or unpleasant?

[10bd01]

I sometimes play a half-assed tabletop drum part into the DAW and then adjust the click to my human performance. I'm not sure I need a humanization plugin if I already have a human "feel" algorithm running in my brain

Same here, I've been having similar thoughts about playing a single part and matching.

And this gets into another interesting question: do people who grew up listening mainly to click-based music hear human-generated intentional tempo variations as "wrong" or unpleasant?

That's a good question. I think the type of humanization we've been describing would sound better to them if they heard them side by side. If it's human and extremely well done presumably the emotional and expressive content will "speak" better and be preferred. I almost think it would sound really good to them in a way they may not be able to articulate but can sense. I'm not sure quantized music sounds "wrong" per se, it's just obvious that it's lacking when compared to inspired performance. That's how I see it, anyway, who knows with an individual who - as you say - barely even knows what unquantized music sounds like.

[courtjestr]

the inability for humans to play exactly in time.

The point is - they are not playing exactly in time for a reason, it's not "random". There's a reason for it, markmann articulated the reason very well.

I missed markmann's articulation of this. I have read what he wrote three or four times now. He stated "One is that even the best ones can't play exactly on the beat, so there will be tiny unintentional variations from a click track even if they try to play exactly in time with it." Note the words "unintentional variations". I understand you to be saying those unintentional variations are not random variations. That may be, but they can seem to be random in the fact that they are not predictable. I am looking at this the same way you can use computer generated random numbers in things like cloud generators for flight simulators or even a "rainstorm" simulator. Do they produce exact replicas of clouds, or rainstorms? No, but you can recognize them as reasonable facsimiles. I not convinced that the same can't be said for the current crop of humanization plugins at least regarding the idea of unintentional variations.

At any rate, I agree with both of you regarding the ideal humanization plugin. To paraphrase, an ideal humanization plugin "would produce a specific type of human performance(s) without mistakes - meaning, without randomization with a nuanced replication of the effects that result from the variables of real human performance."

[markmann]

@courtjestr and @10bd01 - Thanks for your thought-provoking ideas on this interesting topic. It's been a while since I had a productive, interesting conversation like this in a forum.

[10bd01]

@courtjestr and @10bd01 - Thanks for your thought-provoking ideas on this interesting topic. It's been a while since I had a productive, interesting conversation like this in a forum.

You, too, markmann! It's nice to see someone out there on a similar wavelength on this topic. I'll be plugging away here trying to get it the best as possible.

And you, too, courjestr - again, I'm glad you found something that works for you. I can see what you're saying; I don't find it sufficient and I don't think conflating the word randomization and humanization is healthy in this nascent market which is why I throw down on it.

[courtjestr]

@courtjestr and @10bd01 - Thanks for your thought-provoking ideas on this interesting topic. It's been a while since I had a productive, interesting conversation like this in a forum.

You, too, markmann! It's nice to see someone out there on a similar wavelength on this topic. I'll be plugging away here trying to get it the best as possible.

And you, too, courjestr - again, I'm glad you found something that works for you. I can see what you're saying; I don't find it sufficient and I don't think conflating the word randomization and humanization is healthy in this nascent market which is why I throw down on it.

Thanks for giving me food for thought. It was quite stimulating. I wouldn't say it is something that works for me with the idea that it is completely sufficient. I am not under the illusion that people listening music produced using the current crop of "humanization" plugins and algorithms are going to say "that sure sounds like a human playing".

I can see your point of view but I honestly don't think that when "true" humanization plugins are developed, people will be satisfied with just human simulations based on randomization. But I guess we need visionaries like you and markmann to drive the development of these plugins. It is not going to be cheap or easy which is why it has not been done yet.

[jancivil]

Human feel has naught to do with randomness. Unless there's a definite failure of competence, as in a not-fully-developed person.

One may hope that something that has to be too good to be true - like getting your time together isn't so much a thing that you can't just get an algorithm to fake that for you - is going to turn out to be true, but too good to be true is what we are talking about.

I don't mean to disrespect those who do the work or impute bad intent, it's I'm sure fun and interesting and great brain food and all, I mean it may even benefit everyone in the end; but I recommend examination of... <why would I have confidence in this being true?>:

Where is the logic, that makes a life of musical muscle memory and all the training that informs this, and the experience interacting closely with others and developing groove and sensitivity to time, in real time, with all of the memories and internal life that goes into a human musical expression, especially in the realm of timing, be captured in a pseudo-randomization scheme run by a machine with no actual agency?