VST Midi humanizer plugin

[10bd01]

Where is the logic, that makes a life of musical muscle memory and all the training that informs this, and the experience interacting closely with others and developing groove and sensitivity to time, in real time, with all of the memories and internal life that goes into a human musical expression, especially in the realm of timing, be captured in a pseudo-randomization scheme run by a machine with no actual agency?

This was basically what always confounded me as well - it doesn't really make sense to begin with and yet they start with that foundation that makes no sense and run with it to create "humanization" plugins.

I don't see algorithms as a way to do this at all, although I wouldn't want to scratch the possibility. I do think machine learning/AI - though it still wouldn't be perfect - could produce the closest result we could hope for at the moment, depending on the amount of data it has to process. Basically what it would be doing is creating probabilities based off "listening" to hundreds or thousands of hours of a performer, learning their "style" (or tendencies), and then projecting the application of that style onto a set of midi data. So the entire software is then based off of a type of "understanding" and projection of the human agency of that specific performer.

It does get super complex, and the "level 1" version of the above wouldn't be accounting for the variables you or I have mentioned - group playing, genre, etc. but would basically be a "solo" version to start with. With ML/AI what it comes down to is how much data you have - so if we took every recording available of John Bonham, extrapolated the drum parts, analyzed it with tensorflow and then applied the projections onto a new midi drum file, we'd have a sort of "approximation" of Bonhams playing, not accounting for a number of variables but perhaps capturing and then projecting some of the human agency he brings to his performances. If we have a living studio musician and put him with a variety of different combos (Jazz, Rock, etc.), record the data and specifically project his "tendencies" for each genre we would then have that additional data point and would be more accurate in relation to that musicians agency when playing a specific genres.

An expensive but effective way to make it more complete would be to create a type of "ensemble" for this and have a group of say 25 studio musicians that all play different genres of music with one another, of different tempi, different keys, and to record them together performing hundreds of hours of music, and then - after processing the data and having these now virtual musicians available to choose from by the end user - give the end user an ability to specify genre, mood, etc. The ML/AI now has hundreds of hours human agency to interpret and project and a variety of different data points, meaning if you added these "virtual" musicians to a midi rock piece and specify it as such you'd be essentially getting the most accurate portrayal of those musicians playing that music available at this time, and this could be optimized over decades of time by periodically adding more data from the same musicians (to an extent).

I think the benefit of the ML/AI tack is that it is specifically attempting to project based off of a type of statistical analysis of the actual human agency. So say if you have 1000 hours of a drummer playing. You take the first 500 hours of their playing, analyze them, then the analyzer attempts to predict through projection the final 500 hours and then adjusts it's interpretation on whether or not it accurately predicted that second 500 hours. So, with a consistent musician, you may end up with an AI model that with that first 500 hours of playing can predict that performers "moves" with say 80% accuracy on the final 500 hours. That model is then applied to music that is not pre-recorded with the understanding that the interpretation would then be around 80% accurate for those as well, though this may or may not be completely accurate.

The more data that's collected the more accurate the model will become. If you were to extrapolate the drum data from every rock album ever made and pump all of that drum data into it you'd end up with a type of "meta" rock drummer - a generalized extrapolated and projected spirit of all rock drumming that could then be applied - with limited accuracy - to midi data.

Again, not a perfect solution, but a solution that would bring us way closer IMO than applying a randomization algo, which again, IMO makes no sense. Another benefit to this approach is - because it becomes more accurate with more data - it can always be improved on and enhanced to become more accurate as long as the musicians are available to provide more data.

[fairlyclose]

There are two main reasons to pursue a midi humanizer - first one wants to model human musical performance and implementing the model allows one to test one's understanding ie one wants to do science. Secondly one wants to make money ie one wants to do business. Both might overlap.

[The_Anarchivist_00]

Necessary necro:

Idea. A software unit to push and pull MIDI data away from perfect time. Note starts and stops. It would have a one knob interface, and would be controllable in realtime or by automation. Humanization and randomization is not enough. It can't do the job, because this is not 'the job', musicians do, at all. I know because I used to be a drummer in a band (..might still be - haven't checked lately). Adding small amounts of so called humanization and randomization to intentional push pull control data would give access to human effects not yet available especially in the bedroom producer ecosystem.

EDIT: I think I can achieve what I am after using one of the knobs on RandARP. Excellent.

[chokoisamusic]

EDIT: I made a Multi-Channel Midi Humanizer preset for ReaJS that I find works much better for me (link in YT comments):

[jancivil]

To me the "perfect solution" is simple: make your own mistakes and don't quantize them to have to be messed with later in search of making something that is certainly not human "human-ized".

as to the penultimate post before mine, randomization is def. not enough, actually it's bullshit. Compelling musical timing is about feel. There are things that can be known about it and analyzed, it isn't going to be predictable with any facility until you know things about it; music that lives and breathes is not so easily reducible. If one can't develop their own sense of time they're doing something less than genuinely making music. It might still be a good hobby or pastime, but my fear is people reading about humanization are being conned (into spending).

[jancivil]

One interesting method I read about recently involves a person playing the rhythm of a passage without worrying about the actual notes and then laying the melody or chords on top of the recorded passage, keeping the velocities and the timing but adjusting the pitch. I could see that being effective though it still would probably not be accurate to a human playing the actual instrument. True humanization of course results from humans actually playing the instruments.

What has to be human about pitch if our pitch system is 12 tones equal to an octave? Mistakes?
Or are we still in consideration of timing (NB: if it's an instrument that historically reifies pitch decisions that aren't sorted in 12tET - say violins, and 'leading tone is intoned a bit sharper' etc - we have a different ballgame).

I've in fact done at least two compositions that way, because I could very definitely get the rhythm live but at tempo my pitch accuracy was going to be quite poor because I'm not that nimble on a keyboard. I didn't even know what the pitches should be, past more general contour.
Frank Zappa had a composition called "Approximate" (the verb) where the group was left to its own devices as to what happens other than the rhythm, except this version would be 'sing it' or scat it, that version would be stamp your feet etc. So I took that as an idea because I had rhythmic ideas and was rearing to go.

Filling in the pitches for the established rhythm was fairly cut-and-dried. I was cognizant of feel, some things would be nudged to rush or relax the time according to the instrumental role. "the bass player is excited to do this lick and rushes here" etc.

[Shabdahbriah]

To me the "perfect solution" is simple: make your own mistakes and don't quantize them to have to be messed with later in search of making something that is certainly not human "human-ized".

as to the penultimate post before mine, randomization is def. not enough, actually it's bullshit. Compelling musical timing is about feel. There are things that can be known about it and analyzed, it isn't going to be predictable with any facility until you know things about it; music that lives and breathes is not so easily reducible. If one can't develop their own sense of time they're doing something less than genuinely making music. It might still be a good hobby or pastime, but my fear is people reading about humanization are being conned (into spending).

Hence the ubiquitous (artist, lacking finesse and nuance) title: "producer".

[cryophonik]

Hence the ubiquitous (artist, lacking finesse and nuance) title: "producer".

Perhaps it’s time to update that title to something more prestigious: “randomizer”

[Shabdahbriah]

Hence the ubiquitous (artist, lacking finesse and nuance) title: "producer".

Perhaps it’s time to update that title to something more prestigious: “randomizer”

Graduating to: "probability expert", or "innovative", or "original" at some point.

[sandandpaint]

lots of work has been based on probability / randomisation - is just personal taste if you like that or not.
Mimicking human playing is another thing - I seem to remember there was some Russian software long ago that was based on sophisticated analysis of actual playing to provide 'feel'. There is no end of empirical academic work looking at timing variation

[chokoisamusic]

To me the "perfect solution" is simple: make your own mistakes and don't quantize them to have to be messed with later in search of making something that is certainly not human "human-ized".

Hey I agree completely in that specific use case. However, there are use cases apart from this that this plugin solves. For example, I wanted to create a wall of guitars for live midi playing. With 4 different midi guitars starting at slightly random times. This humanizer plugin solves that. OFC playing 4 guitars at once with octopus arms is the best solution but not practical for this case

[hotmitts]

I often use random for velocities in MIDI drum sequencing.
'random' may or may not be what a human player actually does but it's better than a midi sequence with the same fixed velocity all through, which isn't what a human player does either.
I have played around with micro timing too, again this is probably less ear fatigue than exactly on the beat stuff (as long as it's small values)
I've never found the 'shuffle' type option in MIDI useful to me. It's hard to get a real groove going with strict quantisation, but my finger drumming skills aren't up to un-quantised beat patterns.
I've always struggled with adding MIDI drum parts to my 'real' guitar/ audio parts, they'e like 2 different worlds.

[Dirk Diggler]

The Reaper DAW has several options, one built in, and one as mentioned above in the JS category.