All About MIDI Polyphonic Expression (MPE)

[zerocrossing]

[
aftertouch that is according the midi spec the term for polyphonic aftertouch, many developers, of hardware and software use different terms, and beside MPE, aftertouch is always meant to be: channel pressure.

Aftertouch is not always polyphonic.
Polyphonic aftertouch is not the same as Channel pressure.
Pressure starts at zero and goes to the max value. Aftertouch does not start at zero but after a note is played, hence AFTERtouch. Most of the times this last touch is a jump in value instead of a gradually changing value like with pressure.

'Aftertouch' does start at 'zero'. It starts at a value of '0' and goes up to 127. Perhaps what you meant to say is that aftertouch does not start at the actual note-on, which is what happens with continuous-pressure of MPE controllers. But you are right about that 'jump' in values when using aftertouch. This is yet another advantage of MPE controllers, this smooth, continuous-pressure response. If the synth engine supports expression-curves (or mod-maps) it is also possible to imitate the response of the old-fashioned aftertouch with deep exponential shapes (with an initial flat, no-modulation zone). So this way it is possible to have smooth, continuous pressure or aftertouch.

This is a problem with a lot of synths with MPE. I was checking out Plasmonic the other day and while it sounded good to me, it lacked the ability to alter the response in a way that I find crucial. It’s why I basically stick to Roli’s instruments and Vital.

[himalaya]

I did have an interaction with Eric on FaceBook and at first he was reluctant to include MPE compatibly in Omnisphere because of what you’re talking about, but after a few back and fourths he saw the value in adding it and he did say that he’d add it to future revisions. That was a while ago, though and nothing so far. I’d love for it to happen, though.

Sounds intriguing. We all know how amazing Omnisphere sounds, so a full MPE compatibility would be amazing to have. Another thing that Omnisphere lacked was the possibility to modulate the sustain stages of envelopes. This was in the original version, not sure if it has changed in the latest builds. Without this, it is impossible to modulate certain sounds, in the way that MPE demands. For example, if I have a 'pluck' sound, which decays to zero quickly (or to some lower value, so it gets really quiet), I need to modulate the sustain stage of an envelope (or envelopes) in order to bring the sound back to volume via one of the dimensions, be it pressure or CC74, or indeed, LIFT! I wrote to Spectrasonics about this but it was so long ago (2013 maybe).

Interestingly, when I made my patches in Omnisphere, I found that a lot of the samples felt like monolith files containing sub-layers that would move in their own pre-programmed ways. I found these type of samples not really suitable for MPE use, since they would modulate on their own accord without my input. So, as an example, I'd want to reduce the amplitude and tone of a sample, but as I would sustain the sound, it would grow/modulate in volume and tone by itself...These type of multi-layered samples are great for pads that we use on normal midi keyboard, as you sustain a chord and get a lush, sound that slowly shifts as you sustain a chord, but for MPE, this is a bit of a hindrance as we want to have these type of changes in sound under our fingertips.

I must revisit Omnisphere when I get the time. It's such a beast.

[himalaya]

This is a problem with a lot of synths with MPE. I was checking out Plasmonic the other day and while it sounded good to me, it lacked the ability to alter the response in a way that I find crucial. It’s why I basically stick to Roli’s instruments and Vital.

Yes. I wrote about this topic a few times on KVR. An MPE synth without expression-curves (or mod-maps/envelopes) is not really a finished MPE product for me. The implementation of the actual MPE MIDI spec is one thing, but then we need an easy and intuitive way to manage the sound and tune its expression to our MPE instrument of choice.

As an example, all MPE presets in Equator1/2, Cypher2, Strobe2, are tuned (in their 5D response) to the Seaboard RISE. The pressure is set to work with the very sensitive silicone surface, the CC74 is set to travel along the keys and reach the top of the key and stop at its full parameter value before we 'touch' the top ribbon (that's how it is in my sounds at least), all of this makes these sounds less attuned to other MPE instruments/surfaces. The result may be that the dimensions may be either too sensitive or not sensitive enough. They may give too much sonic modulation at once, or not enough. And this is all done with those Expression-Curves. These are a MUST for an MPE synth. That's why Equator (as well as Cypher2/Strobe2) features these as the central part on the GUI. We can not miss them, they are right in front of us, and so we pay attention to them. Hiding such curves (or mod-envelopes) in other menus or deep in a mod-matrix is not helpful (in an MPE synth). I think.

[Noumena]

here, here on all those points. until you have asymmetric lag (or rise/fall) processors and response attenuation and curves either in the synth or (in my mind even better) in your DAW, or a pre-processor or a modular environment, you've got MPE in theory but not in practice.

I would prefer for these capacities to be out of the synth so that I could use them in a general way with all my synths. I feel like an added benefit of this approach at least being available would be 1. that the synth makers wouldn't need to implement these things again and again and 2. the implementation could be more flexible than would be feasible in these synths where the developer time is limited and the percentage of people that would take advantage of polyphonic expression is small.

What I think would be ideal would be one or several MPE utility midifx VSTs that could preprocess MPE. Currently this is not possible in Bitwig and I wish that it were. I look forward to VCV Rack for DAWs where I think this kind of thing would be possible (host a VST in Rack in a plugin and perform all the preprocessing in the same patch.) I've done some experimenting with doing this preprocessing with MIDI loopback in VCV as a standalone process and it works... kinda sorta right now but in a way that is really intriguing for what could be possible down the line with VCV for DAWs.

Agree 100% that Equator and Cyper 2 are excellent in these ways, even if I find the Cypher 2 GUI kind of ponderous. I will check into Cypher 2 again on your recomendation... I was frustrated previously when it was first released with a serious bug in the way that modulation worked that seemed to not be fixed for quite some time, and it sounds like it may be fixed now. That said I've fallen a bit in love with the Grid's open architecture.

Re: Kontakt MPE. Kontakt is not very good at all for MPE, but I am very fond of the Spitfire Audio libraries, especially those that use the EVO grid and they are locked into Kontakt and so I've figure out how to make it work to good result but at no small amount of effort and frustratingly high CPU usage. The main idea is that you create a mono instrument that responds as you'd like and copy it either in Kontakt (each instance listening to a different channel) or use something like the Bitwig note selector and make copies of the Kontakt instance there, using the Round Robin/free voice voice allocation setting. The former is not so CPU intensive and the latter clogs up my CPU's threads pretty badly but is more flexible. I find that per note pitch and amplitude expression applied to these beautiful, excellent sample sets is sublime. Also in Kontakt is Slate and Ash's Auras library, which is built for MPE and requires no fussing (but can be quite CPU heavy) and which I find to be excellent in every way... maybe even the best implementation of MPE around, if the types of sounds it makes suits you. All that said I've been slowly resampling a lot of the Spitfire sounds into Bitwig sampler, which for my purposes works far better and is much more CPU efficient. I am tempted to look for ways around the copy protection so that I can do this easily and have, after all, paid for these things, but I try to be a good boy.

[Noumena]

Screen Shot 2020-12-06 at 1.32.00 PM.png

examples of the kinds of things that I've been working on. The Kontakt multi instance config allows for macro controls to be made that apply to all the instances (as noted above by another poster) which is handy. The Grid patch is a dual sampler with x-fade, some custom sustain pedal stuff that I cooked up, variable depth pitch modulation (so you can go from 1:1 note mapping to more subtle vibrato) and other curves and attenuators. Not shown are a large number of modulation mappings in addition to the modules.

[Tj Shredder]

This is a problem with a lot of synths with MPE. I was checking out Plasmonic the other day and while it sounded good to me, it lacked the ability to alter the response in a way that I find crucial. It’s why I basically stick to Roli’s instruments and Vital.

Yes. I wrote about this topic a few times on KVR. An MPE synth without expression-curves (or mod-maps/envelopes) is not really a finished MPE product for me. The implementation of the actual MPE MIDI spec is one thing, but then we need an easy and intuitive way to manage the sound and tune its expression to our MPE instrument of choice.

As an example, all MPE presets in Equator1/2, Cypher2, Strobe2, are tuned (in their 5D response) to the Seaboard RISE. The pressure is set to work with the very sensitive silicone surface, the CC74 is set to travel along the keys and reach the top of the key and stop at its full parameter value before we 'touch' the top ribbon (that's how it is in my sounds at least), all of this makes these sounds less attuned to other MPE instruments/surfaces. The result may be that the dimensions may be either too sensitive or not sensitive enough. They may give too much sonic modulation at once, or not enough. And this is all done with those Expression-Curves. These are a MUST for an MPE synth. That's why Equator (as well as Cypher2/Strobe2) features these as the central part on the GUI. We can not miss them, they are right in front of us, and so we pay attention to them. Hiding such curves (or mod-envelopes) in other menus or deep in a mod-matrix is not helpful (in an MPE synth). I think.

Absolutely, I really needed to adjust all expression curves in those wonderful Equator sounds to my LinnStrument. Unfortunately I can‘t send them to my iPad...!
Another point for MPE synths which is completely underrated but even more important is that most devs insist on ADSR envelopes in their designs. In the context of an expressive controller they are almost useless! With the five expressions, you need to control with velocity only the first segment of an envelope. In an ADSR the level of the first segment isn‘t independently accessible, instead the velocity controls the overall level. A soft start of a note prevents getting louder with later aftertouch.
Already in the eighties we had better envelopes in the DX7 or CZ-series synths.

[himalaya]

I think there are two main reasons why you see ADSR envelopes dominating here.

1. ADSR envelopes are familiar to all people who use synths. We instinctively know what to do with them, and I guess, it would be easy to explain their function to a newcomer. This 'familiarity' is important, I feel, since designing good MPE sounds is not trivial as it is. So, we need synths that are approachable.

2. Then, there are synths that support MPE and 'standard' presets (or connectivity/compatibility). Equator2 is one such synth. It has been designed from the ground up to cater to both these 'sides'. So, it needs ADSR envelopes in order to support the normal, 'standard' approach of patch making so that it can be successfully used with a standard midi controller.

But I guess all of the above shouldn't prevent additional envelope modes which do as you propose. Right?

A soft start of a note prevents getting louder with later aftertouch.

You can actually do this with normal ADSR envelopes. If that's all you want to do, it is possible. In fact, I do it all the time, and my (plus other sound designer's ) MPE patches wouldn't work otherwise.
What I do in this case, I modulate the sustain stages of an ADSR envelope, or several envelopes. It all depends on the patch. So:

1 I assign the sustain stage of the Amp envelope to pressure. This allows me to bring the sound back up to a volume level I desire. So, if you have a 'pluck' type of sound, which decays to zero (total silence), it can be brought back to full volume with this technique.

2 But the above may not be enough if the sound uses filters. A filter can also close down the sound, or modulate it in many ways that affects the amplitude. So, in many cases, I need to modulate the sustain stage of the filter envelope, to perhaps open the cutoff of the LPF again. Together with the Amp envelope and its sustain, the patch can respond to amplitude and timbral changes via pressure in very nuanced or powerful ways. You can start with a very soft velocity value and have the sound very quiet and then apply pressure to add volume and timbre modulation/boost.

Having said that you are right in pointing out the need to have better means of controlling velocity. In my experience the problem lies with velocity vs continuous pressure. these two can really 'fight' with each other, or 'get in the way'. This can be a problem. Another problem relates to note-off release velocity vs pressure (LIFT vs PRESS). Here too, these can 'fight' and get in the way. So, I agree that we do need more elaborate options here to manage it all. I tend to use specific expression-curve shapes to do it. It can help with LIFT vs PRESSURE, but it's not entirely satisfactory, as the user needs to know what gesture to play in order to use LIFT successfully, and even then, pressure will also be modulated along note-off release (which is not what we may want).

I'm not sure if you have used the Egan Matrix which powers the Haken Continuum, but it actually provides what you desire. Its 'Formula' system allows for such control. It's a powerful, deep and complex design, and definitely not suited to a normal use by the average user. Which brings me back to my first point, that of 'ease of use vs complex options'.

[himalaya]

examples of the kinds of things that I've been working on.

I'd be interested to hear how the actual patch sounds. Do you have any demos?

[Tj Shredder]

I think there are two main reasons why you see ADSR envelopes dominating here.

1. ADSR envelopes are familiar to all people who use synths. We instinctively know what to do with them, and I guess, it would be easy to explain their function to a newcomer. This 'familiarity' is important, I feel, since designing good MPE sounds is not trivial as it is. So, we need synths that are approachable.

2. Then, there are synths that support MPE and 'standard' presets (or connectivity/compatibility). Equator2 is one such synth. It has been designed from the ground up to cater to both these 'sides'. So, it needs ADSR envelopes in order to support the normal, 'standard' approach of patch making so that it can be successfully used with a standard midi controller.

But I guess all of the above shouldn't prevent additional envelope modes which do as you propose. Right?

A soft start of a note prevents getting louder with later aftertouch.

You can actually do this with normal ADSR envelopes. If that's all you want to do, it is possible. In fact, I do it all the time, and my (plus other sound designer's ) MPE patches wouldn't work otherwise.
What I do in this case, I modulate the sustain stages of an ADSR envelope, or several envelopes. It all depends on the patch. So:

1 I assign the sustain stage of the Amp envelope to pressure. This allows me to bring the sound back up to a volume level I desire. So, if you have a 'pluck' type of sound, which decays to zero (total silence), it can be brought back to full volume with this technique.

2 But the above may not be enough if the sound uses filters. A filter can also close down the sound, or modulate it in many ways that affects the amplitude. So, in many cases, I need to modulate the sustain stage of the filter envelope, to perhaps open the cutoff of the LPF again. Together with the Amp envelope and its sustain, the patch can respond to amplitude and timbral changes via pressure in very nuanced or powerful ways. You can start with a very soft velocity value and have the sound very quiet and then apply pressure to add volume and timbre modulation/boost.

Having said that you are right in pointing out the need to have better means of controlling velocity. In my experience the problem lies with velocity vs continuous pressure. these two can really 'fight' with each other, or 'get in the way'. This can be a problem. Another problem relates to note-off release velocity vs pressure (LIFT vs PRESS). Here too, these can 'fight' and get in the way. So, I agree that we do need more elaborate options here to manage it all. I tend to use specific expression-curve shapes to do it. It can help with LIFT vs PRESSURE, but it's not entirely satisfactory, as the user needs to know what gesture to play in order to use LIFT successfully, and even then, pressure will also be modulated along note-off release (which is not what we may want).

I'm not sure if you have used the Egan Matrix which powers the Haken Continuum, but it actually provides what you desire. Its 'Formula' system allows for such control. It's a powerful, deep and complex design, and definitely not suited to a normal use by the average user. Which brings me back to my first point, that of 'ease of use vs complex options'.

I do the same usually, assign the sustain to pressure. But if I want a soft attack, with an ADSR I can‘t get beyond that soft attack, no matter how much I press...
To add a Peak value to an ADSR for sure would not make it less accessible for those who can deal with with AHDSRs or DAHDSRs (both not that crucial). A simple APDSR would do it in most cases and if there are some more options like MSEGs you‘d be covered. In modular environments I usually try to have parallel an envelope for the attack, a path solely controlled by pressure and an envelope for the release stage. But there its a pain to adjust the levels and somehow crossfade between the different stages... A simple EG with four stages like in the DX7 would cover all needs...(dexed is now MPE compatible and therefore more interesting than FM8 or Arturias approach...)

[deastman]

I'm not sure if you have used the Egan Matrix which powers the Haken Continuum, but it actually provides what you desire. Its 'Formula' system allows for such control. It's a powerful, deep and complex design, and definitely not suited to a normal use by the average user. Which brings me back to my first point, that of 'ease of use vs complex options'.

This is it exactly. Conventional synths are not built with MPE in mind, even if they support it. With the EaganMatrix, there is no concept of an ADSR. Your fingers are the envelopes.

[himalaya]

I'm not sure if you have used the Egan Matrix which powers the Haken Continuum, but it actually provides what you desire. Its 'Formula' system allows for such control. It's a powerful, deep and complex design, and definitely not suited to a normal use by the average user. Which brings me back to my first point, that of 'ease of use vs complex options'.

This is it exactly. Conventional synths are not built with MPE in mind, even if they support it. With the EaganMatrix, there is no concept of an ADSR. Your fingers are the envelopes.

Well, Equator2 was build with MPE in mind, very much so. It's a conventional synth, but with MPE DNA. MPE (or back before MPE wasn't devised yet, 5D) was the exclusive reason it was made. To make a synth engine that would be able to make multi-dimensional sounds.

You have to remember that a lot of people simply want traditional sounds that rely on the shape that ADSR envelopes can provide, while being able to add some extra expression as they press, slide and glide. You trigger a traditional 'Synth Keys' patch, as an example, experience the specific shape of the sound (attack-into-decay) and mangle it with CC74 (where the previously mentioned envelope sustain comes into play).

In addition, all of that is very simple to achieve in Equator2, or any synth that is 'conventional' and supports MPE, but to do it in the Eagan Matrix, for the average user, will be impossible. The engine is just too complex. Of course, the reverse is also true: Eagan Matrix can make sounds that more conventional MPE synths won't be able to. Its total attuning to the Continuum helps in that too. It's such a powerful combo. Unique to this day.

[deastman]

Well, Equator2 was build with MPE in mind, very much so. It's a conventional synth, but with MPE DNA. MPE (or back before MPE wasn't devised yet, 5D) was the exclusive reason it was made. To make a synth engine that would be able to make multi-dimensional sounds.

You have to remember that a lot of people simply want traditional sounds that rely on the shape that ADSR envelopes can provide, while being able to add some extra expression as they press, slide and glide. You trigger a traditional 'Synth Keys' patch, as an example, experience the specific shape of the sound (attack-into-decay) and mangle it with CC74 (where the previously mentioned envelope sustain comes into play).

In addition, all of that is very simple to achieve in Equator2, or any synth that is 'conventional' and supports MPE, but to do it in the Eagan Matrix, for the average user, will be impossible. The engine is just too complex. Of course, the reverse is also true: Eagan Matrix can make sounds that more conventional MPE synths won't be able to. Its total attuning to the Continuum helps in that too. It's such a powerful combo. Unique to this day.

That’s all very true. Different synths and controllers are better for different things. That’s why I’ve ended up with the Seaboard, LinnStrument, Continuum, Morph, and soon the Osmose. Not to mention all the various VSTi’s that support them. In addition to the Roli instruments, the Audiomodeling ones are fantastic with MPE.

[himalaya]

That’s a formidable MPE arsenal you have there. It’s great to see users embracing these instruments like you.

The Audio Modelling instruments/sounds are first rate, agreed. I simply love to play them. But the sounds in the Eagan Matrix are also amazing. On another level. That synth engine and Ed Eagan’s sound design talent are very inspirational to me. He’s the guru.

[pdxindy]

here, here on all those points. until you have asymmetric lag (or rise/fall) processors and response attenuation and curves either in the synth or (in my mind even better) in your DAW, or a pre-processor or a modular environment, you've got MPE in theory but not in practice.

I would prefer for these capacities to be out of the synth so that I could use them in a general way with all my synths. I feel like an added benefit of this approach at least being available would be 1. that the synth makers wouldn't need to implement these things again and again and 2. the implementation could be more flexible than would be feasible in these synths where the developer time is limited and the percentage of people that would take advantage of polyphonic expression is small.

What I think would be ideal would be one or several MPE utility midifx VSTs that could preprocess MPE. Currently this is not possible in Bitwig and I wish that it were. I look forward to VCV Rack for DAWs where I think this kind of thing would be possible (host a VST in Rack in a plugin and perform all the preprocessing in the same patch.) I've done some experimenting with doing this preprocessing with MIDI loopback in VCV as a standalone process and it works... kinda sorta right now but in a way that is really intriguing for what could be possible down the line with VCV for DAWs.

To do that in Bitwig, Bitwig would need modulation to be per voice for VST's. I would love that!

And yeah... it also makes much more sense to me to have that capability in the DAW. Bitwig would be perfect for it.

Bitwig's Grid is (imo) the best MPE synth available today.

[Whywhy]

What I would like to see in every mpe synth is a rise and fall lag.( Bitwig, Axoloti and others have this kind of modules).the pressure can really act like an envelope.
The problem with Mpe this day is how Mpe controllers are dealing with data, some send a : return value to zero at note off for pressure, other just before note on, other send nothing. Mpe is a standard not so standard-:). Standard need times to be establish...
Mpe synth company should spend more time with their smoothing algorithm, it can make a big change when performing.
The mpe+ specification from Haken audio should be implemented or adapted in every mpe synth. Mpe synth compatibility often means for developers: pitch bend, press and Slide per note, but it need a lot more,like curves, lag...
Like Himalaya said: Edmund Eagan is a guru, the best for me and totally underrated in the synthesizer sound designers community, and just after, not so far, Electric himalaya !
2 different styles, 2 masters ( it's sound like a fanboy, yes, I am-:))
They are the most experienced Mpe sound designers.
Best wishes, and happy Christmas holidays.
Yuli Yolo.