Channel Pressure Behavior: Fixed in Custom Firmware

[teknico]

Thank you so much for this discussion. After just one week, you identified and articulated clearly something that took me a few years. I came to the same realization as you a few days ago, and was about to raise the issue here (talk about synchronicity).

Thanks for saving me the work, and for investigating the issue and coming up with a draft fix.

The Problem

As you noted, what happens is by design, and I concur that it is suboptimal. The behavior you prefer (sending data from additional touches only when they exceed the currently held ones), in addition to being similar to historical synths, is definitely an improvement for expression with any monophonic synth engine.

As you noted, the Linnstrument already does something similar in One Channel mode, in order to generate a pitch bend value from multiple X values. And the issue you noted applies to the Y axis as well as the Z one, whenever one has to reduce data from multiple values to one.

This happens for Channel Aftertouch in One Chan mode, but also with Poly AT or CC whenever the synth is in Mono mode. My use case is when the Linnstrument is in Channel Per Row mode and each MIDI channel is in Mono mode, whether to imitate the playing articulation of individual strings, or wind and brass instruments.

The Solution

Roger had the foresight and wisdom to publish the firmware source code as open source, and Geert did a great job writing and documenting it, so we have the legal right and the technical ability to make a custom version of the firmware, without depending on or however impacting any third party. We Linux users are very fond of FLOSS.

Implementing the change in some external MIDI processor would be a more expensive, potentially more complex, and less efficient workaround. It's apparent to me that the logic in the Linnstrument firmware is not optimal, and that's fairly low hanging fruit.

I'm going to work on this in the coming days. Let's fix this.

[ryanpg]

Great! Yes, I've been running Linux since the 1.0 kernel. Major pioneer projects like this (and I kinda put Linnstrument and MPE into the same sea change category as the Linux kernel itself) are so important. Linnstrument will be first contact with MPE controllers for many people.

Teknico, I see you replied to my github bug report. Roger explained that Github is not the preferred way to report issues for Linnstrument. (Yeah, as an open-source Linux guy that's where I headed first too!) Just heads up.

In fact, Teknico, his whole topic may not be suitable for public consumption (I wouldn't want to encourage anyone to risk bricking their Linnstrument) so if you would like to collaborate via email, let me know.

That said, I've spent some time looking and I think the start of a solution could possibly be easy:

In ls_midi.ino this section of code seems to handle sending aftertouch (or any Z message).

Code: Select all

// Called to send Z message. Depending on midiMode, sends different types of Channel Pressure or Poly Pressure message.
void preSendLoudness(byte split, byte pressureValueLo, short pressureValueHi, byte note, byte channel) {
  pressureValueHi = applyLimits1016(pressureValueHi, Split[split].minForZ, Split[split].maxForZ, fxdLimitsForZRatio[split]);
  // scale 1016 to 16383 and fill out with the low resolution in order to reach the full range at maximum value
  pressureValueHi = (pressureValueHi * 16 + pressureValueLo) & 0x3FFF;
  pressureValueLo = applyLimits(pressureValueLo, Split[split].minForZ, Split[split].maxForZ, fxdLimitsForZRatio[split]);
  switch(Split[split].expressionForZ)
  {
    case loudnessPolyPressure:
      midiSendPolyPressure(note, pressureValueLo, channel);
      break;

    case loudnessChannelPressure:
      midiSendAfterTouch(pressureValueLo, channel);
      break;

    case loudnessCC11:
      // if the low row is down, only send the CC for Z if it's not being sent by the low row already
      if ((!isLowRowCCXActive(split) ||
            Split[split].customCCForZ != Split[split].ccForLowRow) &&
          (!isLowRowCCXYZActive(split) ||
            (Split[split].customCCForZ != Split[split].ccForLowRowX &&
             Split[split].customCCForZ != Split[split].ccForLowRowY &&
             Split[split].customCCForZ != Split[split].ccForLowRowZ))) {
        if (Split[split].customCCForZ < 32 && Split[split].ccForZ14Bit) {
          midiSendControlChange14BitMIDISpec(Split[split].customCCForZ, Split[split].customCCForZ+32, pressureValueHi, channel);
        }
        else {
          midiSendControlChange(Split[split].customCCForZ, pressureValueLo, channel);
        }
      }
      break;
  }
}

As I've stated, I'm in no way a programmer, but it seems to me (and my experimentation gives hope to the idea) that in the line:

Code: Select all

 pressureValueLo = applyLimits(pressureValueLo, Split[split].minForZ, Split[split].maxForZ, fxdLimitsForZRatio[split]);

I think

Code: Select all

Split[split].minForZ

sets the lower bounds for aftertouch values.

I've experimented with replacing that with

Code: Select all

scale1016to127(sensorCell->currentRawZ, true)

my idea being that the smallest aftertouch value sent should always be >= the next highest value held.

It doesn't work yet though, but I feel I'm on the way!

[edit] This is in no way "the solution" as even if I succeed in setting the lower limit of aftertouch to the prior held level, it would break MPE. That's not a bid deal in the long-run as we just need to break that behavior out to the existing "Aftertouch" setting. But before I get too smacked down I wanted to make that point.

[Roger_Linn]

Hi teknico,

Thank you for kindly helping ryanpg. Some thoughts came to mind that might be helpful in your efforts:

1) The "LinnStrument's Smart MIDI" page isn't accurate in the section about One Channel mode. When performing vibrato on a chord and sending over a single MIDI chanel, LinnStument does not take the average of the X-axis position of the chord's touches as stated on the Smart MIDI page. Instead, it takes the X-axis position of the most-recently played pad only. Sorry about that. We had tried the averaging method but it didn't work well, and I failed to correct the Smart MIDI page. I have just corrected it.

2) When two or more pads are pressed and you are sending over a single MIDI channel, you mentioned an intention to send a Channel Pressure value that is the average of all simultaneous touches. We tried this and it doesn't work well. For example, if you hold one touch at full pressure while pressing a new touch at very low pressure, the Channel Pressure value will instantly jump to a value that is in-between the two pressure values, causing the same abrupt change that Ryanpg is trying to avoid. For this reason, I think Ryan's original proposed solution will work better for him: send the Channel Pressure that is the highest of all simultaneous touches. This has other problems but will give him what he seeks and what he is accustomed to in MIDI piano keyboards, because they sense not the pressure of individual keys but rather the pressure on the entire keyboard mechanical frame, regardless of whether you're pressing one, many or all keys, and therefore can't work any other way.

Regarding whether to implement this alternate behavior in the LinnStrument Source code or in an external MIDI script, I still think the latter would be easier because it wouldn't risk breaking other parts of the LinnStrument source code. All you'd need to do is set LinnStrument to send Poly Pressure in One Channel mode, then write a script that when multiple touches are pressed, only pass the pressure messages with the highest values as Channel Pressure messages. Then if the result is what you intended and didn't present any new and unwelcome surprises, it would be easier to implement the same change in the LinnStrument source code.

[monch1962]

Before you go down the path of either customising Linnstrument firmware or buying new hardware to sit between the Linnstrument and your synths, have a look at https://audeonic.com/streambyter/

Several reasons:
- there might be downsides to what you're trying to do that you haven't considered. Even if you eventually go with a hardware or firmware based solution, it probably makes sense to implement it first in software to find out if it's going to do what you want first
- while it takes a bit of time to get your head around Streambyter, it should be quite possible to implement something to follow Roger's approach of capturing the "highest value" poly pressure message and reuse it. If you're reasonably experienced with coding in another language, I suspect it'd only take a few hours to write a Streambyter script to do this
- if you've got an old iPhone or iPad lying around, it should be possible to run your Streambyter script on it rather than running it on a computer or building a custom bit of hardware to run something. I just confirmed that Streambyter installs and runs fine on an ancient iPhone 6 (stuck on iOS 12) we've got sitting in a drawer
- if you start changing Linnstrument firmware, you have to work out what you're going to do with respect to keeping your changes in sync with any future firmware updates from Geert

[ryanpg]

Thank you for the recommendation. Streambyter looks like a cool app and an excellent tool. Unfortunately, I own no iOS devices. That said, the open source music programming language Supercollider is capable of manipulating MIDI data, has C-family syntax, and is also tons of fun. I've been exploring the Linnstrument code base while also playing around with Supercollider to prototype just as Roger suggested.

BTW, this thread is becoming a great list of gear and programs to manipulate MIDI!

I agree with you, monch1962, and I will not buy new hardware to make Linnstrument work as expected with my existing hardware.

Your concern that fixing channel pressure (and ultimately One Channel mode Y axis data too) may have unintended consequences, is almost certain to be valid. I have seen this play out many times in many complex projects. Those unintended consequences may require fixing too. However, leaving a problem to persist is not the answer I prefer.

As teknico points out, Roger and Geert are to be celebrated for making the Linnstrument firmware open source. Any changes (should they be found useful) will be shared publicly and can be adapted into future updates; if they cannot be adapted to new firmware versions, people can choose to use a custom firmware based on old code, and people can ignore any custom firmware altogether.

Well intended and thoughtful solutions presented so far:

• Buy new hardware synths that support MPE
• Buy a hardware device that might filter the midi
• Use VSTs
• Use a DAW
• Use a computer and script to modify midi when using non MPE synths
• Use Arduino, iPad, other, to sort aftertouch
• Don't use channel aftertouch on Linnstrument
• Don't use Linnstrument with non MPE hardware
• Use additional hardware connected to affected synths to achieve similar results
• Use low-row as a workaround
• Just accept that channel aftertouch will not work as it does on any other controller

Tellingly, no one has said they enjoy or find generally useful the way One Channel mode X and Z data work now. (Of course someone may use it experimentally as-is, but it cannot function traditionally.)

My proposed solution is to modify the firmware to emulate mono aftertouch (admittedly limited, outdated as it may be) as it behaves on other MIDI devices in use for 45 years.

Here's another reason I already love Linnstrument. The other solutions above would be all there is to do, were we talking about other commercial products. It would be "take it or leave it," over and done with. I'd pick my poison, or return my Linnstrument and move on, were it not for the fact that we have the code at hand.

Hurrah open source! And again hurrah to Roger and Geert for making the code available. The fact that the firmware is open means Linnstrument can never become obsolete and is guaranteed to outlive any single person's involvement.

The progression in expressiveness from mono aftertouch to poly aftertouch and finally MPE is exciting. I prefer the latest method, MPE. Yet, I still believe Linnstrument has the potential to represent all these methods of expression very well. Time and effort will tell if I am right.

[Roger_Linn]

Often, the reward is in the journey.

[ryanpg]

Often, the reward is in the journey.

Perfectly stated.

[ryanpg]

Quick update on the project:

I've been investigating the Linnstrument source (while teaching myself the basics of coding) and my current understanding is that the Linnstrument:

• scans pads and applies a series of logic conditions (i.e does pressure exceed a preset threshold) to determine if a touch is a new note or a slide or some other event
• in the case that a touch is determined to be a new note, sends a "note on" message, and a velocity value based on initial Z axis pressure data
• the most recent touched pad has "focus" and only a pad with focus sends Z (or pressure) data
• upon release of a pad with focus, focus reverts to a previous pad (not sure how or what happens when many keys are held, I think it's just the last pad touched)

There is some logic for averaging X (vibrato and pitch bend) data in the case of One Channel mode, but this is disabled in the source.

Because the Z data of all current presses is neither stored or sent--only the focused pad--doing a simple real-time comparison of greatest value simply isn't possible in the current firmware.

Possible solutions:

1. scan held pads, store z-pressure data and do comparisons
2. change the "focused cell" behavior so that z-data is sent for all held pads
3. use the currently unused averaging logic to average z-data and send that in One Channel mode
4. don't reset z-data at release
5. add slew to z-data transitions between focus and prior-focused touches

Problems with each proposed option:

1. adds overhead, probably should limit the number of touches stored, but how and how many
2. blasting out aftertouch messages is going to choke the Linnstrument and potentially connected MIDI gear
3. as Roger pointed out, there will still be jumps in value (though much smaller ones)
4. leaves parameters set in unpredictable ways, i.e. filter half closed with no pads touched while on the synth it appears open
5. doesn't really solve the problem, just makes it sound less ugly, but basically invents a new "portamento mode" for aftertouch

I'm still thinking about this and tinkering. I've gotten some experimental firmware to build, load and run on my Linnstrument, but nothing close to a success yet.

And yes, Roger, I'm having a blast both playing the Linnstrument and traveling on this learning journey! Woo hoo!

[edit] :After some reflection, I think the only way to get what people expect from One Channel mode, is to scan held pads for z-values and only ever send the highest value at any time. Maybe do some smoothing of those values.

In the case of Channel per row, scan held pad z-values and send only the highest value per channel.

[teknico]

Quick update on the project:

I've been investigating the Linnstrument source

Same here. I went through all the code (it's a lot!), and there goes my 30-year, career-long, previously successful avoidance of C++. Thanks ryanpg! (I do have experience with C though.)

(while teaching myself the basics of coding) and my current understanding is that the Linnstrument:
[...]

Agreed, plus some additional sends of zero-value Z that will need to be conditioned.

There is some logic for averaging X (vibrato and pitch bend) data in the case of One Channel mode, but this is disabled in the source.

I'm not sure it's actually disabled, but it does look ineffective. It's still a good starting point for what we need for Z and Y.

Because the Z data of all current presses is neither stored or sent--only the focused pad--doing a simple real-time comparison of greatest value simply isn't possible in the current firmware.

I believe they're actually already stored, and if they're not, it shouldn't be too hard to store them alongside the X data.

Possible solutions:

1. scan held pads, store z-pressure data and do comparisons
   [...]

Problems with each proposed option:

1. adds overhead, probably should limit the number of touches stored, but how and how many
   [...]

Definitely the first option. I don't think there's going to be a lot of overhead, and the number of stored touches is already limited anyway.

I'm still thinking about this and tinkering. I've gotten some experimental firmware to build, load and run on my Linnstrument, but nothing close to a success yet.

Same here. It's annoying that uploading the firmware erases the calibration data too, and the Linnstrument requires the calibration procedure each time.

The way to avoid that is to use the updater, which unfortunately does not have a Linux version. Thankfully I also have a Mac.

The updater is based on the Juce framework, which has been supporting Linux for a while, so it should be possible to add that (and maybe also sprinkle in a little bit of Zig as builder ). No promises though!

And yes, Roger, I'm having a blast both playing the Linnstrument and traveling on this learning journey! Woo hoo!

Same here again, C++ notwithstanding. I should have done this years ago.

[edit] :After some reflection, I think the only way to get what people expect from One Channel mode, is to scan held pads for z-values and only ever send the highest value at any time. Maybe do some smoothing of those values.

That's what we're going to do, and I don't think any smoothing will be needed. We'll see.

In the case of Channel per row, scan held pad z-values and send only the highest value per channel.

Correct, and do that for each row. That might require adding some data.

[ryanpg]

Thanks for the update, teknico!

[ryanpg]

I'm still thinking about this and tinkering. I've gotten some experimental firmware to build, load and run on my Linnstrument, but nothing close to a success yet.

Same here. It's annoying that uploading the firmware erases the calibration data too, and the Linnstrument requires the calibration procedure each time.

The way to avoid that is to use the updater, which unfortunately does not have a Linux version. Thankfully I also have a Mac.

The updater is based on the Juce framework, which has been supporting Linux for a while, so it should be possible to add that (and maybe also sprinkle in a little bit of Zig as builder ). No promises though!

Linux only in my household. My calibration data is long-gone; I just don't re-calibrate between uploads. I saw the updater was based on JUCE. Maybe next project? Hah.

It's probably not useful for me to post non-working code (and in doing so reveal how hacky I am) but I don't see why this doesn't work. It builds, uploads, and runs, but the behavior is very similar to default.

Code: Select all

    case loudnessChannelPressure: {
      int pressure = 0;
      byte currentPressure = 20;
        {
        // iterate over all the rows
        for (byte row = 0; row < NUMROWS; ++row) {
          int32_t colsInRowTouched = colsInRowsTouched[row];
          while (colsInRowTouched) {
            byte touchedCol = 31 - __builtin_clz(colsInRowTouched);

            currentPressure = scale1016to127(sensorCell->currentRawZ, true);
            
            // exclude the cell we just processed by flipping its bit
            colsInRowTouched &= ~(1 << touchedCol);
            if (pressureValueLo > currentPressure) {
               pressure = pressureValueLo;
            }
            else {
               pressure = currentPressure;
            }
          }
        }
        midiSendAfterTouch(pressure, channel);
      }
break;
  }

I've also tried:

Code: Select all

pressure = max(pressureValueLo, currentPressure)

No success yet.

[teknico]

Good news!

It's not done yet, but the dev environment is in place, I know what to change, and I'm confident that it will work.

I didn't recall that the updater does not allow you to choose the firmware file to upload to the Linnstrument, so much for using the Mac for avoiding calibration. I also didn't realize that you can just press the Global Settings button to skip calibration, so that's not a nuisance any more.

I enabled the debug prints by uncommenting the #define DEBUG_ENABLED in ls_debug.h. Then I had to change the choosing of debug levels in ls_settings.ino because, alas, my Linnstrument 128 does not have a column #17. But now I see the debug output on the serial monitor!

As far as the code goes, I finally properly understood that block that averages the X values on the same MIDI channel. The similarity between colsInRowTouched and colsInRowsTouched didn't help: spot the difference!

I am confident that we can use similar, but simpler, code for the Z and Y values without a significant runtime cost: the data we need are already in place. It should work in all modes, even MPE if you have multiple notes on the same MIDI channel (unusual, but still). And only the ls_handleTouches.ino file will be changed: you don't want to touch ls_midi.ino, it's not the right abstraction level.

I also had not realized that the Linnstrument is basically a weirdly sophisticated videogame console. I may try writing a Pong clone for it...

[Roger_Linn]

I didn't recall that the updater does not allow you to choose the firmware file to upload to the Linnstrument, so much for using the Mac for avoiding calibration. I also didn't realize that you can just press the Global Settings button to skip calibration, so that's not a nuisance any more.

Yes, it does. Place your newly compiled OS file in the same directory as the Updater app, which will cause the Updater to load it instead of its internal OS file. This is explained on the Source Code page (accessed from the LinnStrument Support page).

[ryanpg]

Awesome, teknico! I suspected ls_ino.ino was the wrong spot since the existing x averaging was done in ls_handleTouches.ino, thank you for figuring it out.

Look forward to seeing your code and trying to understand what it does!

[tommytea]

AUTO-ADMIN: Non-MP3, WAV, OGG, SoundCloud, YouTube, Vimeo, Twitter and Facebook links in this post have been protected automatically. Once the member reaches 5 posts the links will function as normal.

I've been testing out my Linnstrument with Infinite Woodwinds/Brass which is similar to the SWAM instruments in playability. With these instruments, I map pressure to CC1 which controls Dynamics. In real life, the dynamics of these instruments is controlled by the amount of breath the player uses and any changes in the amount of air is generally smooth.

Achieving this smooth change in dynamics is not possible with the Linnstrument. This is because each note's pressure always starts from 0 before ramping up, which causes the dynamics to jump around and makes it impossible to achieve smooth changes in dynamics. This would be similar to a wind player tonguing every note.

To solve this, I created a Bitwig Controller Script (https://github.com/tommyquant/linnstrument-smooth-pressure) to help smooth the pressure values between notes. It works like this:

- For non-legato notes, velocity will be used as the initial pressure value. The script will then interpolate to the actual pressure value over 100ms.
- For legato notes, the previous note's pressure will be used as the initial pressure value. The script will then interpolate to the actual pressure value over 100-1000ms. Velocity determines the speed of the interpolation. The higher the velocity, the faster the interpolation.

While I know you prefer to change the Linnstrument's firmware to achieve this and my solution works differently to what you want, I thought this might be a good starting point for you