robust identification of CursorDevice change

[gc3]

Hi all! I've been lurking here for a little while but this is my first post (and my first evening of Bitwig extension coding!)

I'm running some experiments to see whether Bitwig is suited for some adaptive controller ideas I've been working with (more about these later, if the experiments come to fruition...)

Thanks to @moss 's REALLY excellent tutorial videos, and the general good design of the Bitwig API, I've been able to get up and running quite painlessly despite not programming in Java since university, so many thanks for all of that! But I've hit my first wall and before I start searching through tons of DrivenByMoss code I thought I'd ask around to see if I'm missing something simple.

Experiment 0 for my use case is to (A) robustly identify when the selected device is changed by any means and (B) print out some information about the newly selected device. (B) is pretty simple; I define a .deviceChanged method that prints out deviceType().get(), name().get(), channel().name().get(), and a few other things I've marked with .markInterested() or am otherwise observing. But I'm having trouble getting (A) to handle some corner cases.

I'll start by defining what I mean by "robustly identify". Crucially, the .deviceChanged method has to fire whenever the device is changed by any means; it can never be missed. Preferably, it wouldn't be called more than once per selection change. Ideally, interface changes (the deletion of an earlier device without the selection of that device) wouldn't call it, but a few unnecessary calls aren't a dealbreaker; they just shouldn't happen on ordinary selection changes.

If each device has a a unique instance ID that could be observed, this would be a one-liner (so I hope I'm just missing something obvious!) If there were some non-observable way of deriving it from PinnableDeviceCursor, it would be more complex, but probably do-able (basically, identify all the ways in which selections can change, watch for them, and store a prevID which gets compared to the newID).

But this seems like it might actually be hard to do properly in an extension. Bitwig obviously has some internal unique value which identifies a given instance of a device, no matter where it is or what happens to it, but I'm having trouble getting the API to follow along. After some experimentation, and a few blind alleys (cursorInsertionPoints seem to compare as equal; createEqualsValue can't be called outside of init(); etc.) I realized that you can safely downcast a PinnableCursorDevice.channel() to (Track), letting you call .position() on it. (The docs say that .channel() can also return a CursorTrack, which may need some thought, but I'm not worrying about that now). Observing this, plus the more obvious PinnableCursorDevice.position(), tracks changes in the y and x coordinate pair that should uniquely identify a device. And I think this would work--if it weren't for editing!

The problem with the y/x method is that if we delete from anywhere except the end, the device that was to the right now occupies the same track (y) and device list position (x), so
Inserting to the left of the current device creates the same sort of problem--now it's the new device that is in the same position. So we miss device updates.

Firing .deviceChanged on, say, name changes would catch most of these, but deleting the first of two instances of the same device (or inserting a new instance of the same device next to an existing instance) would still get missed. Also, the observers fire separately (and I don't know enough about the event loop to effectively consolidate this), so linking to name change now creates double triggering every time the selection changes between two devices with different names.

Similarly, downcasting PinnableCursorDevice.channel() to (Track), creating a trackDeviceBank, and observing its .itemCount() seems to handle the insertion and deletion case. However, now there's double triggering every time we change between channels that have different numbers of devices in them. Also (and this is a problem for the other methods as well), replacing an existing device with a new device of the same type should constitute a device change. With this method, though, even replacing an existing device with a new device of a DIFFERENT type gets missed, and deviceChanged won't fire unless you move to a different device and back. That probably is a dealbreaker.

(EDIT: Also, deleting the currently selected channel breaks this as well; if the channel below has at least as many devices, deviceChanged won't fire, since the device below the current selection will simply slide up into the current position.)

So, if anyone has read this far (which, many thanks!!!) do any obvious solutions come to mind? Or do I need to make an API feature request and (for now) try to combine a bunch of different device-changed checks in a way that can handle a bunch of repeated calls (and which even then won't catch everything?)

[Kirkwood West]

Completely agree with you there. Same issue happens when selecting a track with cursor track in a a group. The cursor position is relative to the parent group. So switching between between the first track in two different groups makes CursorTrack.position() report 1. Which doesn't change so there is no report. I use both position and name() to cross reference... but that doesn't work with devices since they tend to share a lot of the same names... multiple instances etc. can create a bad user experience if you run across an edge case.

It would be great if each channel and device had a permanent guid that is saved with the document that we had access to and also observable from banks to any cursor types. Would solve a lot of this mess and open the door for a lot of stronger functionality within an extension.

[minortom]

So, if anyone has read this far (which, many thanks!!!) do any obvious solutions come to mind?

I feel your pain!

I found a way to get stable IDs at least for tracks, and I wouldn't be surprised if it worked for devices too. The workaround is a hack and relies on Java reflection (and a hope that they won't reengineer the underlying implementation anytime soon, especially after reading this) and goes like this:

* Get the Track object (not the PinnableCursorTrack but a Track you get from bank.getItemAt())
* Using reflection, call .getTarget(), this will return the underlying object the track proxy is pointing at
* Using reflection, look for a method that returns a UUID (in Track's case there is only one, otherwise you'd have to guess which one you need)
* Call that method
* PROFIT

Full Scala implementation for reference (I'm using UUID hashes as identifiers to save space, but it's not required):

Code: Select all

case class TrackId(value: Int) extends AnyVal
...
  override inline def trackId(track: Track): Option[TrackId] =
    val pos = track.position().get
    idForPosition(pos)

  override inline def idForPosition(pos: Int): Option[TrackId] =     
    if (pos != -1)
      val st: Track = bank.getItemAt(pos)
      idForBankTrack(st)
    else None

  override def getItemAt(id: TrackId): Option[Track] =
    LazyList.from(bankRange).map(bank.getItemAt).find(t => trackId(t).contains(id))

  private def idForBankTrack(st: Track): Option[TrackId] =
    val uid = for {
      tMethod: Method <- Option(st.getClass.getMethod("getTarget"))
      tObj: Object <- Option(tMethod.invoke(st))
      idMethod: Method <-
        Option(tObj.getClass().getMethods())
        .filter(_.getReturnType().equals(classOf[UUID]))
        .headOption // let's hope there is just one
    } yield idMethod.invoke(tObj).asInstanceOf[UUID]

    val id = uid.map(_.hashCode())
    id.map(TrackId.apply)

[gc3]

Thanks, both, for the extremely quick and helpful replies! I really appreciate it. Sounds like I'm far from the only one who's bumped up against this...

Same issue happens when selecting a track with cursor track in a a group. The cursor position is relative to the parent group. So switching between between the first track in two different groups makes CursorTrack.position() report 1. Which doesn't change so there is no report.

Argh--the problem is all over the place And yeah, relying on names seems like it would get frustrating pretty quickly. I noted, for example, that duplicating a track doesn't even add a qualifier to the end.

It would be great if each channel and device had a permanent guid that is saved with the document that we had access to and also observable from banks to any cursor types. Would solve a lot of this mess and open the door for a lot of stronger functionality within an extension.

Totally agree. I'm going to file a request (with some motivation for the application) and cite this thread for reference, so if anyone else wants this, maybe consider posting the specific need here and also filing a request? (I wonder if Bitwish has an API tag... too niche for them?)

I feel your pain!

I found a way to get stable IDs at least for tracks, and I wouldn't be surprised if it worked for devices too.

{extremely helpful example of method}

OK, this (1) is completely brilliant; (2) should not be necessary I will try it for devices and report back. Good news is that it suggests that what @Kirkwood West is postulating may already exist, just not in the API. And, seriously, nice catch!!!

(and a hope that they won't reengineer the underlying implementation anytime soon, especially after reading this)

[minortom]

OK, this (1) is completely brilliant; (2) should not be necessary I will try it for devices and report back. Good news is that it suggests that what @Kirkwood West is postulating may already exist, just not in the API. And, seriously, nice catch!!!

Thanks And yes, it should be fully exposed.

What @Kirkwood West is saying does correspond with my own research: channels (and probably devices) do in fact have UIDs that are saved in the document, there just isn't a public way to access it via the API. This reflection workaround lets you access that UID.

[gc3]

Thanks

No, thank you

What @Kirkwood West is saying does correspond with my own research: channels (and probably devices) do in fact have UIDs that are saved in the document

Ah, good--they're saved too, and not regenerated on load. Great! I haven't had a chance to poke yet but I'd be amazed if this applied to channels and not devices, etc.

I've put in the feature request, which is basically to allow something like:

Code: Select all

cursorDevice.getGUID().addValueObserver(newValue -> { this.deviceChanged(cursorDevice); });

Which raises the final question (for my use case) unless/until the API gets this feature--is there a way to make the UUID value returned by the mirror-found UUID-returning method into something observable, presumably by casting it to a StringValue and then somehow registering it manually in the observation table? (No idea if the internals of the observer pattern are exposed or themselves discoverable). Still useful even if not!

[gc3]

(and probably devices)

@minortom, I devolved your code into Java (much nicer in Scala!) and applied it to Device instead of Track. Initial failure, with no methods returning UUIDs, but (after a long exploration which I won't bore anyone with) I noticed that there was a getDeepestTarget() method on the Device (in addition to the getTarget() which works for Channels. Using getDeepestTarget() instead, I got success!

It looks like getTarget() on a device returns a container object (obfuscated class v2J) which holds multiple device types but doesn't contain the methods we care about. getDeepestTarget(), on the other hand, returns the underlying Bitwig (pxB), VST3 (KXy), or VST2 (NUk) device type (I assume CLAP, etc. get their own but I'm just checking these three for now). And pxB, KXy and NUk all contain the aQ_ method, which seems to return the instance UUID. Got it!

Now, this doesn't entirely solve my initial problem (there are still cases where no exposed observer will detect a difference in selection, so nothing will fire) but I should be able to filter out multiple pings on the same device using the instance UUID, which would mean that I can use a bunch of observers without worrying about repeats. That's a big improvement, though I'm not sure it can get any better until the instance UUID is made observable. Maybe there's some other more UI-focused way of detecting selection changes. I'll look...

But wait--there's more! Excitingly, Bitwig native devices have *two* functions which return UUIDs. One of these seems to be the instance UUID, and the other seems to be the device-type UUID. I know from @moss's API 12 video and the associated documentation that Bitwig native devices have UUID type identifiers, whereas VST2 has ints and VST3 has strings. I am nearly certain that with a little experimentation I'll be able to find the relevant methods on KXy and NUk (these will be what the "Copy Device Id to Clipboard" command calls, as appropriate). That's good news because it will allow getting a SpecificPluginDevice from a CursorDevice (see viewtopic.php?t=584998). As it happens, that was going to be my very next experiment, and even using reflection is nicer than the first idea I had (manually putting the identifier into the name of a device page, yeesh). Once I've lined that all up I'll put code in this thread. (BTW, I'm not sure if the obfuscated sub-API names are consistent from version to version or even platform to platform, so I'll include the search code as well).

edit: well, there was yet one more level of indirection; on a NUk (VST2 holder), call F21 to return a ZwW (VST2 instance), then call its .pqa() to get the VST2 integer ID, and on a KXy (VST3 holder), call F21 to return a YqA (VST3 instance), then call its .pqa() to get the VST3 string ID. These all match to the "Copy Device Id to Clipboard" (and, by the way, there's other good stuff in the ZwW and YqA, even right at the surface, including the plugin version and what I think is a parameter count). But we can now take a CursorDevice and get instance IDs AND device type IDs for native devices, VST2s, and VST3s.

I'll work up a code example soon and add it below.

[minortom]

I noticed that there was a getDeepestTarget() method on the Device (in addition to the getTarget() which works for Channels. Using getDeepestTarget() instead, I got success!

It looks like getTarget() on a device returns a container object

Later you mention CursorDevice several times so I'm assuming this is what you're working with. This is why in my example I specifically ignore anything that might be a Cursor and instead get the Track object directly from a bank — because I couldn't find an easy and easy way to get the right object from a cursor, which you did with getDeepestTarget(). Nice find!

BTW, I'm not sure if the obfuscated sub-API names are consistent from version to version or even platform to platform

My thoughts exactly. This is why I avoided delving too deep into deciphering and then depending on the obfuscated stuff and am glad to have found a way to stay at the surface level, where names and structure are least likely to change.

This is a brilliant write up and a commendable level of effort. I can confirm the process and findings closely match my own.

Now, this doesn't entirely solve my initial problem (there are still cases where no exposed observer will detect a difference in selection, so nothing will fire) but I should be able to filter out multiple pings on the same device using the instance UUID, which would mean that I can use a bunch of observers without worrying about repeats.

You should consider hopping onto Discord (https://discord.gg/dzkjwmR6, #controllerism), @kirkwood and I were discussing this very problem yesterday. It's true that a cursor will not tell you when its target changes and there are other trigger sources where hopefully one of them gives the correct indication, though you would need to muck with deduplication.

Maybe there's some other more UI-focused way of detecting selection changes.

There is for tracks, actually, if UI-driven selection is what you need. I have a similar problem of needing to know which clip has been selected, so I have this in my init code:

Code: Select all

  val superBank: TrackBank = ext.host.createTrackBank(256, 8, 256, true)

  def selectedObserver(track: Int): IndexedBooleanValueChangedCallback = (idx: Int, selected: Boolean) =>
    if (selected)
      ext.events.eval("selectObserver")(GlobalEvent.ClipSelected(track, idx))

  (0 until superBank.getCapacityOfBank()).foreach { i =>
    val t: Track = superBank.getItemAt(i)
    t.clipLauncherSlotBank().addIsSelectedObserver(selectedObserver(i))
  }

Basically, wire every possible clip with a selected observer (ext.events is my own little event bus). I see Device has addHasSelectedDeviceObserver, if that works the same way then you could do the same with a large device chain on every track (though it will be significantly less trivial if you're interested in nested devices and not just top-level ones).

[moss]

I can also only strongly advice against using internal APIs. Especially, obfuscated names can change with a new build.

For API wishes write directly to Bitwig support.

I did not fully get what the original poster is actually trying to achieve
If you e.g. want a specific hardware controller work as a e.g. specialised Compressor controller you can create a device filter which only looks for this (these) device(s) on the cursor track.

[minortom]

I can also only strongly advice against using internal APIs. Especially, obfuscated names can change with a new build.

For API wishes write directly to Bitwig support.

I wrote to support about this in June of last year (106328) and am still waiting.

It's true no one should be using internal undocumented APIs, in theory. In practice, though, between being able to get stuff done today with a 20 minute debugger dive and waiting a year for official support I choose the former.

I did not fully get what the original poster is actually trying to achieve

I can speak for my use case (I believe it's quite similar), this is about being able to maintain controller session data that's attached to each individual track/device instance (e.g. in my step sequencer it remembers settings like step size per track, and not by position because that's unreliable).

P.S. I did submit another request (141510), hopefully the signal will get amplified.

[moss]

I can also only strongly advice against using internal APIs. Especially, obfuscated names can change with a new build.

For API wishes write directly to Bitwig support.

I wrote to support about this in June of last year (106328) and am still waiting.

Don't worry, I have currently about 30 API wishes piled up, several since Bitwig 1.0
But there is always stuff happening if you look at what was added since 1.0 and many more people are always crying for other things than API extensions...

[gc3]

@minortom, many thanks for your kind words, continued help, the Discord invite, and the additional pointers regarding addHasSelectedDeviceObserver! It's all very much appreciated. I'll look at your selection tracking code when I sit down with this next and will definitely swing by the Discord when I have a little time!

I did not fully get what the original poster is actually trying to achieve
If you e.g. want a specific hardware controller work as a e.g. specialised Compressor controller you can create a device filter which only looks for this (these) device(s) on the cursor track.

Hi @moss! OP here. I love your work!

What I'm trying to do right now is very simple (much less sophisticated than @minortom saving setup data on individual channels):

1. Whenever a new device instance is selected (by any means--clicking on it directly, due to a track change, with keyboard shortcuts, via a hardware controller), I want to call a method.
2. In that method, I want to get information about the newly selected device, especially its Bitwig UUID or VST2/3 specific ID, so that I can get a SpecificPluginDevice for it.
3. With the UUID or equivalent, I can look up separately defined metadata about the device's controls.
4. I can then dynamically bind arbitrarily large numbers of parameters more than eight controls onto a next-gen controller (varying position, controller behaviour, color, etc.) and write arbitrary adaptations between the controller's controls and the plugin (turning, say, six mutually exclusive buttons with separate CCs into a six-position switch, or vise versa).

So the concept is to let an advanced general controller (right now, I'm experimenting with a custom Yaeltex, the Erae Touch, and a Faderfox EC4) get close to the per-device customization of something like the Soundforce controllers, but for as many devices as the user wants to define. So it's like a much more flexible version of device remote controls, but with the same soft takeover.

In your API 12 example, you show how to build a (say) Neoverb controller that takes over an instance of Neoverb if it finds one on the track--and I think that's what you're postulating with a specialized compressor controller as well. I'd think of that as a "pull" process. Notably, it doesn't depend on the actual UI cursor selection; Neoverb (or whatever) is essentially selected separately, on the device.

Nothing wrong with that! But I want a "push"--whatever device gets selected in the actual UI is sent out to the controller, which gets autoconfigured with advanced adaptive behavior. It really is like device remote controls, but accessing the plugin parameters directly and with a much richer (and therefore, obviously, slower and more complicated) way of defining what the controls are and do, beyond just "name/order/page name/which page" afforded by the remote controls.

Other stuff, like pinning controls or selecting devices from the controller, would be for later; right now I just want a 1:1 map between current Bitwig device and hardware controller, but with arbitrarily complex interactions between the two.

edit; just to assess progress:

For (A) I've gotten to about 80% with defined API observer stuff; it fails on (lots of) corner cases, though, especially if I don't want to call the method repeatedly. With undocumented access to the instance UUID, I can define more observers and dedupe (just as @minortom notes). With the ability to observe the instance UUID, which is what I've requested, this would be trivial. But is there another, better way?

For (B), I think the only way to do this right now is with undocumented methods (exactly as discussed over here: viewtopic.php?t=584998)--unless I were to go with my original idea of manually placing the device IDs onto a device remote control mapping page with some sort of token for ease of recognition, which is obviously clunky but which I think would work.

For (C) and (D), which are the interesting parts, I don't currently see any limitations with the defined API, but I'm sure I'll have questions and possibly problems as I start to implement them!

(By the way, @moss, your caveats about the undocumented internal API are well taken, but I also want to keep coding, especially because I need to find out if Bitwig's internals will support what I want. Right now, this is just for me, so I can re-discover the obfuscated names if they change; to be generally useful this stuff would of course have to move into the official API).

[moss]

So the concept is an advanced general controller (right now, I'm experimenting with a custom Yaeltex, the Erae Touch, and a Faderfox EC4) to get close to the per-device customization of something like the Soundforce controllers, but for as many devices as the user wants to define. So it's like a much more flexible version of device remote controls, but with the same soft takeover behavior.

In your API 12 example, you show how to build a (say) Neoverb controller that takes over an instance of Neoverb if it finds one on the track--and I think that's what you're postulating with a specialized compressor controller as well. I'd think of that as a "pull" process. Notably, it doesn't depend on the actual UI cursor selection; Neoverb (or whatever) is essentially selected separately, on the device.

Yes, that's exactly what I suggest. Since for your idea you need to know anyway which devices you support it does not matter if you have to configure it at startup. So, do the following:
- create a device matcher (to match the UUID) for each of the specfic devices you want to support
- create a device bank of size 1 (or more if you want to deal with several devices of the same type on one channel)
- In your controller code check which of the items (devices) in the different banks do exist and activate your specific device mapping accordingly.

I use this technique already for the EQ+ support with MCU and the LaunchControl XL.

[gc3]

Yes, that's exactly what I suggest. Since for your idea you need to know anyway which devices you support it does not matter if you have to configure it at startup. So, do the following:
- create a device matcher (to match the UUID) for each of the specfic devices you want to support
- create a device bank of size 1 (or more if you want to deal with several devices of the same type on one channel)
- In your controller code check which of the items (devices) in the different banks do exist and activate your specific device mapping accordingly.

I use this technique already for the EQ+ support with MCU and the LaunchControl XL.

Thanks so much for thinking about this! I still believe that I'm asking for a different behavior than what you describe, but I may well be missing something or just miscommunicating. I'll look at how you implement EQ+ (and how you deal with remote control mapping more generally).

I think the main difference is that I want the currently mapped device to reflect what the user has selected in Bitwig, not an arbitrary instance of a particular device on a specific track. Also (less importantly) I want to map *all* devices; if there's not metadata, I want to derive sensible defaults.

I should probably just hack this together and link to a video once I've got it roughly working, but let me lay out an example of how what I think you're proposing would work vs. what what I think I want would work.

Track 1: Synth A => Neoverb
Track 2: Synth B => Neoverb => Neoverb

In your application (as I understand it), if Track 1 is selected and there's a definition for Neoverb, the controller would map to that instance even if Bitwig's UI has Synth A selected. Then if Track 2 is selected, it would either map to the first instance of Neoverb, or have some way of paging between the first and second instance from the controller.

In my application, I never want (at least for right now) the controller to be mapped to something other than what the user has selected in Bitwig. So if Track 1 is selected and Bitwig auto-selects the first device, the controller gets Synth A; if there's no special mapping defined, it uses defaults. Only when the user clicks over to Neoverb does the mapping update. If Track 2 is selected, same deal; the user separately changes the device, and the controller follows.

So I guess my remaining questions are: (1) am I understanding your application correctly? (2) am I identifying and communicating a difference between the two applications? and (3) if so, for the second application, is there a simple and reliable way to call a method if and only if the currently selected device is changed by whatever means?

[moss]

Yes, that's exactly what I suggest. Since for your idea you need to know anyway which devices you support it does not matter if you have to configure it at startup. So, do the following:
- create a device matcher (to match the UUID) for each of the specfic devices you want to support
- create a device bank of size 1 (or more if you want to deal with several devices of the same type on one channel)
- In your controller code check which of the items (devices) in the different banks do exist and activate your specific device mapping accordingly.

I use this technique already for the EQ+ support with MCU and the LaunchControl XL.

Thanks so much for thinking about this! I still believe that I'm asking for a different behavior than what you describe, but I may well be missing something or just miscommunicating. I'll look at how you implement EQ+ (and how you deal with remote control mapping more generally).

I think the main difference is that I want the currently mapped device to reflect what the user has selected in Bitwig, not an arbitrary instance of a particular device on a specific track. Also (less importantly) I want to map *all* devices; if there's not metadata, I want to derive sensible defaults.

Ah, I forgot that part. I did not mean to use RemoteControls but instead use the parameters from the SpecificDevice, which you would also need to lookup in advance.