Storing arbitrary amount of data in a document?

[Kirkwood West]

i'm looking for a way to store data in some sort of container in a bitwig document. I want to store knob positions for cursor device banks and some other custom data that the user generates by interacting with the device.

So far I found some work arounds to store this data into a clip. For example, I store colors for the midi fighter twister into a string '0,12,52,127' and write it to the clip name. So far I havn't found the limits to the amount of characters a clip name can have, so far so good... other than the gui responding VERY slowly when the clip name is large.

So it makes me concerned for the clip_name itself, especially if i want to expand preset slots per clip. Which leads me to the next thing i'm doing. I'm also working on a preset system to recall parameter values but for each slot. And then assigning a knob to morph between those parameter values. This it would require a bit of data. I've been able to save 80 or so parameter values into a clip. but that is only one preset. If I go for 8 presets per clip at 80 parameters thats 640 values. Each value represented by 4 characters(including the seperator) its 2560 characters total. I could reduce this by encoding in HEX which would eliminate the need for a seperator and each value would be respresented by 2 characters. So it would reduce it down to a total of 1280 characters.

I hate to reference Ableton on this forum or any discussion about bitwig (shame on me) but they have a method of storing a dictionary with each live set. It can be helpful for things like this.

Code: Select all

Live.Song.Song.set_data() Method
set_data( (Song)arg1, (object)key, (object)value) -> None : Store data for the given key in this object. The data is persistent and will be restored when loading the Live Set. C++ signature : void set_data(TPyHandle<ASong>,TString,boost::python::api::object)

Live.Song.Song.get_data()...

I was thinking about using the TrackSettings class but its data types are limited in this regard and I don't think that fits the model.

I could also see something like this useful to maintain state of a controller in a bitwig document.

In my excited states of getting to some sucess with the scripts I put the videos up on instagram. Might help show better what I'm attempting to achieve.

Twister Color Clips
https://www.instagram.com/p/B_ipcqAJwkh/

Parameter Recall + Morph
https://www.instagram.com/p/CDzaEbkpag6/

On this tip, once my scripts are done I want to generalize the Preset Recall and MidiFighter Color modules so people can implement their own versions of them. They are quite custom to my controller setup but maybe with some help from Moss or some others we can figure out a way to generalize them for others to leverage out of the box or with some minimal code. For now they are in pretty rough shape and can be seen here...

https://github.com/kirkwoodwest/worm-bi ... Centrifuge

[moss]

The official way is to use DocumentSettings.You could use a texfield and hide it.

[Kirkwood West]

ok. that works. would be nice to be able to store an array in the document settings. I may try to fake one somehow...

[Kirkwood West]

I'm looking for the method to hide the class... but now I realize how I get confused every time when looking through the documentation. Where is the hide() method? How do I know this method exists?

Here was my process for attempting to figure this out but I'm still not sure.

First I navigate to the DocumentState to see the different types of settings available...

Screen Shot 2020-08-13 at 11.45.03 AM.png

Then I look at the settings Interface

Screen Shot 2020-08-13 at 11.44.18 AM.png

Then I look at the SettableString object...

Screen Shot 2020-08-13 at 11.45.32 AM.png

I'm still not finding the hide() function, i'm not sure why. But you said it exists... so I must find it. I decide to search for it! bingo... but still do not know how its connected to the original object. I can make up in my mind how it works, write some test code but thats a bit of trial and error.

Screen Shot 2020-08-13 at 11.44.27 AM.png

So I must be doing something incorrect here. Is there a better way?

[moss]

Hehe, this is well hidden

---- SPOILER -----

....

....


The objects you fet from the getXXXSetting also implement the Setting interface.

[Kirkwood West]

got it... but what im getting at is how can I know this? i appreciate the knowledge and answering questions. you have been so generous and helpful...

i am bummed because i feel totally reliant on you to find out information like this. I feel incapable and want to leave you free to develop your scripts. i would love to have the ability to find this “hidden” information, but do not know how it is possible within the api framework & api manual.

[minortom]

+1 to both of those laments. On the bright side it is visible and discoverable in the API code (just need to proactively inspect and look in the IDE). Can extension code access files? That could be a halfway solution, create a separate storage bucket somewhere in $HOME (Documents/BWS/Extension storage ?), keyed by project.

Bitwig developers, give us custom data storage please! I'd love to someday implement Maschine's snapshotting functionality on the JAM, definitely need that.

[Kirkwood West]

Bitwig developers, give us custom data storage please! I'd love to someday implement Maschine's snapshotting functionality on the JAM, definitely need that.

I suggested this to bitwig. I would wish that you would do the same.

hehe... somehow you know exactly what I'm trying to do here. I've been able to snapshot the parameters of all my cursorDevices and store them as a string into a midi clip which is sweet. but its only about 100 parameters total... would be pretty slick to store it in a Hashmap pair with a key and then a referencce to the stored parameter in the live set.

I also do this to store color settings on the midi fighter twister. I assign one of the side buttons to edit the colors and you can spin the knobs and the colors change. then when you press the edit button again it stores them into the playing clip on a designated track. then when the playing clip changes it reads the data from the clip name and updates the colors.

[minortom]

Bitwig developers, give us custom data storage please! I'd love to someday implement Maschine's snapshotting functionality on the JAM, definitely need that.

I suggested this to bitwig. I would wish that you would do the same.

This is labeled as a "company forum", so I thought I did.

hehe... somehow you know exactly what I'm trying to do here. I've been able to snapshot the parameters of all my cursorDevices and store them as a string into a midi clip which is sweet. but its only about 100 parameters total... would be pretty slick to store it in a Hashmap pair with a key and then a referencce to the stored parameter in the live set.

I also do this to store color settings on the midi fighter twister. I assign one of the side buttons to edit the colors and you can spin the knobs and the colors change. then when you press the edit button again it stores them into the playing clip on a designated track. then when the playing clip changes it reads the data from the clip name and updates the colors.

I also have an MFT, by the way, haven't even gotten around using it with BWS yet.

Here's a crazy idea: there is a ton of data you can store in the clip itself, as notes and CC values. Digital modem! It will be like those old video games on cassette tapes, with a modern twist.

[moss]

I suggested this to bitwig. I would wish that you would do the same.

This is labeled as a "company forum", so I thought I did.

No, see here:
viewtopic.php?f=259&t=420129

[Kirkwood West]

Bitwig developers, give us custom data storage please! I'd love to someday implement Maschine's snapshotting functionality on the JAM, definitely need that.

I suggested this to bitwig. I would wish that you would do the same.

This is labeled as a "company forum", so I thought I did.

Yeah like moss said... its not unfortunatly. I think they come check this forum out from time to time based on some of the fixes the've implemented but certainly not their primary source of user feedback.

I also have an MFT, by the way, haven't even gotten around using it with BWS yet.

I want to unpack my massive controller script to a simplified script so others can use the coloring... Just have to learn how to integrate my class packages between multiple extensions. As I don't think half of the stuff I'm making is really that useful to anyone else.

Here's a crazy idea: there is a ton of data you can store in the clip itself, as notes and CC values. Digital modem! It will be like those old video games on cassette tapes, with a modern twist.

woah dude... not so crazy. I mean we are trying to go deeper with the API than just use a standard control script! At least that my my goal... this is what attracted me to bitwig is the custom api and the way it handles devices and mapping. Clyphx for Live is pretty sweet in that regard but is quite slow and doesn't perform very well. So here I am connecting 6 MFTs, Behringer XTouch, Novation Launch Control XL and Cirklon all together to create a mega control surface like Voltron... Something about it makes it more flexible than ableton live, plus all the other features like grid, modulation and the instrument selector that come with Bitwig Studio smokes Live.

I think you may have just found the golden nugget right there! certainly there is a lot of data you could store... I think you'd want to store parameter data as Float/Double though as every knob is high resolution. I'm not sure how you can write cc data in the NoteStep object, but there are also other values like velocity, release velocity, timbre, pressure. Looks like you could have the same notes on each individual channel, and you could have a note on each step which would be like 127 notes. yeah lots of data right there, might have to try it for this purpose.

Screen Shot 2020-10-05 at 12.29.08 PM.png

I'm just trying to finish my script conversion from .js to Java so I can get back to making some music for once. But will have to look at packing/unpacking data in this way, so I could actually use clip name for something useful.

[minortom]

No, see here:
viewtopic.php?f=259&t=420129

Thank you for clarifying.

woah dude... not so crazy.

Haha, I'm glad you like it. I'm not sure MIDI content is programmatically accessible, not finding anything in the docs.

Here is another slightly saner one and even somewhat achievable. I don't know if there is an upper limit on the number of modulators a device can have, but it's definitely a lot. All of them are recordable as automation. Every macro knob is a double value. So, any device with a bank of N Macro-4 modulators is technically a vector of programmatically accessible N*4 double values, storable in a clip as automation.

[Kirkwood West]

Haha, I'm glad you like it. I'm not sure MIDI content is programmatically accessible, not finding anything in the docs.

Here is another slightly saner one and even somewhat achievable. I don't know if there is an upper limit on the number of modulators a device can have, but it's definitely a lot. All of them are recordable as automation. Every macro knob is a double value. So, any device with a bank of N Macro-4 modulators is technically a vector of programmatically accessible N*4 double values, storable in a clip as automation.

This actually sounds like it would require the set up of many many macros!

Maybe check out the Clip object and this observer...
void addNoteStepObserver ( NoteStepChangedCallback callback )

Looks like that should update when a different clip is selected in the ClipLauncher and then you have access to all the note data in the clip. I can't recall how to set up the observers on clips since they don't exist when the controller interface is initialized...

[Kirkwood West]

Haha, I'm glad you like it. I'm not sure MIDI content is programmatically accessible, not finding anything in the docs.

Got some of that data stored in midi notes now. Thank you!! This was such a good idea man, the code and everything is soo much cleaner than what I was working with before because getting and setting the name of a clip is pretty much two different things according to Bitwig.

Here it is in action...
https://imgur.com/a/MC64vzY

I might consider NOT storing the data into the NoteStep.y parameter and trying a different slot because apparently y is only ranged from 0 - 126! i wonder why because I'm pretty sure the midi spec is 0 - 127 but bitwig says 127 is out of range. "Parameter y(=127) must be in range 0 - 126"

[moss]

I might consider NOT storing the data into the NoteStep.y parameter and trying a different slot because apparently y is only ranged from 0 - 126! i wonder why because I'm pretty sure the midi spec is 0 - 127 but bitwig says 127 is out of range. "Parameter y(=127) must be in range 0 - 126"

I guess you have set the 2nd parameter of "host.createLauncherCursorClip" to 127 but you need to set it to 128.