KVR :: DSP and Plug-in Development » SysEx using VST v2.4 [View Original Topic]
There are 35 posts in this topic.

---

grymmjack - Fri Feb 10, 2012 7:41 pm

---

Hi.

Considering getting into plugin coding using C++ but only really interested if SysEx is doable using VST v2.4.

What am I up against, and is this a fools errand?

Thanks in advance!

---

arakula - Fri Feb 10, 2012 10:45 pm

---

SysEx is doable... but there are quite some hosts that don't implement this feature, or only in a limited way (for example, there were Cubase versions which could only do SysEx with up to 16 bytes), so don't expect that your PlugIn will be universally usable.

---

grymmjack - Sat Feb 11, 2012 9:56 am

---

Thanks.

---

4damind - Sat Feb 11, 2012 10:24 am

---

All developers having a workaround because of this restrictions.

1.) Instead of sysex, transmit controller messages if the plug-in is something like a hardware controller/patchmanager. This works with devices with not too many controller data. But this needs eventually some routing for the user so the plug-in can send and receive controller messages.

2.) Only handle loading/saving of sysex files, the user must use another (standalone) tool to receive/transmit data.

3.) Implement a own sysex handling with some settings where the user can select the midi devices to use.

---

grymmjack - Sat Feb 11, 2012 7:37 pm

---

Thanks 4damind.

---

aciddose - Sat Feb 11, 2012 8:40 pm

---

if you have any option, it's a much better idea to go with NRPNs in almost every case.

even sending whole presets can be managed quite efficiently by setting up the NRPN (98, 99) and then sending out data to data entry (6). since midi has the ability to drop all data except the new 7+1 bits you send it means you get almost full bandwidth AND you have both parameter and channel selection.

it's also guaranteed to work with any device that will transmit CCs correctly, which includes pretty much everything.

sysex is, for the purposes of transmitting control information or blocks of data, pretty much obsolete.

---

grymmjack - Mon Feb 13, 2012 6:03 am

---

aciddose; Thanks. I don't have an option. I am already using CC's (and NRPNs) in Max4Live. I guess some context would help; I'm attempting to create VSTi/VST editors for my hardware gear, that don't suck and are complete to the spec including sysex.

Up until now I've been doing it without sysex, and I am missing out on some of the deeper editing that I am OCD and perfectionist enough about to be driven mad by it For example on microKORG, the delay type, on MS-2000 the swing value for the MSEQ# lines. I've got CC's and NRPN's already working in MAX, but because Ableton Live completely ignores inbound SysEx through it, and requires OSC, it's kind of a pain in the ass, hence going native might be best way to go. I have been getting into Renoise as well, and perhaps lua is an option too. Also Renoise has native OSC integration, which kind of simplifies that part if I need to make OSC client/servers for even my native plugins to work right. I know it can be done because novation did it with Ultranova, and I think Korg did it with the R3 or MicroX something or other. So it's just a matter of getting there

You can look here for my M4L work so far:

microKORG: http://www.kvraudio.com/forum/viewtopic.php?p=4532880

MS-2000: http://www.maxforlive.com/library/device.php?id=900

These dead end at sysex. I am aware of the OSC client/server work around in MAX, and I'm considering that as well, but for now what I'm thinking is just bite the bullet, make some MIDI VSTi plug-ins, and get right at the hardware, hence this thread topic.

I would like very much to emulate to the maximum degree the relationship I have with my hardware just as I do my software plugins. The convenience of software is too huge to miss, and the dedicated processing and sounds of external hardware is too worthwhile not to pursue.

So being glutton for punishment I'm thinking about it...

I've looked at FLStudio Dashboard, no SysEx support there. I've looked at MIDIQuest and SoundQuest, and those mostly work, but they aren't plug-ins. The plug-in version of those things I've heard is not perfect; and I'm of the opinion that I can do a good job emulating the look and feel to further enhance that native integration or total integration (if you're a virus owner, I am not) aspect of this pursuit. I've got quite a lot done on MAX4Live and I'm very happy with most of this work, it's just the little fiddly bits I cannot access driving me nuts. I've considered SynthEdit, and emailed Chris and he said that I could do some of this with SE but to create messages I'd need modules or something. I'm not fond of SynthMaker very much, even though I know FLStudio uses it quite successfully on the sysex front, I've never had good luck with those sysex SM plug-ins in FLStudio, or any SM plug-in except 1 or 2 (even when Atomsplitter and I made them and so I'd prefer to skip SM altogether). I know a little about CTRLR, but when I downloaded and used the MS-2000 plug-in, it was not able to reach some parameters either, which could mean that the designer of that panel didn't implement everything or perhaps CTRLR is unable to do sysex through their VST wrapper.

I know enough about programming to know that once the initial problems are solved, it should be not too difficult. What I'd like to do is perhaps start with my MS-2000, and then microKORG and get those editors built and perhaps sell them for a small profit. If the sales warrant it, as I extend my own hardware collection, I could then create other native plug-in editors for each I own.

I've been looking for a reason to get back into C++ and this seems like a great way for me. I have a very fixed goal, and very rigid path to get there which will hopefully keep me on task

Thanks for your help.

---

obiwanjacobi - Mon Feb 13, 2012 11:28 am

---

How will those plugins communicate with the hardware device? Aren't all midi ports locked by the host application the plugin is running in?

I also have some hardware that I would love to write some editing software for but the issue I described above, always kept me away from building a plugin.

MidiQuest never really worked for me. Always some error when loading information over MIDI into the program.

I'd be interested in how you're doing implementing these plugins.

BTW: If you ever consider building a generic editor (like MidiQuest), I have a prototype-idea lying around that could help...

---

4damind - Mon Feb 13, 2012 11:53 am

---

The midi device/driver must be support multi-clients without this it doesn't work.

---

Caco - Mon Feb 13, 2012 12:16 pm

---

Hey Rick, if you want to keep things simple then just build it all with Sensomusic Usine. Usine has a Sysex Editor and can even export your patch as an independent VST if you want

---

aciddose - Mon Feb 13, 2012 4:30 pm

---

you can also just route the output of the controller to the right port easily enough in most hosts, if not all.

that's what i do. it's easy of course to get drivers to wrap your hardware to allow them to support multiple clients and do mixing of the different ports as desired; it's easy to handle midi yourself and deal with events and so on directly; it's easier to just use the existing capabilities of the host that you need to be running in anyway.

for example, if you wanted to have a stand-alone version you could just write a mini vst host with input/output for midi and pre-route the plugin as desired.

this makes the most sense of any approach, it's modular and re-uses existing systems to the max while being compatible with the manual implementation with zero overhead.

---

grymmjack - Mon Feb 13, 2012 5:38 pm

---

Well as a plugin they would receive on MIDI channel sent, for example.

I am not trying to build a MQ type of thing but I'd definitely create wrapper code to make the stuff reusable. Thanks for your offer though.

I'm still learning what options there are. Usine is new one

---

grymmjack - Mon Feb 13, 2012 5:39 pm

---

Why? If its through the software VST would it not just use this and not need redundant parallel connection? This is how M4L works at least.

---

grymmjack - Mon Feb 13, 2012 5:41 pm

---

Caco long time friend

Can Usine created stuff be fully skinned up with custom controls? If so I'll definitely check that out. Part of the fun is the programming part is the design work

---

grymmjack - Mon Feb 13, 2012 5:47 pm

---

I am already doing this with some success in M4L the problem is the gear I own requires sysex. Live ignores sysex at input hence OSC work arounds.

The goal is a 2way connection manifest in a plugin. Wiggle knob on hardware wiggles software and software settings do vice versatile sans physical movements. I can achieve this with some effort in a rigged way using ,y M4L devices and midi map the knobs but Ableton does not remember these. I have worked around this by assigning cc and nrpn thru modules in M4L but it's kludge.

I want a plugin, routed automatically with a beautiful interface and named VST parameters for automation

---

aciddose - Mon Feb 13, 2012 7:14 pm

---

well the only really safe way to do that is to have your own interface. this is how for example versions of synthesizers such as virus ti accomplish it.

otherwise you're depending upon drivers anyway. either the system's midi drivers support multiple clients or they don't. you have no control over that. what you can do is add a midi wrapper which is provided by some tools such as midi ox.

the advantage is that of course you get the mixing/multi-client support ensured regardless of what midi hardware the user has. you also don't need to provide special hardware. the disadvantage is that the user is forced to manually configure their midi devices using your wrapper instead of the native interfaces.

of course you also have the option of leaving this to the user who is unlucky enough to have drivers that don't support multiple clients. you can also leave out support for hosts that are not sysex compatible.

since you want things to actually work i guess you're forced to implement the wrapper somehow and provide both direct access to the midi i/o and host i/o in your plugin.

i think this is why we don't see many successful attempts at these types of controllers. i've seen a lot of attempts but they've never become very popular.

it's a shame really that it has to be so difficult. i'm not sure why more hosts don't support sysex.

---

arakula - Mon Feb 13, 2012 10:15 pm

---

Presumably because their developers don't have the background that would make it a necessity. If you don't have external hardware devices that implement SysEx as the major means to control non-standard features, why bother? Just because some freaks want to do things outside the wonderful, feature-packed, overcomplete DAW control program of theirs? I mean, all an artist needs to have today is a laptop, right?

Just to make sure there's no misunderstanding - the above is deeply sarcastic; I'm owner of some 30 or 40 hardware synths myself, so I think it's a shame, too - and of course my hosts support it, both receiving and sending SysEx from VST(i).

---

Caco - Tue Feb 14, 2012 12:04 am

---

Yes the controls in Usine can be skinned with your beautiful art work
http://www.sensomusic.com/wiki/doku.php?id=manual:guide:tips:using_frame_bitmaps#step_cursors_bitmap

http://www.sensomusic.com/wiki/doku.php?id=manual:modules:sysex_editor

---

grymmjack - Tue Feb 14, 2012 12:58 pm

---

Awesome. Thanks very much for this tip.

---

grymmjack - Tue Feb 14, 2012 8:25 pm

---

Usine looks incredible. I love the mnml

@Caco; i've downloaded the pro demo but I can't export a VST/VSTi, do you own it and can export a sample one? I just want to see what kind of shell is around it. If it can be fully "private label branded" or if it's got some kind of Usine shell that's unchangeable, etc.

Thanks man. Regardless of this, you've opened a deep pit with the Usine recommendation. I had no idea it was this robust. Thought it was just a weird avant garde kind of toy, didn't realize it was like reaktor/max and could compile to VST/i.

---

Caco - Wed Feb 15, 2012 12:25 am

---

sent you a pm

---

grymmjack - Sat Feb 18, 2012 10:42 pm

---

OK so I'm trying to compile the Tutorial project with VSTGUI v4.0.1 and VST SDK 2.4.

I've got AGain compiling, and also VSTXSynth.

I've got my bearings on how things are setup sorta, and now I'm trying to take that next step and getting something to work with a real GUI interface.

I've got the tutorial compiled, but opening it in Cantabile, it shows absolutely no window or anything, just the toolbar.

From what I can see in the resources folder, there should be a knob or something. I'm seeing no window at all.

Any help is appreciated.

I'm using Visual C++ 2008 Express

---

mkdr - Sat Feb 18, 2012 11:47 pm

---

Doesn't sending one NRPN parameter take up 16 bytes compared to just 1 on sysex(inside a program dump)? Midi interfaces also drop CC's if there's too much of them coming through(run out of buffer).. you'll need a timed stream for sending all that data.


In any case, i've seen better results with sysex than with CC based patch changes.. and i've build dozens of patch loaders for synths. But it is a slippery slope as most hosts don't support sysex. The best approach in my opinion is having your own midi-port, independent of the host.

---

aciddose - Sun Feb 19, 2012 11:22 am

---

no, it only takes one byte. it depends upon the support your hardware has, but it's possible to output a stream of continuous bytes in a non-standard format which will be compatible with other midi devices. (they'll just ignore it.)

so what i do is output the destination code i want, then output a string of values for the entire block of data i want to send. the code on my device looks up the address and starts writing.

this method obviously isn't safe for every possible case, but it works fine. (actually i have no idea if this works with every driver and host... there is no reason it should but i'm sure it's possible assumptions were made about message length being = 4 byte in lots of places.)

sysex would be better if hosts supported it, because it works identically when you look at the uart stream.

---

mkdr - Thu Feb 23, 2012 5:00 am

---

I didn't quite follow.

If a midi message like you say is 4 bytes (status, channel and 2 data bytes), how would you get an NRPN that takes 4 of those messages to be sent with just one byte?

Or do you mean to send each "value" as one of the midi message bytes? In that case you are basicly doing sysex, but out of spec. And in that case the driver, the interface or the recipient could just drop a few if there's too much traffic. So, worst of both worlds. I don't see the point.

---

arakula - Thu Feb 23, 2012 10:04 pm

---

A MIDI message in VST 2 is defined as an event that has a 4 byte data area for the MIDI message. Presumably defined this way since it can be casted to a 32-bit integer (at least in Windows, this is the format used for normal MIDI messages). That, however, has nothing to do with the MIDI messages that are sent to and from the device drivers; here, normal MIDI messages take up 1 to 3 bytes, depending on their type and Running Status.

To set up an NRPN, you need to send 2 messages:
1) a 3-byte controller message for the higher 7 bits of the number
2) another 3-byte one for the lower 7 bits of the number; assuming there's no other MIDI message in between, this is shortened to 2 bytes because it's got the same status byte.

Then, you need to send data bytes. Here, I don't follow aciddose's logic that they would take up only one byte. A message sent using Data Entry is a 3-byte controller message. The status byte may be optimized away by Running Status, but the second byte, which defines the Data Entry controller, and the third byte, the real data, must be sent, so it's always 2 bytes for one data byte. Or 3, if a non-realtime MIDI message with another status byte was sent in between.

So, sending an NRPN message needs a minimum of 7 bytes for 1 data byte, and 2 more for each following data byte. That's considerably more than a SysEx, which normally has a fixed size of 3 or 5 (F0, 1 or 3 bytes manufacturer ID, F7), plus how many bytes it needs to specify the command (implementation detail), plus the data bytes themselves, each taking up 1 byte.

Aciddose's "1 byte" can only be achieved with a MIDI device which uses specific code that violates the MIDI standard, together with a matching custom device driver on the computer side that violates it in the same way. In this scenario, it's possible.

---

aciddose - Thu Feb 23, 2012 10:22 pm

---

i was going to post that yes, that's obvious.

i was talking about sending packets longer than three bytes, but the hardware would need to support that. it really just isn't practical.

so it's a 1/2 ratio on long packets assuming you use n/rpn and with sysex you also have the nasty overhead of the id components.

it would be easier to deal with if midi allowed for packing any number of bytes using the msb as the flag to end the stream.

well, it does, but if you had the ability to ignore the standard you could be using sysex anyway.

the bright side of n/rpn though is that once you set up the destination it's possible to leave it set (don't write 127/127) and keep sending data-entry. so if you're manipulating one parameter it does actually become only 1/2.

it would be possible to send single bytes like i suggested if after a data-entry command, the device didn't clear the status byte or first data byte. (call this uber running-status or lsd mode)

where would this break the standard? if the next message were another CC the byte would be expected to write into data1, not data2 followed by handling the message. it would work fine as long as data-entry were always followed by another byte with the msb set - that is if running status were not used immediately following data-entry.

that's essentially how sysex works - once the codes are set data is read continuously until the end of stream flag.

so actually it would make sense to implement as part of the standard, it's just a little too late now i guess. ...and you already have sysex.

---

grymmjack - Thu Feb 23, 2012 10:28 pm

---

aciddose: I'm stuck with whatever the hardware mfg's are using man.

Have you guys done much with VSTGUI 4? It seems like it's very robust, cross platform, etc. (i'm talking about the googlecode vstgui project).

Thanks for keeping this thread alive.

---

arakula - Thu Feb 23, 2012 10:33 pm

---

Running Status only keeps the Status byte, not the first data byte. Never. This would violate the standard.

If you look at the special case where you would want to transfer a data byte containing 0x06 or 0x26, you'll see why your idea is not going to work. These are the IDs of the Data Entry Coarse and Fine controllers. How should the device find out whether these are data bytes, or a new controller ID with the same status byte?

---

aciddose - Thu Feb 23, 2012 10:39 pm

---

did i add that all in an edit? lsd-mode indeed.

i meant to post it all together. it was a question to myself actually and i did go through how it would break the standard if other devices didn't implement it the same way.

we have a lot of extra cc values left undefined. it's possible special "uber running-status" modes might be defined this way in the future.

i would like it if we had this ability to send larger packets (extra data bytes) and replace polyphonic aftertouch with such a system for sending variable-length control messages rather than laying it on top of the existing control code system.

you'd have to call that midi version X though.. incompatible with previous versions.

no i've got my own gui code so i haven't used vstgui since i tried it back in 2003-2004. i wrote photo-shop capabilities into the current one. has vstgui4 stepped up on that? do they support alpha blending so on and software / accelerated pixel buffers yet?

the one thing i've been wanting to implement is acceleration of my pixel buffer operations using gl shaders.

---

grymmjack - Fri Feb 24, 2012 9:20 pm

---

@acciddose; Not sure, looks like they do some neat things.

I used to love Ultima Underworld. You've earned some cool points for that screenshot.

I have to wonder why you are using your graphics kit to make a trainer for a game (assuming that's what it is) though

---

aciddose - Fri Feb 24, 2012 9:44 pm

---

inventory/stats editor. i was bored one day and loaded up UW, decided this is a good way to test the gui lib. actually i also figured out how to fully decode for uw2 as well, so it works for both games. i haven't put any effort into flags though so no mucking up your position in the game yet. i thought it was a real shame when the few projects out there for uw rewrites died off - at least we still have arx.

another idea is for installers and so on. i like the simple nsis as things are now, but i'll probably get around to adding some custom graphics eventually. i'll be able to link into the dll (effects, xhip, whatever) inside the installer and display the same widgets as the plugin has for the installer controls.

oh man that's ridiculous

it'll be cool though.

---

mkdr - Sat Feb 25, 2012 12:53 pm

---

Ok. I didn't know that.
So in essence the driver packs the needed 4 bytes that a CC for example needs, to just 3? Combining status and channel right? Sounds clever as theres only 16 channels. Makes sense

To sum stuff up: you could use a special nrpn address you'd use for binary data input(if only one address is used you can just keep sending the new data values without the address data..). That could be a nice crossover of sysex vs nrpn, that would take just 2x bandwidth of what a sysex takes, would work from a vst and on ip-midi, and wouldn't break the midi standard. That could work if your building your own hardware and can implement that on the device-end.. and if midi-interfaces won't drop any of the cc messages.

---

aciddose - Sat Feb 25, 2012 2:07 pm

---

no the 2x data doesn't require anything special.

you could get 1-1 data like sysex if you broke the standard like i do. (with one midi device on one cable, it can't break anything.)

---

arakula - Sat Feb 25, 2012 9:56 pm

---

No. It's 3 bytes all the way. As the VST SDK doc says:

---

There are 35 posts in this topic.