Roland Sysex?

[mztk]

i'm trying to remember how to send system exclusive messages
for Roland 4x partial units - you have to sum a start address and
offset address, and then calculate the checksum.

here's what it looks like for the MT-32 ( 4 partial voices, same
as D-series) (skip to #4)

http://www.youngmonkey.ca/nose/audio_te ... -MT32.html

(this is easier to read than from manual.pdf)

Byte Description
---- -----------
F0H Exclusive Status
41H Roland ID
DEV Device ID
16H (14H) Model ID
12H Command ID (DTI)
aaH Address MSB
aaH Address
aaH Address LSB
ddH DATA
:
sum Checksum
F7H EOX

this is the message format, and the variable will be
the LSB in 'DATA'

"4. Address Mapping of Parameters:
...
The actual address of a parameter in a block is the sum of the start
address of each block and one or more offset addrses. That is addresses
marked by *4-1, *4-2, and the other in the rhythm setup table or in the
partial parameter table."
-------------------------
first you get:
Parameter Base Address

Temporary area (accesible on each basic channel)
Start Address Description
------------- ---------------------------
00 00 00 Patch Temp Area (parts 1-8)
01 00 00 Set Up Temp Area (rhyhtm part) *4-1
02 00 00 Timbre Temp Area (parts 1-8) *4-2

then:

Whole Part (accessible on unit#)
Start Address Description
------------- ----------------------------
03 00 00 Patch Temp Area (part 1)
03 00 10 Patch Temp Area (part 2)
:
03 00 60 Patch Temp Area (part 7)
03 00 70 Patch Temp Area (part
03 01 10 Set up Temp Area (rhythm part)
04 00 00 Timbre Temp Area (part 1) *4-2
04 01 76 Timbre Temp Area (part 2) *4-2
:
04 0b 44 Timbre Temp Area (part 7) *4-2
04 0d 3a Timbre Temp Area (part *4-2
04 00 00 Timbre Temp Area (part 1)
05 00 00 Patch Memory #1
05 00 08 Patch Memory #2
:
05 07 70 Patch Memory #127
05 07 78 Patch Memory #128
08 00 00 Timbre Memory #1 *4-2
08 00 00 Timbre Memory #2 *4-2
:
08 00 00 Timbre Memory #63 *4-2
08 00 00 Timbre Memory #64 *4-2
10 00 00 System Area
20 00 00 Display *4-3
7F xx xx All Parameters reset *4-4

Notes:
*4-1 Structure of Setup Temp area is as follows:

Offset Address Description
-------------- ------------
00 00 00 Rhythm setup for key #24
00 00 04 Rhythm setup for key #25
00 00 08 Rhythm setup for key #26
00 00 0C Rhythm setup for key #27
:
00 01 78 Rhythm setup for key #86
00 01 7C Rhythm setup for key #87

*4-2 Structure of Timbre Temp area is as follows:

Offset Address Description
-------------- -----------
00 00 00 Common Parameter
00 00 0E Partial Parameter for partial #1
00 00 48 Partial Parameter for partial #2
00 01 02 Partial Parameter for partial #3
00 01 3C Partial Parameter for partial #4


the bit for editing a 'tone' starts here:

Common Parameter 4-5

Offset Address Description
-------------- -----------
00H 0aaa aaaa Tone Name #1 32-127 (ASCII)
:
09H 0aaa aaaa Tone Name #10 32-127 (ASCII)
0AH 0aaa aaaa Structures of Partials #1&2 0-12 (1-13)
0BH 0aaa aaaa Structures of Partials #3&4 0-12 (1-13)
0CH 0aaa aaaa Partial Mute 0-15 (0000-1111)
0DH 0aaa aaaa Env. Mode 0-1(Normal, No sustain)
Total Size 00 00 0EH

Partial Pameters *4-5
Offset Address Description
-------------- -----------
00 00H 0aaa aaaa WG PITCH COARSE 0-96 (C1,C#1-C9)
00 01H 0aaa aaaa WG PITCH FINE 0-100 (-50 - +50)
00 02H 0000 aaaa WG PITCH KEYFOLLOW 0-16 (-1,-1/2,
0,1,1/8,1/4,3/8,1/2,5/8,3/4,7/8,
1,5/4,3/2,2.s1,s2)
00 03H 0000 000a WG PITCH BENDER SWITCH 0,1 (ON/OFF)
00 04H 0000 000A WG WAVE FORM 0-1 (SQU/SAW)
00 05H 0aaa aaaa WG PITCH PCM WAVE# 0-127 (1-128)
00 06H 0aaa aaaa WG PITCH PULSE WIDTH 0-100
00 07H 0000 aaaa WG PITCH PW VELO SENS 0-14 (-7 - +7)
00 08H 0000 aaaa P-ENV DEPTH 0-10
00 09H 0aaaaaa P-ENV VELO SENSITIVITY 1-100
00 0AH 0000 aaaa P-ENV TIME KEY FOLLOW 0-4
00 0BH 0aaa aaaa P-ENV TIME 1 1-100
00 0CH 0aaa aaaa P-ENV TIME 2 1-100
00 0DH 0aaa aaaa P-ENV TIME 3 1-100
00 0EH 0aaa aaaa P-ENV TIME 4 1-100
00 0FH 0aaa aaaa P-ENV Level 0 1-100 (-50 - +50)
00 10H 0aaa aaaa P-ENV LEVEL 1 1-100 (-50 - +50)
00 11H 0aaa aaaa P-ENV LEVEL 2 1-100 (-50 - +50)
00 12H 0aaa aaaa P-ENV SUSTAIN LEVEL 1-100 (-50 - +50)
00 13H 0aaa aaaa P-ENV END LEVEL 1-100 (-50 - +50)
00 14H 0aaa aaaa P-LFO RATE 0-100
00 15H 0aaa aaaa P-LFO DEPTH 0-100
00 16H 0aaa aaaa P-LFO MOD SENSITIVITY 0-100
00 17H 0aaa aaaa TVF CUT-OFF FREQ 0-100
00 18H 0aaa aaaa TVF RESONANCE 0-30
00 19H 0aaa aaaa TVF KEY FOLLOW 0-16 (-1,-1/2,1/4,0,1,
1/8,1/4,3/8,1/2,5/8,3/2,7/8,1
5/4,3/2,2,s1,s2)
00 1AH 0aaa aaaa TVF BIAS POINT/DIR 0-127
(<1A-<7C >1A-7C)
00 1BH 0aaa aaaa TVF BIAS LEVEL 0-14 (-7 - +7)
00 1CH 0aaa aaaa TVF ENV DEPTH 0-100
00 1DH 0aaa aaaa TVF ENV VELO SENSITIVITY 0-100
00 1EH 0aaa aaaa TVF ENV DEPTH KEY FOLL0W 0-4
00 1FH 0aaa aaaa TVF ENV TIME KEY FOLLOW 0-4
00 20H 0aaa aaaa TVF ENV TIME 1 1-100
00 21H 0aaa aaaa TVF ENV TIME 2 1-100
00 22H 0aaa aaaa TVF ENV TIME 3 1-100
00 23H 0aaa aaaa TVF ENV TIME 4 1-100
00 24H 0aaa aaaa TVF ENV TIME 5 1-100
00 25H 0aaa aaaa TVF ENV LEVEL 1 1-100
00 26H 0aaa aaaa TVF ENV LEVEL 2 1-100
00 27H 0aaa aaaa TVF ENV LEVEL 3 1-100
00 28H 0aaa aaaa TVF ENV SUSTAIN LEVEL 0-100
00 29H 0aaa aaaa TVA LEVEL 0-100
00 2AH 0aaa aaaa TVA VELO SENS 0-100
00 2BH 0aaa aaaa TVA BIAS POINT 1 0-127
(<1A-<7C >1A-7C)

00 2CH 0aaa aaaa TVA BIAS LEVEL 1 0-12 (-12 - 0)
00 2DH 0aaa aaaa TVA BIAS POINT 2 0-127
(<1A-<7C >1A-7C)
00 2EH 0aaa aaaa TVA BIAS LEVEL 1 0-12 (12 - 0)
00 2FH 0aaa aaaa TVA ENV TIME KEY FOLLOW 0-4
00 30H 0aaa aaaa TVA ENV TIME V FOLLOW 0-4
00 31H 0aaa aaaa TVA ENV TIME 1 0-100
00 32H 0aaa aaaa TVA ENV TIME 2 0-100
00 33H 0aaa aaaa TVA ENV TIME 3 0-100
00 34H 0aaa aaaa TVA ENV TIME 4 0-100
00 35H 0aaa aaaa TVA ENV TIME 5 0-100
00 36H 0aaa aaaa TVA ENV LEVEL 0-100
00 37H 0aaa aaaa TVA ENV LEVEL 2 0-100
00 38H 0aaa aaaa TVA ENV LEVEL 3 0-100
00 39H 0aaa aaaa TVA ENV SUSTAIN LEVEL 0-100
TOTAL SIZE 00 3AH
-----------------------------------------------------------------------
so that's how you get around a MT32/D-110 - i'm not sure how
many offset addresses need to be included. are there temporary
patch areas for each part? could you do temporary edits to
a tone on any channel? or is there only one edit buffer?

so if i wanted to edit filter cutoff of partial1...
00 00 00 start address
00 00 0E partial#1
00 17 TVF cutoff
..i'd have to add these to get the parameter address, somehow.
there are 3 bytes for address, so it looks ok:
aaH Address MSB
aaH Address
aaH Address LSB
(but does "00 17" align left or right?!)

edit: it aligns right. see here:

http://forums.rolandclan.com/viewtopic.php?f=3&t=45675

right, so that's clear then - but how do a calculate a checksum
for a varying value?

[mztk]

if you never look at these midi implementation charts,
1st column (eg: 00 39H) is the hexadecimal value, and
the 2nd ( 0aaa aaaa ) is a representation of bits used in
binary for the value range: thus, 0aaa aaaa tells us 7 bits
are used (range 0-127) - 0000 000a means the value range
can be 0-1, with the rest of the values on that address
unused.

good page for reference:
http://www.2writers.com/eddie/tutsysex.htm

[mztk]

quick aside (more for own annotation, because i've been looking
for this too): you can combine + send 2 values in a single message
(i didn't know this until just now). here he's doing bank change +
program change in a single message(for a sc88):

" Example 2 : Patch Change. (or Sending more than one byte.)

[1] [2] [3] [4] [5] [6] [7] [8] [9]
F0 41 10 42 12 401100 0801 26 F7

We can send more than one byte of data in a single SysEx message. This is very useful when we want to set the value of several settings that follow each other in the address map. By sending more than one byte in part [7], which is the data part, the first byte will be put in the address specified and the following bytes in each successive address location. Each address location can only hold one byte of data.

In the example above we are putting the value 08h into address 401100h and the value 01h into address 401101h. Think about it this way, if we send more than one byte to an address then each successive byte will be put in the address below the previous one in the map in the manual.

For the above example, the SC-88 manual has the following in the address map:

Address(H) Size (H) Data (H) Parameter Description
---------- -------- -------- ----------- -----------------
40 1x 00 / 00 00 02 / 00 - 7F /Tone Number/ /CC#00 /Value 0-127
40 1x 01# " " " " " " " / 00 - 7F ---------------/P.C. /Value 1-128 "

so here, it sends bank change and program change by sending an
extended value to a single address, and the excess gets sent to
next consecutive address
(totally new to me, and i don't quite get it)

[mztk]

anyway, here's how to calculate the checksum:
("..")
1. Convert the hex values to decimal.
2. Add the values together, but if the answer to any sum exceeds 127 then subtract 128.
3. Subtract the final answer from 128.
4. Convert the resulting value back to hex and use as the checksum.

ah: but is he referring to the entire message? all bytes? or just those
relating to the parameter address?

-----------
here it is again:
ftp://ftp.monash.edu.au/pub/midi/DOC/Roland-checksum
"
Okay, now for the checksum calculation. Remember it only uses the address
and the data. The first example we will use is the example at the bottom of
page 39 of the SCC-1 manual, for adjusting the master volume. The address is
40h 00h 04h and the data 64h. Here's how the checksum is derived:

Add each of the values, one by one. Each time the value exceeds 127 (7Fh)
subtract 128 (80h) from it. Finally subtract the resultant value from 128
and you have your checksum!

In hexadecimal: (I won't put h after each value for clarity)
40 + 00 = 40 (Surprise!)
40 + 04 = 44
44 + 64 = A8 ( A8 > 7F so...)
A8 - 80 = 28
80 - 28 = 58 (The checksum is 58h)

In decimal:
64 + 0 = 0
64 + 4 = 68
68 + 100 = 168 ( 168 > 127 so...)
168 - 128 = 40
128 - 40 = 88 (The checksum is 88)


To show what is actually happening I'll show you the last line in binary:

10000000 - 00101000 = 01011000


According to the manual "The error checking process uses a checksum that
provides a bit pattern where the least significant 7 bits are zero when
values for an address, [data], and that checksum are summed."
The low bits in binary are on the right. In the binary calculation above,
add the address, data and checksum and you get back to 128, which has all
seven lower bits clear:

address & data + checksum = 128
00101000 + 01011000 = 10000000


Note that in this example, while summing the values we exceeded 127 and
had to subtract 128. This is not always necessary, as long as the
summed values do not exceed 127.

If the sum of the values comes to 128 then I'm sure it is clear that
the checksum is 128! Look at the arithmetic:
x + y = 128 ( 128 > 127 so...)
128 - 128 = 0
128 - 0 = 128 (The checksum is 128)
"

[mztk]

(another aside: RPN/NRPN refresher. note how you specify
parameter address with MSB + LSB MIDI CC#s, before sending
a MIDI CC#6 'data entry', followed by MIDI CC#38
NB: ... #6 is in fact data/value MSB, and #38 is data/value LSB,
can be omitted if it is zero -see pdf)

http://www.2writers.com/eddie/TutNrpn.PDF

----------------------------------------------------------------------
here's s'th more useful for adding the start+offset addresses:

http://forums.rolandclan.com/viewtopic.php?f=3&t=45675

[mztk]

right well that says you add the addresses and then the data,
and do the checksum on that - so how do you do that if the
data is a variable? if you have a stream of values coming
from a remote editor/controller, the checksum has to vary
too... (how did i do this before? i was sure i'd only had to
calculate the checksum once per controller object created)

[mztk]

(what a waste of MIDI CC#s in the MIDI protocol... #s 96+97: data inc and data dec...)

more RPN/NRPN info on old philip rees page:

http://www.philrees.co.uk/nrpnq.htm

(trying to stick to sysex, but it's all useful info)

[mztk]

quote (again):

"you can normally put any group of parameters into a single sysex message
as long as they are next to each other in the parameter map."

this is interesting. as stated on the previous page, you combine the bytes
for data, and the overflow gets kicked on to the next parameter in line.
as in, 'send to addresses starting xx xx xx'.

can't think of any examples where you might do that.. (?)

nb: this isn't normally what you'd be doing with a sysex message. what
this is about is specifying a parameter and its (variable/varying) value.
(or in other words, building control maps for hardware)

[mztk]

start again !

i'm cheating now: i have free MIDI OX running, listening to Maple Midi Cable#1
and Windows MIDI Watch/wmv30, which is a free D-110 editor (with a really
obvious helpful name, so you know it works with a D-110, right..)

eventually figured out how to get it to listen+display and am now sending
data from my virtual D-110(wmv30) to midiox, and can capture each sysex
event in the window.

so, the most obvious parameter, filter cutoff - i sent: partial1 cutoff 1, 2, 3
in increments, and got:

F0 41 10 16 12 04 00 25 01 56 F7
F0 41 10 16 12 04 00 25 02 55 F7
F0 41 10 16 12 04 00 25 03 54 F7

04 00 25 is the 3 byte parameter address for P(artial)1 'TVF cutoff'

only 1 value/data byte is being received (data MSB? i was wrong, above,
saying it was LSB, if these are used only for fine changes, or higher reso-
lution parameters, such as full-range 14-bit controls that use msb+lsb)

and the checksum changes each time: 56,55,54..
so how do you handle that, when you only have 1 variable?
(was there a checkbox in atari cubase referring to checksum?
is there on in the Novation Remote editor -called?..)
(ReMote Zero SL Editor..)
No: (but it does support 14-bit - that's (got to be) how you send
MSB and LSB together(?) (lol this is the poking-around-in-the-dark
thread)

[mztk]

Novation's website has had a revamp(de-vamp) and they have a few
of what they call 'MIDI Templates' (not many though, this looks like
it hasn't been updated for content for a while)

http://novationmusic.fr/community/libra ... -templates

so how do you vary the checksum as well as the variable?

[mztk]

right, well it turned out to be quicker to cheat with midiox and
a D-110 editor, wmv30(windows midi watch) - since we now
know how to sum + calculate checksum, right..i just captured
the sysex by wiggling virtual faders, and copied address+data+chcksum
block to Notepad, ready to copy into Remote Editor - which it has to
be said, is a bit of a disappointment, as it is only a very simplified
version of what is possible on the unit.

most of the parameters were at consecutive addresses anyway, so it
was easy to just fill them all in, checking for gaps + mistakes (counting
backwards in hex)

so in theory i have all parameters+messages listed, ready to use.
now i have to figure out how to lay it all out on the Novation:
a tone has 4 partials in 2 pairs, each with 3 multistage+level EGs, LFO,
and the rest. and then there's the 'patch' multimode setup -it's a lot of
panel space - what's the best layout?

and how do you do it on the Remote? are you supposed to program
conscutive templates, or can you program menus/pages into a single
template? it looks like the editor was a rush job and never got updated
properly: it lacks parameters found on the unit.

as for how to do the checksum - this is one of the modes missing in the
$oftware; you have to do it on the machine. and then save and export
the finished map. so it would seem.

maybe the machine also lets you do pages - need to peer into it some
more.

hey Focusrite guys: how about updating the Remote Editor? the mkII
one doesn't look too different, either.

[mztk]

(update)

ok so i now understand that you're supposed to work with linked
consecutive 'templates' - you define this on the Remote.

so if you want 4 pages, you need 4 template memory slots,
eg: 1/2/3/4 and you navigate between those

tbh, that is not immediately obvious on 1st approach. i'd given
it a look over, but i like to see what i can figure out without
help before looking at the manual.

i think it's bcs you don't always get back to the template# screen.
there's an annoying thing that happens when you press
'template' button, to go back to an 'initial view': it takes you to
a sub-page of that,- that is far from obvious if you do not know -,
and you have to press 'up' to get to the template# selector.

because the mk1 is loaded up with software maps i wasn't going
to use, i just looked it over. the first V-station map is confusing
bcs of how it navigates - just figured i'd figure it out later eg: now.

i made a mockup of a few controls with names to see what it looks
like, how it behaves across 2 pages, copying the first one, to have
a 'common' controls section, and it works really well. programming
manually on the unit is in fact really easy and quite relaxing in a way,
but you wouldn't want to input all those hex bytes manually. i'm sure
it's fine, just looks offputting. and working with sysex is actually far
easier and quicker than it looks once you get started: most of the
parameters are in sequential order, -in blocks, at least- so you can
just bash through, 0,1,2,3.. and the checksum goes in reverse/declining
order - if you calculate it at initial '0' value for the 'data' byte.

the editor doesn't want the F0 header or F7 end of sysex, but it ok's
the extra checksum byte after 'DV' (the 'variable' byte). i haven't tried
uploading anything into the unit yet. still checking out the various
elements involved

but for convenience, that's where the editor comes in:
just copy+paste from notepad, save there, then upload into unit,
and finish assignment there, with testing, before being able to export
a viable 'MIDI Template'(as distinct from 'automap' as that's something
different)

[mztk]

should mention: the 1st stage has to be to sit down with
the sysex datasheet of the device, and, first, list in notepad
all the parameter names each on a new line, and then go
through and input in a new 'column':
3 parameter address bytes, 1 value/variable byte, and 1 checksum byte.
then another column with value range of each parameter.

it's a bit of a chore but quite useful as it gives a good overview of
the instrument before starting, and it is thorough. you can work
quickly and identify mistakes easily. and that's when you realise
you don't even have to work out most of the addresses, you just
need to figure out the first one in a block, and the parameter #
goes up while the checksum goes down.(but it was useful to have
the wmv30 editor as a 'virtual D-110' to send sysex into midiox
without having to have the hardware present. you can no doubt
'cheat' to get your sysex string with any editor+midiox, so that's
good to know. wonder if it's good for Casio stuff, which has
another sysex format. it should be possible to run Steem virtual
Atari and load old editors there and send sysex to midiox..)

[mztk]

then you open the Remote Editor to input date from your
now-very-good-looking homemade sysex data sheet:
-copy+past into Remote controller/object definition dialogs

and you realise you need to think carefully about layout.
that's when i picked up the Remote, to really go through
how you create a definition on the unit without the editor,
and how it works out with multipage maps, and what can
be done to make it as useful as possible.

for example, dealing with a D-110(D-20/D-10..JV etc.)
you have:
-4 partials, each with 3 time+level envelopes = 12 EGs
-each has an LFO/filter/etc etc
-then there's a 'common' section
-then there's multichannel setup+global settings

if you think about 'timbre', in addition to the basic tone, and
patch memories, it gets complicated - luckily the author of
wmv30 (Windows MIDI Watch, a W95 program that runs on
Vista ok) had a good take on how to incorporate timbre with
the tone settings, which will make it easier.

yep haha still not there, easy does it..but i'm getting a better
overview of how the effort should be invested. the mockup
i've done 'plays' nicely so far. partial mute buttons staying in
same place through the pages, for example, has a good feel,
and you can do more sophisticated button assignments than
you see in the factory maps(another misleading thing when
you 'browse' the machine).

ah: in fact i should just look at the map for roland jv1080
that is on the mkII - is that available anywhere?

next steps:
-sit down again with new 'datasheet' and a piece of paper,
and figure out a knob, fader, or button for every parameter,
and where it will/can be. and have a good, useful interpretation
of the instrument on the panel. figure out how many pages, etc.
-do 1st partial page, with a 'common' section that will be on all
4 pages.
-name all objects in editor+set to sysex. set button behaviour+
values display+range.
-then copy+paste address+DV(data/variable)+checksum into editor,
and save..then copy, change the sysex+object names, etc.
-then these get loaded into Remote, and each Roland sysex object
then needs to be defined manually on the unit.

(plenty still to do)