How Long Does Other DAWs Take To Save 20,000 Midi Items?

[Guenon]

Harry, I had an idea, and after trying it out, turns out it does work.

Here I have >20,000 individual one bar MIDI items loaded in Reaper with drum sequence content:





(Shows as more than one bar per item as I just quickly loaded them without accommodating the tempo ; ), but trust me, based on your description they are pretty much the kinds of files you are working with.)

Then, instead of doing a batch resave, select them all on the track, right click one of them, and do a "Convert active take MIDI to .mid file reference"



This will result in all of the MIDI items being saved individually into the media directory you have specified in Reaper preferences. Effectively batch resaving each and every one of those items into a new file. You can then operate on them further, copy them to a safe place to be stored and reimported and so on.

This takes about six seconds on the system I tried it on

You will probably want to do some management on the file/item names when reimporting into a new project, but batch renaming items is another function that takes mere seconds.

This way you will be able to experiment iterating your clips batch saving them as often you'd like, without waiting hours on end in the mean time.

Remember that immediately after the conversion, the items on the timeline now refer to the newly generated .mid files, so don't get burned by that in some way (changing the references in this manner in an ongoing project and habitually saving it that way, then moving the files away, etc).

[IncarnateX]

Epic! ^^^^^ Especially if it works for Harry but epic nonetheless.

[Guenon]

Also, adding to the above, re:

Remember that immediately after the conversion, the items on the timeline now refer to the newly generated .mid files, so don't get burned by that in some way (changing the references in this manner in an ongoing project and habitually saving it that way, then moving the files away, etc).

Note that there is an action called "Convert active take MIDI to in-project MIDI source data" in the menu, right above the action I highlighted for doing the .mid file dump.

This means you can also create a custom cycle action with these two in sequence, like this...



...and call it "batch dump selected MIDI items to .mid files" or similar. Then you can simply call it with a press of a hotkey or a toolbar button.

It will instantly dump all selected MIDI items into the preferred media directory as individual .mid files, then switch the source references back to project internal data for those selected items. In effect, it just dumps the .mid files for you to move as you choose, and leaves the items themselves unchanged. I tested this with those >20,000 items, also tried saving the project, deleting the files, etc, and this works. Calling the action is around a 6-7 second process over here.

This eliminates the danger of accidentally saving the project with the data sources changed to external file references after dumping the MIDI in this manner.

[harryupbabble]

@Guernon
Wowee this is exciting. I hope it works.

So, I loaded my 20,187 files in to a REAPER track. It took about 4 minutes.
I opened the MIDI Editor and that also took about 4 minutes. I deleted all the closed hi-hats.
I then closed the Midi Editor and now I'm back at the main window.

With all the items selected, I'm about to right-click on one of the midi items and do the "Item Processing + Convert active Take MIDI to .mid file reference" actions.

But before I proceed I need to ask a question.
You stated "This will result in all of the MIDI items being saved individually into the media directory you have specified in Reaper preferences."

Huh? What media directory? I don't recall ever seeing that before. How do I do that in REAPER's Preferences?

I have to follow your instructions one step at a time. I see you've added some more.

[Guenon]

See the Reaper Preferences window, General > Paths.

Or just try it out with a couple of MIDI items and see where Reaper throws 'em

My guess is, if you haven't changed the preference, it's the same path the project is residing in. Or maybe it's always the same path the project is residing in, and it uses the separately specified path only if the project is unsaved. Just try it ... The location doesn't matter much, as long as you can find where you are dumping the .mids, hah. Then store them where ever you please, after the fact.

[harryupbabble]

@Guernon,

I'm going to have to do this offline. My ancient computer is having problem coping with the CPU load or something. It;s really slow. I'm going to save this page containing your instructions but before I do that, do you have more instructions?

[harryupbabble]

Every time I save a project, to the desktop for example, REAPER only creates one item, and that's the .RPP file. Nothing else.

[Guenon]

No other instructions, but why not try it with a handful of MIDI items first, as I suggest above. So that you get the feel of how these actions work, without burdening your computer with all of those 20,000 items ... Take your time, see where the files are being saved, experiment with it manually and using the cycle action, and so on. Again, with just a handful of files.

[harryupbabble]

Also, I did a whole-disk file search of one of the midi items and it's only showing up at one place: in the folder where the 20,187 midi files are.

[harryupbabble]

Okay will do. Be back in a bit, depending on how well I follow your instructions. Okay, thanks very very very much. Okay bye.

[Guenon]

Also, I did a whole-disk file search of one of the midi items and it's only showing up at one place: in the folder where the 20,187 midi files are.

If you are by any chance importing the MIDI items into your project already in such a manner, by default, that they are handled as external file references (the "Convert active take MIDI to .mid file reference" should be greyed out if the items are already such file references), select them all and use the other action I mentioned to make the items refer to internal project data instead. Then do your edits and proceed as described. It will save the items as new .mid files.

[harryupbabble]

It wasn't 6 seconds on my computer, more like 2 minutes but...
It worked! It worked! It worked! It worked! It worked!

I have a GIF made of the whole thing working but it's 79 MB and KVR only accepts 1 MB. I tried to upload it to YouTube but I guess GIF doesn't count as a video file. I will probably make a proper video tomorrow.

I followed Guernon's instructions... and at the end there at the save prompt,
I just put a check beside the "Move all media into project directory" option.

Thank you Guernon so very very very much. I owe you big time. Literally.
What a huge time-saver this will be. This is ultra-super-mega great!
I don't know what else to say. Happy happy joy joy.
I'm so excited. I have to calm down because I won't be able to sleep.
What a great day this has been. Thanks again Guernon, and goodnight.
Yeeeehaaaaaaaaa.

[Guenon]

Good to hear it worked. Happy experimenting!

If the "thousands of randomized clips inside a linear DAW" approach reaches a plateau at some point, now that you can iterate the material considerably faster, you can then of course keep exploring the algorithmic/assisted composing stuff that is available and see if it helps. So... don't get discouraged if this alone doesn't quite yield the results you are hoping for, in the long run. Just expand from there, on your chosen path, and check out something like the aforementioned RapidComposer, or something similar, that might be right up your alley when you are pulling it all together and managing the structure and progress of harmony throughout the song, and that sort of thing.

But yeah, what ever works and seems to lead to finished songs in the end.

Take care!

[IncarnateX]

Amazing Guenon. Deep respect from here.

If there ever was a KVR price for the best ending to a twisted thread, this one would surely be the candidate of the decade. This thread now belongs in KVR classics, imo.

And best of luck to you Harry

[harryupbabble]

Hi Harry,

Have you got any songs you can point us to that demo your random midi stuff ?

I don't have a full song using my current method yet. I have one in progress. It's only 4 bars long at the moment. It's mostly there for test purpose.