U-he preset randomizer tool (open-source CLI)

[AtomOfScent]

I tried Zebra and got this error, likely because I have a ridiculous amount of presets. Most of my library is duplicated, original by bank and also grouped by category. I'm probably an outlier here so don't expect a fix. It works fine if I choose something other than all presets so not a showstopper.

Code: Select all

<--- Last few GCs --->

[24232:000001B25DF51000]    44900 ms: Scavenge (interleaved) 4042.6 (4120.8) -> 4038.3 (4138.5) MB, pooled: 0 MB, 14.53 / 0.00 ms  (average mu = 0.312, current mu = 0.234) allocation failure;
[24232:000001B25DF51000]    47255 ms: Mark-Compact 4055.6 (4142.0) -> 4045.8 (4148.0) MB, pooled: 0 MB, 2314.70 / 0.00 ms  (average mu = 0.218, current mu = 0.096) allocation failure; scavenge might not succeed
<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 00007FF695E8DA2B node::SetCppgcReference+15611
 2: 00007FF695DFAFE4 DSA_meth_get_flags+86548
 3: 00007FF696A5E351 v8::Isolate::ReportExternalAllocationLimitReached+65
 4: 00007FF696A4AF76 v8::Function::Experimental_IsNopFunction+2918
 5: 00007FF696894DA0 v8::internal::StrongRootAllocatorBase::StrongRootAllocatorBase+31760
 6: 00007FF696891DAD v8::internal::StrongRootAllocatorBase::StrongRootAllocatorBase+19485
 7: 00007FF6968A7613 v8::Isolate::GetHeapProfiler+7267
 8: 00007FF6968A7EAA v8::Isolate::GetHeapProfiler+9466
 9: 00007FF6968B2FFD v8::Isolate::GetHeapProfiler+54861
10: 00007FF6968C2EC2 v8::Isolate::GetHeapProfiler+120082
11: 00007FF6968C7FA8 v8::Isolate::GetHeapProfiler+140792
12: 00007FF6968B7ABD v8::Isolate::GetHeapProfiler+73997
13: 00007FF6968BD4FC v8::Isolate::GetHeapProfiler+97100
14: 00007FF696568B9F v8::internal::Version::GetString+382575
15: 00007FF636AED5BA

As for glob, I'm getting used to **/xx *, but it still feels awkward. As I doubt the average user will be familiar with glob most user friendly IMO would be 3 options:

1) All [*.*]
2) Folder [text entry filters folders]
3) Presets starting with [text entry filters presets]

Text also seem to be case sensitive. In most cases I don't think I'd want to differentiate between Pad and PAD, but it's another personal preference thing.

For Repro I can't get Repro-5 to work. It will only use \Repro-1.data\Presets\Repro-1, not \Repro-1.data\Presets\Repro-5.
Entering Repro-5 as a folder still looks in \Repro-1.data\Presets\Repro-1 so nothing is found.

[Fannon]

Oh, wow - you really must have a LOT of presets to run into such out of memory issues. I could have a look if there's something I could do about it, but since the tool loads your entire preset library in memory (and then also creates analytics / statistics on top of it), this will run into a limit.
I wouldn't like to change this though, because having this all in memory is what makes the tool rather fast.

Can you try this, to increase the memory limit of the node.js process? By default, its limited to ~1.7GB. With this, it would increase it to 4GB:

Code: Select all

npx --node-options='--max-old-space-size=4048' u-he-preset-randomizer@latest

Yes, I agree that glob patterns are not excactly user friendly. But I also don't want to further complicate the CLI prompts. Let me think about it.

The situation with Repro-5 is indeed a bit weird, because the Presets for Repro-5 are in the Repo-1.Data folder. I'll think about if this can be fixed easily.

[Fannon]

Update: 0.8.2 adds Repro-5 support and may reduce the amount of memory usage a tiny bit.

[Fannon]

As for glob, I'm getting used to **/xx *, but it still feels awkward. As I doubt the average user will be familiar with glob most user friendly IMO would be 3 options:

1) All [*.*]
2) Folder [text entry filters folders]
3) Presets starting with [text entry filters presets]

Text also seem to be case sensitive. In most cases I don't think I'd want to differentiate between Pad and PAD, but it's another personal preference thing.

Ok, I've thought about it a bit and have now released 0.9.0 which adds an optional randomization mode where it lets you narrow down by preset category. The CLI will let you pick parent or subcategories in interactive mode if you select the mode. Or you pass it via --category parameter.

I hope this is a good compromise, because the tags do not rely on file name conventions. It's also possible in the u-he preset browser to search by some file name conventions and if tags are missing, to batch-assign them.

[AtomOfScent]

Amazing update!
That's definitely superior to using the file name alone. I assumed it would be more more hassle to implement, but I suppose you were already reading from each file already.
I'll likely use fully random presets a lot more now that a type can be specified.

Maybe a bug, but when using a category and no filter browsing with the arrows is extremely slow, taking ~1s to select the next/prev entry (Diva, category with ~300 presets.)

Also, there might be a different way to do this already, but I'd love if we could use (random pick) with the text filter so instead of choosing a random preset from the entire list it picks a random one from the filtered results.

(keep random pick as a choice or support something like "PAD ?")

Repro-5 working well by the way. Thanks for the prompt fix (pun intended)

-------
Usually I find a bunch of presets worth randomizing in the same synth and run the script a bunch of times in a row where usually the only parameter that's different is the source preset.

Instead of terminating after each run, what do you think of having a set of options?
1) Run again with same configuration [choose new preset(s)]
2) Run again with new configuration [same synth]
3) Exit

There's probably a better set of choices, but the idea is to keep the script running with the preset library for the current synth in memory so if you're running the script 10x in a row for Diva it doesn't need to re-analyze and load the same presets 10x.

Another approach might be to make randomize existing presets more like merge.

Pick presets to randomize
-filter and select
-choose next preset
(press enter with no selection to batch run on chosen presets)

Another random thought:
Sometimes I find it's hard to predict the right amount of randomization for each preset and end up with too much or too little. Being able to enter the randomize % as a range might help, for example 5-15%

The obvious way this could be used is creating the presets with a range of randomization (some lighter, some stronger) vs. 16 presets with 5% which might sound nice too samey, or 16 presets with 15% randomization but only 2 keepers.

The other way would be using the range within each preset where each parameter is randomized between 5-15%. This might be totally inferior to a fixed % and the first method is simpler, but maybe worth experimenting with.

Some parameters/sections can probably take more randomization (fx, filter, envelopes?), and others less (FM, tuning, tune mod, cross mod, detune?), but with the large number of supported synths it would be a mess to sort out and unpractical.

Anyway, thanks again for the continued work on this. I've been having a lot of fun...

[Fannon]

Maybe a bug, but when using a category and no filter browsing with the arrows is extremely slow, taking ~1s to select the next/prev entry (Diva, category with ~300 presets.)

Maybe this is related to memory issues with too many presets? I cannot really replicate this issue, but I'm working on a fast machine

Also, there might be a different way to do this already, but I'd love if we could use (random pick) with the text filter so instead of choosing a random preset from the entire list it picks a random one from the filtered results.

That sounds convenient, but I'm not sure if it would fit well with the current architecture, sorry.

I like the idea with asking if to do something again with less input. But unfortunately this would also be difficult because I partially throw away some information, which I would then need to re-organize. Having the CLI just run one time through makes the process more "stateless" and simpler to reason about. But for this purpose, I've provided the output at the very end how you can copy 'n paste the command to quickly redo it. There you can easily change something and run it again.

In general, I think I'm now happy with the feature state of it. But I'm always open to good ideas and suggestions! Also I'm still wondering if it would also run on a Mac. Don't have one to test it...

[Fannon]

Also, there might be a different way to do this already, but I'd love if we could use (random pick) with the text filter so instead of choosing a random preset from the entire list it picks a random one from the filtered results.

Ok, I thought about it some more and I think I found a solution to this, that doesn't break the current architecture. Now it's possible to have *seach string* or ?search string* selections. They either select all presets including the search strong (*..*) or pick one out of the selection (?..?). This is indeed convenient for some things. You can try that out with v0.9.9 now.

2024-05-13 21_20_04-randomizer.ts - u-he-synth-preset-randomizer - Visual Studio Code.png

If all goes stable now, I think it's time to make it a 1.0.0 release. Maybe along with a free preset pack that contains some hand picked random presets coming from the generator?

[AtomOfScent]

Ok, I thought about it some more and I think I found a solution to this, that doesn't break the current architecture. Now it's possible to have *seach string* or ?search string* selections. They either select all presets including the search strong (*..*) or pick one out of the selection (?..?). This is indeed convenient for some things. You can try that out with v0.9.9 now.

If all goes stable now, I think it's time to make it a 1.0.0 release. Maybe along with a free preset pack that contains some hand picked random presets coming from the generator?

Great you were able to figure that out. I'm looking forward to testing it.
Saw the fix for the merge issue too.

I agree it seems nearly ready for 1.0.0, and I'm sure you're getting sick of it by now.

There's a minor 'bug' using presets with certain paths, but I doubt it would affect many people.
I like to name primary folders so they stand out and group at the top, i.e. "=Factory=", "=Legacy=". It works fine when selecting, but they can't be found when processing.
No preset with name "=Factory=/*rest of path*.h2p" found!

Maybe it's a u-he thing. I usually use Explorer but I just noticed it won't let me create folders with = in the preset browser.
----------

IMO, the biggest remaining improvement would be excluding a few key parameters. I haven't had time to go through all the synths, but there are a few that are mostly universal. Ones that are synth specific could just be ignored if not found. That could allow a single 'blacklist' of parameters rather than one for each synth.

CcOp - Main output for all (most?) synths.
Trsp/FTun - Usually global tuning. Lots of generated presets change this so the output doesn't match the notes played.
Tune* - Commonly oscillator tuning. Randomizing this by % could turn 12 to 10 etc. and then you have a bad sounding chord.

This is an incomplete list by synth. There's lots of overlap (and likely some errors.)
Entries with ** are parameters I think are high importance.

Repro
**NVoices - Polyphony
#cm=OscA/B
**Freq - Oscillator pitch. Values other than -12, 0, or 12 (+/- 1%) will likely sound bad.
**LowFrq=0 - Value of 1 makes the OscB an LFO, rarely sounds good.
#cm=Pitch
Tune - Master tuning in semitones.
**KPitch=1 - Value of 0 doesn't send OscB to the keyboard.

Diva
Voices - polyphony
Trsp - Master transpose
#cm=OSC
Tune1/2/3 - Oscillator octave. Range -24 to +24 in steps of 12.

Zebra
#cm=VCC
Trsp - Master transpose
FTun - Master tuning in cents.
#cm=ZMas
Mast - Master output (separate from CcOp main output.)

Hive
#cm=VCC (this whole section might be best to leave alone.)
Voices - Polyphony setting. 7 is 16, 6 is 8, etc.
Mode - 0 is polyphonic, 1 is mono, 2 is legato
Trsp - Master transpose. -12, 0, +12 best.
FTun - Master tuning in cents.
#cm=Osc1/2
**Semi - Osc pitch. 0 to stay in tune.
**SubTune - Sub Osc pitch. -12, 0, +12 best.

A lower priority wist list item is an option to filter by author in addition to category (if possible). There's a handful of sound designers I rate above the rest so being able to narrow down to them individually could be a big help when choosing presets to merge, randomize, or as the basis for "Fully randomized presets."
--author 'Luftrum'

Including multiple categories (and/or authors) in the filter would be cool too, but could be more effort than it's worth.
--category "Pads:Drones, Pads:Soundscape"

I'll set a few presets aside for the free pack.

[Fannon]

Maybe it's a u-he thing. I usually use Explorer but I just noticed it won't let me create folders with = in the preset browser.

Yes, could be. In that case I would avoid having "=" in the file name. Or "*" / "?" / ":", "/", as I imagine they could also cause issues.

IMO, the biggest remaining improvement would be excluding a few key parameters. I haven't had time to go through all the synths, but there are a few that are mostly universal. Ones that are synth specific could just be ignored if not found. That could allow a single 'blacklist' of parameters rather than one for each synth.

Have you tried the "stable" mode? It should actually lessen this problem. Btw. the random generator in fully random or randomize preset mode doesn't really just randomply pick a number between -24 and +24 (in case of pitch). It chooses real values on real distributsions on the preset library. That makes it rather unlikely that you run frequently into unmusical results. At least with the osc tuning (**Freq). With global tuning, I agree. That will very likely just mess things up unnecessarily.

I would need more time to look into the proposed properties. Thanks for preparing this!

A lower priority wist list item is an option to filter by author in addition to category (if possible).

That would actually be quite easy now to add, just as a second option besides picking a category.

I'll set a few presets aside for the free pack.

That would be very nice! Before releasing it as a pack, I would however at least do some slight adjustments (in case some mod assignment gone wrong), volume-level and properly tag the presets.

[AtomOfScent]

Yes, could be. In that case I would avoid having "=" in the file name. Or "*" / "?" / ":", "/", as I imagine they could also cause issues.

* " / \ < > : | ? are all illegal characters for directories in Windows, but = should be fine (MacOS and Linux too). I haven't had an issue with any other programs, or u-he other than trying to make a folder (which I rarely do). I'll make a thread in case it's a bug.

No idea if this will work, but I asked ChatGPT-4o (released today), and it suggested escaping the character and sanitizing the input paths (converting back when the path is actually used.)

Lines 124-131 in randomizer.ts

Code: Select all

else {
  const sanitizedPresetTitle = presetTitle.replace(/=/g, '\\=');
  basePreset = presetLibrary.presets.find((el) => {
    return el.filePath.includes(sanitizedPresetTitle);
  });
  if (!basePreset) {
    console.error(`No preset with name ${presetTitle} found!`);
    process.exit(1);
  }
}

If this doesn't resolve the issue it suggests "checking how the presetLibrary.presets array is populated..."

Have you tried the "stable" mode? It should actually lessen this problem. Btw. the random generator in fully random or randomize preset mode doesn't really just randomply pick a number between -24 and +24 (in case of pitch). It chooses real values on real distributsions on the preset library. That makes it rather unlikely that you run frequently into unmusical results. At least with the osc tuning (**Freq). With global tuning, I agree. That will very likely just mess things up unnecessarily.

I would need more time to look into the proposed properties. Thanks for preparing this!

Typically I've been using stable mode since it was added. The most common issue with otherwise promising results has been global tuning, but it's also easily noticeable as the value isn't covered by the preset browser.

I'll pay attention to whether it's still happening. If so, it could be related to the additional randomization on those parameters. 5-10% on an oscillator tuning might be just enough to sound terrible!

The results have been far better in "fully random" mode when categories were added, I assume that's because it's using the value distribution from only that subset?

That would actually be quite easy now to add, just as a second option besides picking a category.

Great to know. I might re-organize my folder structure a bit (like a dedicated folder for each of my favorite sound designers)

I also realized that in the the favorites section you can select all the presets and easily copy the files to a different location. I think I'll copy them all to one folder and temporarily make that the main preset folder (naming the other Diva2 or whatever) so when I run the script it's only seeing my favorites.

That would be very nice! Before releasing it as a pack, I would however at least do some slight adjustments (in case some mod assignment gone wrong), volume-level and properly tag the presets.

Good idea. Usually I always tweak them a bit, but I wasn't sure if you were looking for unadulterated output from the script.

[Fannon]

I'll pay attention to whether it's still happening. If so, it could be related to the additional randomization on those parameters. 5-10% on an oscillator tuning might be just enough to sound terrible!

Yeah, you're right. The additional randomization will mess those values up because it actually interpolates between the presets current value and a new randomly picked value. For tuning parameters, this will not work well.

Well.. I'm not sure how much time I can put into fine-tuning this. But I've added now some logic to exclude some parameters (a few of those you suggested in an earlier post) from randomization (in general, or just when using stable mode). See https://github.com/Fannon/u-he-preset-r ... er.ts#L107 for the current choices.
But my feeling is that there's diminishing value in tweaking this. Some accidents may actually lead to interesting results, so I'm a bit wary of overdoing this.

I've also added now narrow down by author. That was easy to add, as it was mostly just copy 'n paste from how the narrow down by category works.

The results have been far better in "fully random" mode when categories were added, I assume that's because it's using the value distribution from only that subset?

Yes, that's exactly how it works. You not only narrow down possible choices, but also narrow down the range and distribution of the param values.

[glokraw]

(Couldn't find a way to attach these in a PM )
Here are the linux U-he paths in a screenshot

u-he-linux-folders.jpg

[glokraw]

also, installing Diva gets these dialogs:

Diva-installer-wine.jpg

Diva-install-locations.jpg

[AtomOfScent]

Well.. I'm not sure how much time I can put into fine-tuning this. But I've added now some logic to exclude some parameters...See https://github.com/Fannon/u-he-preset-r ... er.ts#L107 for the current choices. But my feeling is that there's diminishing value in tweaking this. Some accidents may actually lead to interesting results, so I'm a bit wary of overdoing this.

I agree with not fine-tuning away the happy accidents. I think those are good choices. I'll test with a wider variety of synths/modes to see if anything else is glaringly problematic...

I've also added now narrow down by author. That was easy to add, as it was mostly just copy 'n paste from how the narrow down by category works.

Thanks a lot for this.
Personally, I think it might be better to pick the author first as the category choice will be dependent on the availability/quantity of presets by the author of that type (that's how I'd use it at least)

I picked a category/author and it said there were no results before I noticed the presets hadn't been tagged. Annoying quirks in the browser lead me to RTFM and what could be a really useful 'discovery':

The replacement for Favourites.db.txt in standard JSON format!

You can export Favourite status, all at once or individually: Shift+click and drag the ‘Favourites’
folder onto the desktop to create a file called Favourites.uhe-fav

Code: Select all

            {
                "db_path": "/Local/Howard Scarr Performer Soundset",
                "extension": ".h2p",
                "name": "HS Stringy Cheese"
            },

Yes, that's exactly how it works. You not only narrow down possible choices, but also narrow down the range and distribution of the param values.

Depending on how easy it is to support, maybe if the script finds Favourites.uhe-fav in the preset root folder Favourites could be an option at the "Which presets to load for consideration" step (only those would be loaded.)

This essentially adds a layer of personalization, and since the value distributions will be based on individual preferences, there's a higher likelihood the output will be tailored to each user's taste.

It might be the final big puzzle piece.

[Fannon]

Here are the linux U-he paths in a screenshot

Thanks for posting! I'm still not sure from the screenshots / locations there whether it will really work, but I've added now something to detect the linux preset library location. Can you try again and report if it works?

Personally, I think it might be better to pick the author first as the category choice will be dependent on the availability/quantity of presets by the author of that type (that's how I'd use it at least)

You're right. And I messed something up about how those two choices would each build upon each other. Now it first narrows down on author and then the category will narrow down only on what's left after it has been filtered by author. Should be fixed now with v0.11.2.

Depending on how easy it is to support, maybe if the script finds Favourites.uhe-fav in the preset root folder Favourites could be an option at the "Which presets to load for consideration" step (only those would be loaded.)

That was already on my mind, as I use this .uhe-fav export format myself.
Probably this could be easy to add, as yet another "narrow down" option. But ideally, the CLI would auto-detect those .uhe-fav exports and offer to select one of them.