Handling broken presets

[Vokbuz]

I am working on a preset system for my plugin. The thing that bothers me at the moment is handling of broken presets or errors that happen during preset loading. Situation: you start reading from buffer or stream and at some point you meet malformed data or stream has no more data. The problem here is plugin state that partially updated when error discovered.

How do you guys handle this? Do you use pre-validation of the preset buffer? Save state before loading new preset and restore old state in case of error? Use some kind of transaction mechanism for plugin state? Maybe something else? Or maybe it's not worth it to handle error and leave plugin in broken state?

I will appreciate any hints. Thanks.

[Aloysius]

Try using Cytomic 'The Glue' on your broken presets.

[quikquak]

Save presets in XML format as a readable list of parameter names followed by their values. So anything added or taken away can be catered for easily.

[mystran]

I am working on a preset system for my plugin. The thing that bothers me at the moment is handling of broken presets or errors that happen during preset loading. Situation: you start reading from buffer or stream and at some point you meet malformed data or stream has no more data. The problem here is plugin state that partially updated when error discovered.

The easy way is to keep another copy of the plugin state. When you load a preset, you load it into this "extra copy" and if the loading completes successfully you can then copy over the result to the actual plugin state. If you encounter an error, you can just bail-out and the actual plugin state stays as-is.

[Vokbuz]

The easy way is to keep another copy of the plugin state.

Not sure what you mean by another copy of the state. State usually distributed in the different parts of the plugin. Do you mean some intermediate object that keeps the state and then distributed to actual plugin internals? Do you use this method in your plugins? I wonder if it works in practice.

[Music Engineer]

i also use xml for storing presets. parameter values are stored in xml attributes and if the patch loader doesn't find the attribute in question in the xml, the corresponding parameter is set to its default value (usually some value that could be considered most neutral, if applicable - such as 0 for a detune)

edit:
this, by the way, makes it easy to later expand the parameter set and still be able to load old patches that don't have any values for the additional parameters - they are just set to neutral, if missing - which should make the instrument/effect behave as if they weren't there in the first place. i actually even rely on this behavior and store parameters only when they are different from their default values. let's me get rid of a lot of unnecessary clutter in my preset xml files and makes them more readable

[Vokbuz]

I see XML is mentioned twice already but I don’t get how it can help in this case. XML can be broken too. Some required attribute or element can be missing or contain invalid data.

[Music Engineer]

I see XML is mentioned twice already but I don’t get how it can help in this case. XML can be broken too. Some required attribute or element can be missing or contain invalid data.

yes - and if it's missing, then just fall back to a default value. and if the data is wrong, it probably means that the user has messed with the xml file so s/he duefully should expect the plugin to potentially misbehave

[Vokbuz]

yes - and if it's missing, then just fall back to a default value. and if the data is wrong, it probably means that the user has messed with the xml file so s/he duefully should expect the plugin to potentially misbehave

My question is not about missing state parts in preset created by earlier version. It is about really corrupted preset file. The reasons for this can be various.

State is not only parameter values and not every data has default values. In wavetable oscillator it can be custom oscillator waveform that is missing. Or it can be MSEG data that does not contain all required points in it.

Even if we assume that any data can have default value it is still wrong to treat corrupted data as correct. For example, if it is factory preset it is safer to report error to the user than load incorrect preset that make user disappointed.

[BertKoor]

Errors can always occur. So what's wrong with reporting "Sorry, an error occurred: premature EOF" to the user?

Any idea about the likelyhood of such errors? How much time are you willing to invest in scenarios that occur just once in a zillion?

[quikquak]

The reason for a text file is that it’s human readable and any parts missing can be guessed at and manually repaired. Corrupted data is just corrupted data, and there’s nothing you can do about it, other than pray for a recent backup being available. Of course the user will be disappointed, but that’s computer’s for you.
You can check for valid floating point numbers if you are using them, so you can report an error there.

[mystran]

The easy way is to keep another copy of the plugin state.

Not sure what you mean by another copy of the state. State usually distributed in the different parts of the plugin. Do you mean some intermediate object that keeps the state and then distributed to actual plugin internals? Do you use this method in your plugins? I wonder if it works in practice.

The point I'm trying to make here is that this "problem" goes away, when you separate parsing of the preset file (whether it's an actual file or not) from setting the plugin state. However your actual preset state is organized inside your plugin, you can always parse the "file" into an intermediate structure of some sort, checking for errors as you go. If the file-parsing part completes successfully, then you know that your intermediate structure now contains a valid preset that you can set as the current state.

ps. the other possibility is to do a "dry run" over the input just parsing it.. but it's potentially easier to just dump the data into some intermediate structure as you go over it, so that the second pass doesn't need to bother with any details of the external serialisation format

[Youlean]

One thing comes to my mind: When you create preset you will have a buffer of values that will hold parameter data. Why don't you use a simple fast hash to get hash of that data and store it with the data in the file?

For example:

file.preset
{
// hash of data below
de9f2c7fd25e1b3afad3e85a0bd17d9b100db4b3

// data
ofidhasfoihewp2930r72po34iygp834ty7p348yp0vt7npv07n2v-0t7vnt4v7p34nvt7v34-0t7n9nt0vn0tv340-v0
}

Now when you load that preset you will hash data again and if hash don't match you have broken data and you display the message and don't load the preset.

Note that ether could be broken and it will still work. Also, you can know how much exact bytes to read for hash on the begining.

[Mayae]

Great suggestions already I usually:
- Embed size, name, stamp and hash of payload (preset data) inside chunk, for a first line of defense
- Each data node in the chunk is tracked in size and integrity is checked every time you walk
- Associative mappings for larger, unrelated/optional/versioned sets of data (key/value structure like XML)

Additionally, I would highlight these points that should ensure you have a rigid system:
- Separate parsing and applying the preset as mentioned
- Serialize your plugin as a fallback copy you can reapply, if applying the input preset went bad as you said
- Make sure you validate user input data (it is, effectively) in your deserialization
- Make sure your plugin works in "default" state, and start out with loading a known good preset always

XML is great and all but it really solves a small part of the problem, and carries a pretty large overhead. Also XML usually don't help you when it is embedded inside a binary project file.

... this, by the way, makes it easy to later expand the parameter set and still be able to load old patches that don't have any values for the additional parameters

If you're changing parameter layout that would still break old projects right?

[Vokbuz]

Errors can always occur. So what's wrong with reporting "Sorry, an error occurred: premature EOF" to the user?

The problem is not in showing error report. Problem is that when error discovered plugin state is only partially updated and its state is likely producing garbage sound. That is not what user expects.