Huge Library for ARTURIA DX7 V

[Stefken]

Hey PietW
There are several tools out there:
https://www.google.com/search?q=find+duplicate+files

[planetearth]

Hey PietW
There are several tools out there:
https://www.google.com/search?q=find+duplicate+files

As other KVR members (like BlackWinny) who have tried to "de-dupe" lists of DX7 presets have found, there is no easy way to do it. Software that simply compares files names, or that tries to peek inside simple text/.DOC/.PDF file is useless here. Many of these patches will have the same names, even if they don't sound the same. Many will have different names, even if they're the same file internally. And off-the-shelf de-duplicating software can't peek inside SYSEX files or proprietary synth files to compare the parameter settings.

Unfortunately, most of the de-duping here would have to be done by hand--and that is a very time-consuming process.

Steve

[Stefken]

Hey PietW
There are several tools out there:
https://www.google.com/search?q=find+duplicate+files

As other KVR members (like BlackWinny) who have tried to "de-dupe" lists of DX7 presets have found, there is no easy way to do it. Software that simply compares files names, or that tries to peek inside simple text/.DOC/.PDF file is useless here. Many of these patches will have the same names, even if they don't sound the same. Many will have different names, even if they're the same file internally. And off-the-shelf de-duplicating software can't peek inside SYSEX files or proprietary synth files to compare the parameter settings.

Unfortunately, most of the de-duping here would have to be done by hand--and that is a very time-consuming process.

Steve

Can you rely on this : when 2 presets have the same settings, the content of the file will be exactely the same, character per character?

Or is it possible that the same exact settings result in a different presetfile depending e.g. on the synth that has written/ saved the file.

[planetearth]

Hey PietW
There are several tools out there:
https://www.google.com/search?q=find+duplicate+files

As other KVR members (like BlackWinny) who have tried to "de-dupe" lists of DX7 presets have found, there is no easy way to do it. Software that simply compares files names, or that tries to peek inside simple text/.DOC/.PDF file is useless here. Many of these patches will have the same names, even if they don't sound the same. Many will have different names, even if they're the same file internally. And off-the-shelf de-duplicating software can't peek inside SYSEX files or proprietary synth files to compare the parameter settings.

Unfortunately, most of the de-duping here would have to be done by hand--and that is a very time-consuming process.

Steve

Can you rely on this : when 2 presets have the same settings, the content of the file will be exactely the same, character per character?

Or is it possible that the same exact settings result in a different presetfile depending e.g. on the synth that has written/ saved the file.

If two presets have the same settings, the content of the SYSEX files will be the same. They pretty much have to be the same, since the settings are the same.

To answer your second question, no, the exact, same settings will not result in a different SYSEX file, regardless of which synth created or saved the SYSEX file. The preset files may be different, but if you were to export them as SYSEX files, they should be the same.

Of course, this doesn't mean that the same SYSEX file loaded into a real DX7, Dexed, and Arturia's DX7 emulation will sound the same, even though all three are using the same data/parameters/settings/whatever you want to call it. They will sound different because of hardware or other software differences that the SYSEX files don't address.

Steve

[Stefken]

I'm guessing a hash function should be able to identify identical presets then?

[planetearth]

I'm guessing a hash function should be able to identify identical presets then?

As far as the contents of SYSEX files, yes. But again, many files have the same name, even though they don't sound the same. For example, there are probably a couple dozen patches called "Slap Bass", "Horns", or "Bell", even though they don't sound like other patches with the same name. A hash won't help sort those out, unfortunately.

Steve

[wagtunes]

Trying to sort out duplicate sounds is virtually impossible. Here are all the different possibilities comparing two patches.

Same Name, Same Sound
Same Name, Different Sound
Different Name, Same Sound
Different Name, Different Sound

Because of 2 and 3, any kind of compare program, system or whatever would be futile in absolutely determining duplicates. In some cases you'd end up including two patches that shouldn't be (different name but same sound) and in other cases you would end up excluding patches that you should have included (same name but different sound). Without actually examining every parameter in the patch (and I don't know any program that would do this outside of a file compare) there is no way to tell. And even if you had such a program (I use Total Commander) the process would be a total nightmare to navigate through depending on how many sysex files you had to sort through.

In short, it just isn't worth the time and effort.

[Stefken]

I'm guessing a hash function should be able to identify identical presets then?

As far as the contents of SYSEX files, yes. But again, many files have the same name, even though they don't sound the same. For example, there are probably a couple dozen patches called "Slap Bass", "Horns", or "Bell", even though they don't sound like other patches with the same name. A hash won't help sort those out, unfortunately.

Steve

No, once you have removed presets with identical content you would have to fix this in a second sweep and rename all the files with the same name. Like automatically rename them by e.g. appending a number.
Like slap bass1, slap bass2, slap bass3.
I won't be perfect but at least you'll have unique names.

[planetearth]

I'm guessing a hash function should be able to identify identical presets then?

As far as the contents of SYSEX files, yes. But again, many files have the same name, even though they don't sound the same. For example, there are probably a couple dozen patches called "Slap Bass", "Horns", or "Bell", even though they don't sound like other patches with the same name. A hash won't help sort those out, unfortunately.

Steve

No, once you have removed presets with identical content you would have to fix this in a second sweep and rename all the files with the same name. Like automatically rename them by e.g. appending a number.
Like slap bass1, slap bass2, slap bass3.
I won't be perfect but at least you'll have unique names.

But how will you determine which of the 21,500+ sounds have identical content unless you listen to each one and assign a hash to each one? As Wagtunes summarized, it would be extremely difficult to tell which patches sound the same if you're only basing it upon file name or SYSEX data. And unless you have a program that will read SYSEX data (or you choose to listen to all 21,500 files), you're stuck basing your comparisons only on the file name.

Even if you renamed the files by automatically appending a number to them, unless everyone used those SYSEX files, the list would get messed up again, when your "Slap bass3" becomes someone else's "Slap bass6", and they share their SYSEX files with someone else.

Not to throw more cold water on this, but any of the freeware file-renaming apps I'm aware of won't strip off any numbers that are already at the end of file names, so you couldn't easily rename "Slap bass3" to "Slap bass4". The file-renaming apps will append numbers to the end of the file name, but you'd have to have all those files in the same folder in the first place, and you won't be able to have a dozen (or so) files called "Slap bass" in the same folder, waiting to be batch re-named.

Don't get me wrong--I wish there was a quick, easy way to do this. But as BlackWinny found out more than a year ago, there isn't. He did the best he could with automating as much as possible, using apps to compare file data, and eventually opening many files manually to compare the contents.

Steve

[Stefken]

I'm guessing a hash function should be able to identify identical presets then?

As far as the contents of SYSEX files, yes. But again, many files have the same name, even though they don't sound the same. For example, there are probably a couple dozen patches called "Slap Bass", "Horns", or "Bell", even though they don't sound like other patches with the same name. A hash won't help sort those out, unfortunately.

Steve

No, once you have removed presets with identical content you would have to fix this in a second sweep and rename all the files with the same name. Like automatically rename them by e.g. appending a number.
Like slap bass1, slap bass2, slap bass3.
I won't be perfect but at least you'll have unique names.

But how will you determine which of the 21,500+ sounds have identical content unless you listen to each one and assign a hash to each one? As Wagtunes summarized, it would be extremely difficult to tell which patches sound the same if you're only basing it upon file name or SYSEX data. And unless you have a program that will read SYSEX data (or you choose to listen to all 21,500 files), you're stuck basing your comparisons only on the file name.

Even if you renamed the files by automatically appending a number to them, unless everyone used those SYSEX files, the list would get messed up again, when your "Slap bass3" becomes someone else's "Slap bass6", and they share their SYSEX files with someone else.

Not to throw more cold water on this, but any of the freeware file-renaming apps I'm aware of won't strip off any numbers that are already at the end of file names, so you couldn't easily rename "Slap bass3" to "Slap bass4". The file-renaming apps will append numbers to the end of the file name, but you'd have to have all those files in the same folder in the first place, and you won't be able to have a dozen (or so) files called "Slap bass" in the same folder, waiting to be batch re-named.

Don't get me wrong--I wish there was a quick, easy way to do this. But as BlackWinny found out more than a year ago, there isn't. He did the best he could with automating as much as possible, using apps to compare file data, and eventually opening many files manually to compare the contents.

Steve

That's why i asked whether identical settings results in a identical file content. You have duplicate finder tools that hash each file. And a bit of a programmer could write a renamer tool that works over nested folders.

Off course once people start altering that cleaned up list and adding their own "slap bass6" you start messing it up again. Nothing you can do about that.

Happy 2019.

[aaron aardvark]

fmr and others,
Finally got it to work! Thank you for your extreme patience! Here is where I think I was screwing up earlier: not until tonight was I able to find that darn "db.db3" file. Here is the other thing that might have messed things up: the original User file (with the 9,500 patches): I changed its name to User-9500 DX7 patches. The original name (User) was so generic, I thought I would get it mixed up with other User files. I'm guessing that was a bad idea. I tossed my original files (that included only the new 9,500 patches) in the trash and started over from scratch.

[v1o]

This doesn’t sound like an impossible problem. Someone with coding skills could make a tool that compares the internal contents of the sysex files while completely ignoring the file names. Any patches with the exact same settings gets deleted and patches with the same name get intelligently renamed. It’s 2019 we have machine learning algorithms at an API level instuff like iOS and Mac OS.

[pinbot]

db.db3? Where would I find that file?

[Tappistry]

I'm guessing a hash function should be able to identify identical presets then?

As far as the contents of SYSEX files, yes. But again, many files have the same name, even though they don't sound the same. For example, there are probably a couple dozen patches called "Slap Bass", "Horns", or "Bell", even though they don't sound like other patches with the same name. A hash won't help sort those out, unfortunately.

Steve

No, once you have removed presets with identical content you would have to fix this in a second sweep and rename all the files with the same name. Like automatically rename them by e.g. appending a number.
Like slap bass1, slap bass2, slap bass3.
I won't be perfect but at least you'll have unique names.

But how will you determine which of the 21,500+ sounds have identical content unless you listen to each one and assign a hash to each one? As Wagtunes summarized, it would be extremely difficult to tell which patches sound the same if you're only basing it upon file name or SYSEX data. And unless you have a program that will read SYSEX data (or you choose to listen to all 21,500 files), you're stuck basing your comparisons only on the file name.

Even if you renamed the files by automatically appending a number to them, unless everyone used those SYSEX files, the list would get messed up again, when your "Slap bass3" becomes someone else's "Slap bass6", and they share their SYSEX files with someone else.

Not to throw more cold water on this, but any of the freeware file-renaming apps I'm aware of won't strip off any numbers that are already at the end of file names, so you couldn't easily rename "Slap bass3" to "Slap bass4". The file-renaming apps will append numbers to the end of the file name, but you'd have to have all those files in the same folder in the first place, and you won't be able to have a dozen (or so) files called "Slap bass" in the same folder, waiting to be batch re-named.

Don't get me wrong--I wish there was a quick, easy way to do this. But as BlackWinny found out more than a year ago, there isn't. He did the best he could with automating as much as possible, using apps to compare file data, and eventually opening many files manually to compare the contents.

Steve

So interestingly enough, I tried the new, giant 21,500 preset file that @PietW posted. That one loads up into DX7V without a problem.

However, the smaller BlackWinny collection does import automatically in the presets/dx7v/user folder. Each of the .syx files has to be imported individually.

And happy 2019, everybody!

[aaron aardvark]

db.db3? Where would I find that file?

This is from fmr:

OK, here is a step by step guide. Try to follow it STEP BY STEP

1. Unzip the ZIP file. You should have now a folder named User, with several folders inside it;
2. Copy the entire User folder (do not mess with the contents) to Macintosh HD/Library/Arturia/Presets/DX7 V (you don’t need to create a new folder User because you are already copying a folder called User)
3. Delete the file called db.db3 that’s in the root of Macintosh HD/Library/Arturia/Presets
4. Launch DX7 V stand-alone.
5. Wait for a few minutes while the presets database is rebuilt.
6. Click in the library icon (it's the icon at the right o the Arturia logo). Hopefully, you will see ALL the banks listed after the Factory Bank.