Integer is King? - the challenge

[living sounds]

Just check the threads I posted recordings made with (hardware) FM synths and reverb. The funny thing is that most of the "plugins rule supreme"-apologists stop posting in a thread where actual evidence contradicting their view is presented.
And have you even bothered to check out the TDM-vs-RATS-files from the other thread?

[stefancrs]

Has any actual double-blind testing been done yet?

I used my ABX tool (see http://en.wikipedia.org/wiki/ABX_test) where only the computer knows exactly what 'X' is. So it's more or less double-blind. I only did it one time with 15 trials (since I'm notorious out of time). 11 times out of 15 times I heard a difference. That may still be coincidence. Plus I couldn't really say what it was that I've heard here...

The best would be if thorkz came around presenting some real double blind results...

You didn't ever hear a difference when there wasn't one? As in you clicked on "new take" or whatever it's called in your ABX tool, and it actually didn't change the algorithm, and you noticed a difference anyway?

[Christian Budde]

This is completely significant but, I suspect, not in the way you think. Do yourself a favour and read that link that Meffy posted and you'll realise that in pre-empting the outcome you've already invalidated this as a scientific test. You are already showing the experimenter effect.

Ok, than please read the link I supplied and you can see, that using an ABX test I didn't had any clue about what was what. I couldn't cheat, even if I wanted to.

[Meffy]

@Christian: Good enough, that mechanism sounds... erm, "sound" enough to me for a one-person test. My comment wasn't so much concerning the specifics as ensuring people were speaking the same language -- [reverb="too much"]the Language of Science![/reverb]

[nuffink]

This is completely significant but, I suspect, not in the way you think. Do yourself a favour and read that link that Meffy posted and you'll realise that in pre-empting the outcome you've already invalidated this as a scientific test. You are already showing the experimenter effect.

Ok, than please read the link I supplied and you can see, that using an ABX test I didn't had any clue about what was what. I couldn't cheat, even if I wanted to.

I'm not suggesting that you're cheating but, in any test situation where the results are leaked before the cutoff, those results will sway the outcome. It's basic test procedure and it matters not one jot whether your expeiences were positive or negative. You've skewed the outcome.

[Christian Budde]

You didn't ever hear a difference when there wasn't one? As in you clicked on "new take" or whatever it's called in your ABX tool, and it actually didn't change the algorithm, and you noticed a difference anyway?

It still all maybe coincidence and trust me I really hope it's all coincidence. That would make everything much easier. If I would have heard 15 times out of 15 times a difference than I would say that I really can heard a difference.
I'Ve done the ABX test prior to any further SNR analysis. If I'd repeat it now, I'd surely would loose myself in testing. It was just the kind: 'Let's see, what happens if I do an ABX test'. I didn't expect anything and now I better stop saying anything.

[JonHodgson]

But you can't just swap doubles for ints in a filter and expect it to sound the same.

Please educate me. Why can't I do that? I didn't expect that they sound the same though. I know that there are differences and that more care is necessary (at least a dither/noiseshaper). However they do sound very close although difference can be measured.
Furthermore at least 2 people seemed to hear a difference. This is far from being significant yet.
While educating me, please don't forget to suggest modifications.

Thanks,

Christian

PS: I know that I can't use the harmonic distortion for that type, but it's the only method which came into my mind and I could clearly see differences here.

You're feeding different coefficients into the filter, then feeding back different errors.

In a certain range the differences will be negigible, but at more extreme settings they can become significant, especially if your coefficient is very small (you might end up with very few bits of precision in your integer version, but still the same number in the double).

[Meffy]

BTW, I'm not on any "side" in this. My ears are plain old steel, not even gold plated, so as a rule I can't hear the fine differences y'all can. Just so nobody thinks I'm rooting for Mancs or Arsenal or the Toledo Mud Hens.

[JonHodgson]

Try setting one of the extreme settings that you think you've heard a difference at, then feeding an impulse through it, then analyze the differences between the outputs with different internal processing.

[Christian Budde]

You're feeding different coefficients into the filter, then feeding back different errors.

In a certain range the differences will be negigible, but at more extreme settings they can become significant, especially if your coefficient is very small (you might end up with very few bits of precision in your integer version, but still the same number in the double).

What do you really suggest? It's a test integer vs. double. I know that both formats are not equal and thus will result slightly different results. If both were completely equal, I do not need to do this test.
If someone can reliable hear a difference than one can investigate to see, what that different is. Right now the difference seems to be by far inaudible. Something below -130dB is in fact masked by the DA converter. But please no discussion about DA converters again.

[Christian Budde]

Try setting one of the extreme settings that you think you've heard a difference at, then feeding an impulse through it, then analyze the differences between the outputs with different internal processing.

Suprisingly there was no differece here! I did it already. I mean not the same setting, because my ABX test was just something like: let's see what we get.
I produced an extreme setting an compared both IRs. I didn't plot the difference, but I did a power plot of the IR and compared it 'by eye'. They looked damn equal I can say.
However the harmonic distortion plot does not. I could see a differecne of 10dB there.
I'll post my plots later, I promise.

[JonHodgson]

You're feeding different coefficients into the filter, then feeding back different errors.

In a certain range the differences will be negigible, but at more extreme settings they can become significant, especially if your coefficient is very small (you might end up with very few bits of precision in your integer version, but still the same number in the double).

What do you really suggest? It's a test integer vs. double. I know that both formats are not equal and thus will result slightly different results. If both were completely equal, I do not need to do this test.
If someone can reliable hear a difference than one can investigate to see, what that different is. Right now the difference seems to be by far inaudible. Something below -130dB is in fact masked by the DA converter. But please no discussion about DA converters again.

The point is that an "integer vs double" test is basically complete bollocks.

You're not comparing integers to doubles, you're comparing one set of mathematical errors to another set of mathematical errors, not only that but when you get into the idea that one is better than the other then you are making a subjective judgement of the results of those errors... which incidentally are going to be different in the case of any different filter architecture, choice of coefficients, input signal, whatever.

[nuffink]

Just check the threads I posted recordings made with (hardware) FM synths and reverb. The funny thing is that most of the "plugins rule supreme"-apologists stop posting in a thread where actual evidence contradicting their view is presented.
And have you even bothered to check out the TDM-vs-RATS-files from the other thread?

I'll give it one more go.

In order for any valid comparative test to occur the Devices Under Test must be similar. This means, for instance, that their outputs must be closely matched for amplitude.
On a device as complex as a synth as soon as you turn a dial the devices are so different as to make comparison daft. You are in the realm of the subjective.

Now you can believe that any hardware is better than any software (even if means that the shittiest little plastic casio is better than Jon's Minimonsta). That's up to you.

What you can't do is compare them objectively and say that one's better than the other. There is no common frame of reference.

[JonHodgson]

Oh, and anything which is 130dB below full scale is not only masked by the DA converters, it's also masked by the rest of the electrics in your system, the cars passing outside, your own breathing, and the fact that you are human ( I assume ) and don't have bionic ears... any one of these is more than enough to mask that

[Kingston]

And have you even bothered to check out the TDM-vs-RATS-files from the other thread?

just goes to show how clueless you are. what's happened here is that christian budde has nearly completely eliminated any guess work that would happen in audiophile RTAS vs TDM threads. you cannot possibly know what was done in those plugins other than what DSP processor they used (which means almost nothing to a programmer, but seems to make a world of difference to a golden ear like you).

hence clueless remarks like yours have absolute no ground in a thread like this.

It would seem you would rather keep guessing than being presented with hard evidence. your ears have already failed. only a shaky belief system remains.