44.1 vs 96khz music - double blind study conducted...

[Locus M]

About 50% of responses were correct, but whether this effect is produced by chance, guessing or a difference in perception we do not know. This just means that we have no valid results from a statistical point of view.

sorry, but it seems you don't know much about statistics. do you?

Well, sorry if I was unclear and you got me wrong here! The test conclusion is valid from a stats sign. of course. What I meant was the correct responses (about 50% after all) can not be said to have been caused by a difference in perception by statistical validity, because they are not above chance. However, it does not mean that they actually WERE produced by chance, of that we will not know, and thus its open to speculation if you like. The test states what it states: That the correct responses can not be proven to be dependent on an actual difference in perception with statistical validity, because then they would have to be above the level of chance.

Since I do not really know what produced the correct responses, I am inclined to keep the posibilities open a little while yet untill further evidence and not make too exclusive conclusion besides from the point that I do not believe audiophlia to be worth it or have a an effect with respect to liking music or not anyway.

For instance a training of the subjects would do it for me. If they can not exceed the level of chance in correct responses after a month or two with this test, well then it sure will make the initial conclusion even stronger and I would go for the "audiophlia is pure imagination" hypothesis too.

[tony tony chopper]

I mean i could argue that my cochlea can pick up satelite TV

mpeg2 or mpeg4?

[zeoy]

About 50% of responses were correct, but whether this effect is produced by chance, guessing or a difference in perception we do not know. This just means that we have no valid results from a statistical point of view.

sorry, but it seems you don't know much about statistics. do you?

Well, sorry if I was unclear and you got me wrong here! The test conclusion is valid from a stats sign. of course. What I meant was the correct responses (about 50% after all) can not be said to have been caused by a difference in perception by statistical validity, because they are not above chance. However, it does not mean that they actually WERE produced by chance, of that we will not know, and thus its open to speculation if you like. The test states what it states: That the correct responses can not be proven to be dependent on an actual difference in perception with statistical validity, because then they would have to be above the level of chance.

Since I do not really know what produced the correct responses, I am inclined to keep the posibilities open a little while yet untill further evidence and not make too exclusive conclusion besides from the point that I do not believe audiophlia to be worth it or have a an effect with respect to liking music or not anyway.

For instance a training of the subjects would do it for me. If they can not exceed the level of chance in correct responses after a month or two with this test, well then it sure will make the initial conclusion even stronger and I would go for the "audiophlia is pure imagination" hypothesis too.

Classic (or frequentists') statistics work like this:
you state a null hypothesis (called usually H0) which in most cases you want to reject in favor of the alternative hypothesis (H1).
In the specific study the null hypothesis is: "there is no perceivable difference between 96K and 44.1K material thus the probability of a correct guess in a blind test is 50%"
Then the researchers collect their data and they calculate the proportion of correct answers. Their statistician has to estimate what is the probability of seeing the observed proportion of correct answers (or a more extreme proportion) if the null hypothesis was true. If this probability is low enough (usually less than 5%) we tend to reject the null hypothesis in favor of the alternative.
Given the number of correct answers vs. total experiments I saw in the Mix article and without having read the original paper I'm sure that we cannot reject the hypothesis that these data are the result of pure chance.
Simply put these results show that the probability of a correct guess is very close to 50% which is the same as when you're trying to guess the result when tossing a fair coin. You'll find the correct answer with probability 50% (but the wrong answer with exactly the same probability too, unfortunately)

As for training the individuals taking part in this experiment I'm not sure how can you train someone to discriminate the two formats correctly if there is no preceivable difference. Seems like trying to train a blind man to correctly guess if the ball he holds in his hand is red or blue.

I'm not an audio engineer or anything similar but I know a couple of things about research methodology and this study seems very well conducted and it's already published in a peer reviewed journal (with not a single objection/comment from the reviewers, which, believe me, is very rare in acadenmic journals)

PS I'd be curious to see the statistical methods section of the original paper since the analysis of such data (repeated tests with a binary outcome for each individual) is not trivial even if it seems so.

[Locus M]

I'm not an audio engineer or anything similar but I know a couple of things about research methodology and this study seems very well conducted and it's already published in a peer reviewed journal (with not a single objection/comment from the reviewers, which, believe me, is very rare in acadenmic journals)

I do not say otherwise! I support the conclusion. Go read my first post! However you would not say the correct responses there are proven to be by pure guessing would you? They are not above chance = chance? How do I reach this conclusion? Even so that should not prevent anyone from trying to train the ability if they want to validate this further, this could confirm or disconfirm the conclusion. Nothing wrong with that.

[zeoy]

I'm not an audio engineer or anything similar but I know a couple of things about research methodology and this study seems very well conducted and it's already published in a peer reviewed journal (with not a single objection/comment from the reviewers, which, believe me, is very rare in acadenmic journals)

I do not say otherwise! I support the conclusion. Go read my first post! However you would not say the correct responses there are proven to be by pure guessing would you? They are not above chance = chance? How do I reach this conclusion? Even so that should not prevent anyone from trying to train the ability if they want to validate this further, this could confirm or disconfirm the conclusion. Nothing wrong with that.

I have not read the whole thread and I wasn't even aware which side are you on. I just made a comment about statistics which happens to be my area.
Now about your question: yes I believe the correct answers were pure chance (good luck) just as the wrong answers were due to chance too (bad luck). Think about that coin toss again. There's no magic or some special ability if you guess the result 5 or even 6 or even 7 out of 10 times. Now if someone can guess correctly 600 times out of 1000 then there's a very high probability he has some special skills.

Your idea about "training" the individuals is not bad but given the initial results I tend to believe there's no way one can train them really (think again the example with the blind man: if you're sure he's blind, no matter how hard you try to train him, he will never have greater than 50% chance, on average, to guess correctly the color of the ball)

In general, I tend to believe results published in peer reviewed journals (esp. if they're respected ones and there is no conflict of interest e.g. I wouldn't believe a study showing that 96K is the holly grail if it was funded by some hifi brand which has just released their new line of SACD players). Of course if the authors did something wrong or missed some detail some other team can try to publish new results showing something different. That's how it works for many many years of research.

[hifiboom]

Hi zeoy,

I think that are good points.

what you call "training" is what I meant earlier in this threat the the test person may not be use to the listening environment.
----

So I think you cannot fool the maths of such tests, but you can do wrong final conclusions depending on the specifications of a test.


Lets say you do compare a HD quality picture with a normal TV picture. You do a test with a distance of 30 feet. and a 50" screen. 49% of the test persons identified the wrong picture as the hd version and the other 51% did identified right.
Now a conclusion like " nobody can see the difference between HD and normal picture quality" would be a faulty conclusion
but "from a viewing distance of 30 feet, nobody can detect the difference between HD and normal picture" would be a valid statement.
----

Now with this audio test its a bit more difficult. The test persons may likely not be used to the listening environment (maybe training needed) and so couldn't hear any audible difference at all.

It may be possible that these test persons wouldn't even have heard a difference between a 128kbit mp3 and an redbook audio cd on this system, not because they have unsensitive ears, or the speakers are bad, but simply because they are not used to the speaker environment.

----

Anyone every bought new speakers and tried to decide in the shop which one to go for?

its a very tuff decision and you can easily think you have found your favorite speaker and at home it turns out after 2 days that you don't like its sound.

Good speaker shops normally allow you take the speaker with you to your home to check it for a longer time period.
----

so the training aspect should really not be underrated.

[Locus M]

I have not read the whole thread and I wasn't even aware which side are you on. I just made a comment about statistics which happens to be my area.
Now about your question: yes I believe the correct answers were pure chance (good luck) just as the wrong answers were due to chance too (bad luck). Think about that coin toss again. There's no magic or some special ability if you guess the result 5 or even 6 or even 7 out of 10 times. Now if someone can guess correctly 600 times out of 1000 then there's a very high probability he has some special skills.

I do agree that you have all the statistics you need to "support" your claim, but I am asking for a "proof".

As far as I am concerned the argument for statistic significance goes like this: If the results could be achieved by chance, then you have not any proof that it is NOT chance. True, no one can claim there is proof for the effects in question to be based on perception, because they could be achieved by chance. But this does not mean that it is PROVEN to be chance:

Premise: The results of the test COULD be achived by flipping a coin (or guessing), Conclusion: Therefore the subjects flipped a coin (or were guessing)

This I believe to be a fallacy. Truth is you do not know for sure, but you have all the statistical evidence you need to support your belief in contrast to one that will claim that the results were based on perception, of which they have no statistical proof at all.

[zircon]

Now with this audio test its a bit more difficult. The test persons may likely not be used to the listening environment (maybe training needed) and so couldn't hear any audible difference at all.

How would being used to the testing environment have anything to do with the result, providing the test environment is well-constructed and designed? Even if it WAS a factor, there were ~550 experiments conducted. Considering the sample size that means that each subject would have been tested nearly 10 times each. Surely by the second or third experiment they would have been acclimated. But more importantly, audio engineers and audiophiles with their "refined" ears should be able to hear things regardless of monitoring setup or environment.

It may be possible that these test persons wouldn't even have heard a difference between a 128kbit mp3 and an redbook audio cd on this system, not because they have unsensitive ears, or the speakers are bad, but simply because they are not used to the speaker environment.

This is just complete nonsense. If this happens to YOU, then there is a serious problem with you and you only, because not a single person I have ever met (or read about) has experienced that significant a loss of hearing acuity simply by being in a new environment. In fact, audio professionals and studio musicians regularly move back and forth between studios and venues, and must RELY on their ability to adapt to any new monitoring setup or acoustic situation. And you know what? They do. Easily.

[hifiboom]

@Hskovlund
thats why it is statitical/stochastical tests.

thats called "probability value" or "p-value". and gives a the chance value, that you are wrongly accepting a hypothesis as right although it is wrong.

so you cannot really prove something to 100% with significance tests.

its no real science. And you can easily make tests do deliver the results you want to have.

[hifiboom]

It may be possible that these test persons wouldn't even have heard a difference between a 128kbit mp3 and an redbook audio cd on this system, not because they have unsensitive ears, or the speakers are bad, but simply because they are not used to the speaker environment.

This is just complete nonsense. If this happens to YOU, then there is a serious problem with you and you only, because not a single person I have ever met (or read about) has experienced that significant a loss of hearing acuity simply by being in a new environment. In fact, audio professionals and studio musicians regularly move back and forth between studios and venues, and must RELY on their ability to adapt to any new monitoring setup or acoustic situation. And you know what? They do. Easily.

I didn't say anthing about me.

but such a "check test" in front would have at least shown that the probands could hear a difference at all in the dedicated listening environment, before you start checking the differences between the obviously very little audible differences of 44khz vs 96khz.

[TotcProductions]

The "vaguely definable and often irrelevant adjective" I like the least is "transparent". If your audio is visible, you're on acid.

My favorite quote so far on KVR. Nice Pough!

[hifiboom]

The "vaguely definable and often irrelevant adjective" I like the least is "transparent". If your audio is visible, you're on acid.

My favorite quote so far on KVR. Nice Pough!

[zircon]

As far as I am concerned the argument for statistic significance goes like this: If the results could be achieved by chance, then you have not any proof that it is NOT chance. True, no one can claim there is proof for the effects in question to be based on perception, because they could be achieved by chance. But this does not mean that it is PROVEN to be chance:

Premise: The results of the test COULD be achived by flipping a coin (or guessing), Conclusion: Therefore the subjects flipped a coin (or were guessing)

But some conclusions are more rational than others.

1. The people who conducted the test did not note that any individuals had a particularly high success rate. I think they would have, if it was statistically significant compared to the average.

2. The individuals with the best hearing (>15khz) fared poorly. If there WAS a difference, these people logically should have had the highest, not lowest, success rate.

3. Let's assume your hypothesis, that some people were consistently successful, was correct. This would have been demonstrated in the statistics. For example, if 50% of the people could detect a difference, and 50% could not, you would still see an average success rate much higher than 50% - the people who could reliably tell a difference would, over the course of all the experiments, heavily influence the average. The people that could not reliably detect a difference - and thus, would be guessing - would not bring the average down to 50-52%. For example...

* Group A (5 individuals) cannot detect a difference. Their success rate is 50% over the course of 50 trials.

* Group B (5 individuals) can detect a difference reliably (80%).

The average success rate would still be 65%. For the average rate, even among the audiophile/engineer group to be only 52%, it is not possible that there was any significant number of people able to reliably detect a difference, and/or that there was a smaller number of people able to detect a difference at a very high rate.

[Lunch Money]

It absolutely exhausts me just imagining I'd be in TTC's place. To debate against such utter lack of logic...

how wearisome.

[DWb]

@Hskovlund
thats why it is statitical/stochastical tests.

thats called "probability value" or "p-value". and gives a the chance value, that you are wrongly accepting a hypothesis as right although it is wrong.

so you cannot really prove something to 100% with significance tests.

its no real science. And you can easily make tests do deliver the results you want to have.

True. A well conducted experiment with a thorough statistical analysis concluding with a high probability that the results were effectively random and not based on any percieved difference in the sounds isn't real science. Real science is directional cables, $1500 power cords, edge markers, chicken bones and pennyroyal oil, stuff like that...