DAW Audio Quality does vary... and Live 7 sneak peak!

[nuffink]

If I do a 32 bit mixdown in Audition of two tracks, each with an identical file on it -
Then I do the same in Reaper using the same file as the source -
Then I invert one against the other in Audition, in that mixdown I get a file peaking to -58dB, when I'd expect silence if there was no audible difference between them.

Even more strangely, when I normalise that mixdown (the difference of the mixdown of the two DAWs) I get what sounds like the original source file back (which is a testament to the dynamic range of 32 bits).

Doesn't a simple test like that "prove" that there are differences, and possibly audible differences, between different DAW programs?

Yours innocently and probably ignorantly,

Art.

Normalise the source files and try it again.

[soulibertad]

re: routing allmidi channels to channel 1: Yes it is a bug. I banged my head against the wall trying to figure this out to route midi from GURU. However, this has nothing to do with the audio quality of Live.

[Art Evans]

Normalise the source files and try it again.

I'm happy to try that but I can't think why it should make a difference if both apps are using identical source material (and there's no effects or panning or anything else involved - just summing two stereo files). The source material is probably within a whisker of peaking to zero dB anyway.

[nuffink]

Normalise the source files and try it again.

I'm happy to try that but I can't think why it should make a difference if both apps are using identical source material (and there's no effects or panning or anything else involved - just summing two stereo files). The source material is probably within a whisker of peaking to zero dB anyway.

If you have that amount of difference in a nulling test it suggests something's fundamentally wrong. Experience says it's a gross difference in amplitude or a phase problem.

[shogger]

If I do a 32 bit mixdown in Audition of two tracks, each with an identical file on it -
Then I do the same in Reaper using the same file as the source -
Then I invert one of those mixdowns against the other in Audition, in that mixdown of the mixdowns I get a file peaking to -58dB, when I'd expect silence if there was no audible difference between them.

Even more strangely, when I normalise that mixdown (the difference of the mixdown of the two DAWs) I get what sounds like the original source file back (which is a testament to the dynamic range of 32 bits).

Doesn't a simple test like that "prove" that there are differences, and possibly audible differences, between different DAW programs?

Yours innocently and probably ignorantly,

Art.

Hey Art,

there are things that could have flawed your test. Pan laws, correct positioning of the tracks, correct positioning of the resulting tracks, etc.
It's an old fact that e.g. Steinberg hosts had (still have? dunno) a playback prob when starting at bar 1.
Pan laws create different levels in the rendered tracks and therfor you won't get complete cancelation when one rendered track is phase reversed.

Shogger

[Art Evans]

Pan law shouldn't affect centered audio, I would have thought (if set to zero gain at centre). And if what you are saying is that pan law makes an audible difference between DAWs then... there's an audible difference between DAWs - end of discussion. Correct alignment of tracks is evident from the waveforms when zoomed to sample level. I can't see any test factor which would invalidate the test with these two apps anyway. Put it this way - what test proves that at least at the most basic level, two DAWs produce the same output from the same simplest-possible processing of the same files(s)? Can anyone say "I did this with app A, the same with app B, inverted one against the other and got a null?"

[jens]

And if what you are saying is that pan law makes an audible difference between DAWs then... there's an audible difference between DAWs - end of discussion.

sorry, but that's a bit stupid - it's not the end of discussion - if you know what you are doing while mixing you should automatically compensate for different pan-laws so that in reality the possible difference between different pan-laws should not exist result-wise.

[tylenol]

And if what you are saying is that pan law makes an audible difference between DAWs then... there's an audible difference between DAWs - end of discussion.

That's not really the end of the discussion, because the difference is only in the level of the audio (so they will usually be 3 or 6db off, with no other differences, depending on the pan law). The sounds will otherwise be bit for bit identical (I guess modulo rounding errors or whatever), and you can see this pretty straightforwardly if you use hosts that allow you to change the pan laws. You could even just compensate for the pan law in centered audio simply by adjusting the levels correctly.

The reason pan laws have to be compensated for in this kind of test is that there is a weird but well-known fact about sound perception: in blind tests people will judge the louder of two otherwise identical sounds as sounding "better". They will do this even when they can't reliable perceive that there's an actual level difference. (This is in large part because loudness affects the frequency response of the ear, and high/low freq. content is more audible at overall higher levels.) So if you're trying to show that host x sounds "better" in some sense than host y, you absolutely must control for the effect of a simple level difference.

[Art Evans]

OK, in that case we've eliminated pan law as making a difference (set to zero gain at centre, and the inaccuracy that I'm getting is certainly not the result of a 3db let alone 6dB difference) - we've eliminated alignment as making a difference, that's checked - I've not had a reason offered for needing to normalise - so why is there a difference in the rendered output? Is Reaper wrong or is Audition wrong? Can this very simple test succeed with any two DAWs? If we found two DAWs that matched, would it be safe to assume that they were the correctly coded ones, or would it be a matter of them both being equally incorrect?

Actually I've though of a way of pinning this down a bit with single sample transients - I'll try that later.

Don't get me wrong, I'm entirely on the side of those who say there is, or should be, no audible unexplained difference between the sound of two DAWs - but it should be very easy to prove that there can't be, and I'm having difficulty doing that (at least with the two to hand).

[jens]

so why is there a difference in the rendered output? Is Reaper wrong or is Audition wrong?

hard to say - but there are only two possible causes:

a) you made a mistake, or

b) at least one of the two applications you used for rendering has got a serious bug which in some way corrupts the rendered file

Can this very simple test succeed with any two DAWs? If we found two DAWs that matched, would it be safe to assume that they were the correctly coded ones, or would it be a matter of them both being equally incorrect?

If both files give out the same result I see no reason to assume that they might not be working correctly.

fyi: we did ths summing test several times in the past and all hosts we tried nulled at least to ~ -150db so personally I don't see any reason to go through all this again - but if you insist on it I'm all game.

[Art Evans]

Heh, I realise I'm perhaps testing patience here...

I did a stunningly simple test - created ten seconds of silence (mono, 32 bit float, 44.1) in Audition. Take a sample somewhere around the middle of it, and give it a sample value of 2150400 (24 bit range). That's pretty close to -12dB. So in audio terms, it's ten seconds of silence with a -12dB click in it.

Insert that file into Audition's multitrack twice (on two tracks) and double check that the click is in the same place on each track by zooming in to sample level. Mix down and note the resulting single sample click sample value - 3041124.8 - which is very close to -9dB.

Repeat the previous paragraph in Reaper, and note the result - 3044733.3 - which is (when you play it) also very close to -9dB, but not exactly the same.

The fact that the results are very close seems to me to eliminate the usual test error suspects - except Reaper has a 64 bit engine and Audition 32 bit, I think.

I wonder whether anyone here knows enough about the math that goes on in DSP to manually calculate the correct sample value you'd expect from summing a sample of 2150400 with itself? Or is the difference in the output from the two DAWs within the range of expectation?

Edit - -3dB centre pan law operating in both programs, so the calculation would have to take that into account. Hmmm, although the playing field was (I'd say) level, perhaps I should retest with the pan law set to zero.

[jens]

Art, did you render without dithering in both applications? (I don't use Audition so I don't know but e.g. Samplitude dithers by default when rendering)

[Art Evans]

OK, taking the pan law to zero dB with the same original click (value 2150400) I get a summed result in Audition of 4300800 which is clearly correct, and the same in Reaper - whew!

So... when the pan law in each app is in operation, although it's dropping the level of each track in each app by -3dB, it's a different -3dB (slightly) in each. Maybe that's where the rounding error thing starts to appear, when the calculation is no longer quite as simple as 1 + 1 = 2?

And why did it appear not to come out the same with a music file test? Obviously I'll retest that next...

[Art Evans]

Reaper doesn't dither, as I understand it, and I think Reaper doesn't either when in 32 bit float (as, in my limited understanding, you wouldn't need to dither in that scenario anyway). But indeed, there's room for error on my part or difference in the programs' part there. Thanks for following along with these simple attempts to get things 100% clear in my mind!

[Art Evans]

Now using a stereo file, with panning law set to 0dB, I'm not getting a null, using exactly the same procedure as I did with the mono click file. The difference in this test is that it's stereo and it's music, so maybe I need to look more closely at what happens with a stereo click file or a music mono file.

With the music file I can actually zoom in to a particular identifiable sample and look at what's going on. The summed result from the two apps for that sample are...

Audition 5587968
Reaper 5588138.5

The original sample value before adding it to itself was 2793984, so Audition's result is correct and Reaper's result is wrong (subject to my test procedure in Reaper being correct). Very strange... it's getting late at night here so I may have to return to this tomorrow...