DAW Performance Testing

[Serhii Kot]

I have a question, do we have some standard of testing to test different DAW performance on a particular system?

I made improvised test, not scientifically relevant but, relevant in terms of real life test.
Testing machine: MacBook Pro 2021 M1 Pro (10c), 16 gb of ram, 1tb ssd.
Idea of a test: CPU hungry synth, with stock EQ and Compressor. Duplicating to audio system overload. In this case it was a Pigment synth, setting on EQ and Compressor identical thru all DAWs. All apple silicone native software. 128 samples buffer size. Built in Audio. Here is the results with screenshots of an activity monitor (processor and memory).
From best to worse performance on my machine.

1-Reaper.png

2-Logic.png

3-Cubase.png

4-Ableton.png

5-Studio One.png

6-Bitwig.png

[Amon1973]

You didn't look for an easy task, did you Creating simple tracks/instruments tracks and duplicate up to overload works, it is more of a 'number crunching' test than real-world, but works. One can also create projects with say 4, 8, 16 and 24 tracks going to groups, some instruments, and duplicate. But keeping track and not make errors gets very complicated.

What's really difficult is to match settings between DAWs. Besides matching the buffer size, one would need to match the pre-fetching, pre-processing or whatever every DAW calls it. Reaper, for example, has a default of 200ms, way bigger than what you can set in Studio One or Cubase with buffer size of 128, for example, meaning it has an advantage. Logic is not as transparent on pre-processing settings, no clue how to match it. Also no clue on how Live and Bitwig work on that account.
These features have a huge impact, as they effectively raise the buffer size to a whole different value, except for 'live tracks' (and usually one has none when doing these tests, that would belong to a real-time test).

This does not really answer the question about a standardised test, but just wanted to throw it out there.

[Serhii Kot]

I think this way of testing has its use. It shows how different DAWs handle similar task. Yes, their stock plugins may differ with cpu usage, but I as a client want their software optimised for its task. If some DAW finds a way to get more performance from same hardware, without visible or sensible limitation for a user - great.

For me, this test has some representation of subjective feel of snappiness or responsiveness of the DAW, Bitwig for example fells sluggish, and Logic and Reaper feels fast and responsive.

But it would be grate to have some unified test for DAWs.

[Amon1973]

It absolutely has its uses, I prefer 3rd-party plugins that are available for all DAWs, as the same kind of processor may have very different performance. Beside matching settings, which as said have a very big impact.
All tests have their uses though. Have fun

[rardier]

i think what Armon1973 means, is that for example @128 buffer size cubase has not the same overall amount of latency with or without asio guard and with different level of asio aguard (for none live track=> monitor or rec armed)
For reaper i think there s also an extra adaptive buffer size added to the 128 , but the only pb i found with reaper (on windows don t know on mac os) is that when you reach the limit it s the whole system that become laggy not only reaper.

so if you are only mixing you don t really care about the latency except maybe for automation.
But when recording (especially virtual instrument) it can be annoying.

i think there s already an unified test protocol with a wav file and free vst (sga1566), it s better to use a free vst like that everybody can make the test

[mgiambro]

For reaper i think there s also an extra adaptive buffer size added to the 128

I was wondering about this because I'm curious to know what kind of sorcery the Reaper guys are using to make Reaper so good at handling multiple hungry vsts in a stress test. When I've done stress tests my results have pretty much mirrored those of Serhii Kot. Reaper is able to handle more than double the amount of tracks with Repro 5 playing a sustained triad than certain daws, and at least 25% more than the next best. It isn't just the anticipative processing either because I have switched that off and set all those optional buffer settings to zero and Reaper still blows the other daws away when it comes to handling multple cpu-hungry synth tracks. And obviously I made sure the audio interface sample rate and buffer settings are the same for each daw.

[Amon1973]

i think what Armon1973 means, is that for example @128 buffer size cubase has not the same overall amount of latency with or without asio guard and with different level of asio aguard (for none live track=> monitor or rec armed)
For reaper i think there s also an extra adaptive buffer size added to the 128 , but the only pb i found with reaper (on windows don t know on mac os) is that when you reach the limit it s the whole system that become laggy not only reaper.

so if you are only mixing you don t really care about the latency except maybe for automation.
But when recording (especially virtual instrument) it can be annoying.

i think there s already an unified test protocol with a wav file and free vst (sga1566), it s better to use a free vst like that everybody can make the test

What I meant is that besides the device buffer setting (e.g. 128 samples), you generally have a pre-processing feature - when mixing, or stress-testing with no tracks/instruments working in real-time, your buffer is not that of the device, but the device buffer plus the pre-processing buffer. In Studio One you have your buffer plus 'Dropout Protection', in Cubase buffer plus 'ASIO Guard', in Reaper you actually have a few options that affect both pre-processing and the CPU behaviour/thread priority, media buffer and allow to target tracks w/o FX and MIDI tracks with open editors. Not that I think you didn't understand what I wrote, just to make it clearer.

When testing with no record-armed tracks/live instruments, the host is not processing with the device buffer, but with the total buffer and that's what need to be matched to have a fair comparison. Studio One actually displays both values in samples, Cubase displays the ASIO Guard latency in ms, Reaper also displays Anticipative FX Processing in ms.

Not citing Logic again, just because it has a 'Process Buffer Range', but I never found how large it actually is.

[Trancit]

Nevermind...I should learn to read...

[Serhii Kot]

Similar test for fl studio 21:
14 instances...

[machinesworking]

Your numbers are about where I always come out at as well.

One caveat with this is that different VST/AU's react differently in different DAWs.
For instance last time I checked U-He synths do really really well in Bitwig, whereas Reaktor does not. Reaper with the normal tweaks does a little better than the other basically dual buffer DAWs, and all the "real time" DAWs like Live, Bitwig etc. do a lot worse.

My conclustion was that with the same settings and the normal tweaks and optimizations Reaper being 100%, DP would be 95% Logic 90-92, Bitwig 65% and Live 60%. Makes sense, Reaper, Logic and DP have aggressive dual buffing, Live and Bitwig do not have any of that. They do however behave a lot better when the sequencer is running, almost zero audio glitches when adding instrument's etc.

[Serhii Kot]

Reaper utilises 90% of cpu in this case, as you can see on the screenshot. While others under 70% And S1 only 55%, FL studio was 35% and started glitching.

[machinesworking]

Reaper utilises 90% of cpu in this case, as you can see on the screenshot. While others under 70% And S1 only 55%, FL studio was 35% and started glitching.

Yeah like I mentioned, different plug ins will give slightly different results, but the average being Reaper at the top, Logic and DP second place, Cubase and other traditional DAWs third, and Live Bitwig, FL, MPC 2, Reason at the low end plug in count wise. These numbers change for PC users somewhat, but Reaper remains the CPU king, and Live is usually dead last here. IMO FL Studio for Mac is still a work in progress, but I don't really use it so I don't know the tweaks to get it running well if there are some.