DAW Stress Test: Logic/ProTools/StudioOne/Cubase

[Jim Rosebrook]

Now bootcamp into Windows and run the same tests. How do Studio One and Cubase fare?

I've thought about doing this out of curiosity... but that would likely chew up an entire weekend. Maybe in the winter sometime..

[Jim Rosebrook]

Any stress test for a DAW and/or your computer setup for DAWs should be to failure. There are a massive amount of reasons for this, DAWs do not use CPU cores disks etc. in a way that relates directly to System tests 100%, it's a great indicator but it's not going to tell you too much.

A great example of this is how various DAWS will perform near 70% CPU readings in their own built in meters etc. Some will get massively better results out of that last 30% than others, as in you can load the same amount of plug ins, (another seeming "70%"), into the DAW and others will only give you 15% more.

I understand the concept of the different DAW's not having the same linear relationship of more track count = more CPU usage, but within my limited testing, this didn't emerge as a huge distinction.

When I increased the track count from 25 to 30 (one of each type in my example), Cubase was the first to start showing some flakey behavior, and have some runaway temps.

Using the Intel Power Gadget was helpful in that it gave the same measuring stick for each DAW vs the CPU metering built into each DAW.

[Dewdman42]

Its somewhat true that one DAW could be more or less efficient then another...and still result in plenty of track playback without problems..its just that one daw is not as efficient that's all. In some cases you can build up so many tracks that it doesn't really matter that one DAW is slightly less efficient then another because they can both playback 70 tracks with lots of plugins, etc. Meh.. but.. when I did my tests a few months ago, Cubase 10.0.20 could not play 70 tracks while the other DAW's were handling 100. Cubase with its higher CPU usage definitely started to crap out before the others. So.. There is still something to be said about CPU efficiency. Some DAW's are doing things more efficiently than others...which may or may not be a problem for everyone. I think most of the DAW's are pretty darn efficient now and can handle a lot. Steinberg did fix Cubase with the next release to be more efficient on the mac right in line with everything else. Did they see my test report on their forum right before that? Who knows. But clearly it was not efficient enough for playing back an orchestral mockup at that time until they fixed it later. It was using double the cpu %usage and it crapped out way sooner then the others..I didn't take them all the way to crap out stage to see how far they would go.

Another interesting test would be to attempt to max them all out. How many simultaneous tracks until LogicPro starts dropping out, at each buffer size. Then use the same tracks, same plugins, etc.. in another daw...same thing...how many tracks until it starts crapping out. Etc. I suspect that the above cpu % usage would correlate pretty well with which DAW's would crap out in which order. However that being said, if we are talking one DAW can play 150 tracks and another DAW can play 175 tracks...does it really matter? Most people aren't mixing 200 simultaneous tracks. So pick your flavor with the workflow you like..they all work very well.

[machinesworking]

Any stress test for a DAW and/or your computer setup for DAWs should be to failure. There are a massive amount of reasons for this, DAWs do not use CPU cores disks etc. in a way that relates directly to System tests 100%, it's a great indicator but it's not going to tell you too much.

A great example of this is how various DAWS will perform near 70% CPU readings in their own built in meters etc. Some will get massively better results out of that last 30% than others, as in you can load the same amount of plug ins, (another seeming "70%"), into the DAW and others will only give you 15% more.

I understand the concept of the different DAW's not having the same linear relationship of more track count = more CPU usage, but within my limited testing, this didn't emerge as a huge distinction.

When I increased the track count from 25 to 30 (one of each type in my example), Cubase was the first to start showing some flakey behavior, and have some runaway temps.

Using the Intel Power Gadget was helpful in that it gave the same measuring stick for each DAW vs the CPU metering built into each DAW.

So again, to failure is the only way to get measurable results. The reasons for this are not entirely obvious and in no way is your thinking on this "wrong" it's just that the way plug ins instantiate in DAWs is not based on a linear path per say. So a smaller than to failure track count will give seemingly random results. Examples as I gave already is/are that a DAW will show higher usage CPU wise with just a few plug ins, and will not show a significant increase until more than you would expect are added in.

System CPU meters are interesting as it mostly points out that on modern computers at least in my experience, you're not going to get a lot of track sharing, it's possible to get more CPU out of tracks are 60-70% but it's also possible to get readings and results in the red with system CPU reporting plenty left.

Mostly various DAWs will do better or worse in tests of any kind with less than failure results, but that doesn't always translate to the way they behave with track counts right below issues like crackling, drop outs, and GUI slow downs etc. In my testing most DAWs are pretty close these days, with Reaper slightly better. Modern or "performance oriented" DAWs like Live, Bitwig etc. have a CPU hit though, and it's noticeable.

[dupont]

I did a very similar test a few months ago, the results are on Google Sheets here:



Full data here: https://docs.google.com/spreadsheets/d/ ... 1483189818

Reaper was not the most efficient on mac, sorry to tell the Reaper fans. LogicPro beat everything. Cubase was particularly bad when I first tested it on 10.0.20, but then shortly after I reported this awful results, Steinberg released 10.0.30 with greatly improved performance, which is shown in the graph above(the improved performance).

Suprising to learn Logic X lighter on the CPU than Reaper.

[machinesworking]

I did a very similar test a few months ago, the results are on Google Sheets here:



Full data here: https://docs.google.com/spreadsheets/d/ ... 1483189818

Reaper was not the most efficient on mac, sorry to tell the Reaper fans. LogicPro beat everything. Cubase was particularly bad when I first tested it on 10.0.20, but then shortly after I reported this awful results, Steinberg released 10.0.30 with greatly improved performance, which is shown in the graph above(the improved performance).

Suprising to learn Logic X lighter on the CPU than Reaper.

It's not.

I'm not trying to be argumentative here, but tests of resting CPU aren't going to prove much. It's how many plug ins a DAW can run before failure that matters, that's it. All other tests are just showing how much CPU a DAW grabs to run plug ins, not what the added effect of those plug ins will be.

DAW X might read that it has a heavy CPU load with 10 instances of some plug in, and DAW Y will have a lighter load, seemingly. When those 10 plug ins are increased to right below the breaking point of the DAW, it's entirely possible that DAW X significantly outperforms DAW Y. This is and has been my experience with DAW stress tests VS just loading a heavy test project in DAWs, and this is why I stand by my belief that the only test to measure performance in a DAW that matters is a test to right below failure.

Again, in to failure tests I did recently with Diva Logic came out third, DP second and Reaper first, the differences were relatively minimal, less than a 20% increase in performance low too high.

[Dewdman42]

You are speculating as much as anyone machinesworking and your speculation is not even based on simple reasoning. Please provide data. It would not surprise me at all if Reaper was the most CPU efficient on PC by the way, but it appears on mac it is not.

[Freaqpeak]

Interesting. On Windows, When I tested Studio One, it draw a lot more CPU than Cubase, which was something that made me stay on Cubase back then. But that was a few years ago, it might have changed.

Did you check at playback time with all plugins processing audio at the same time (VST2/3?)?

What ASIO settings did you use in Cubase? ASIO Guard on/off? Soundcard/drivers?

I'm sure these tests would show something different in Windows (Win 10 1903). Except that Logic can't be tested.

[Dewdman42]

Comparing one system to another was not the test that *I* did. I was attempting to keep everything as equal and the same as possible and only change one thing...which DAW was being used. On Mac. A similar PC test could come out completely differently. Actually originally my test was meant to compare DAW+VEP compared to DAW-alone. I ended up repeating the test for every DAW I own and it turned into an interesting comparison for DAW vs DAW in addition to DAW+VEP vs DAW-alone.

In my case, I used a real orchestral mockup because I wanted to also see how much CPU % I could expect from each daw addressing a typical real-world task. Logic gave the most CPU efficiency and breathing room in that light. Cubase was the worst until they fixed it, then DP was the worst but only by a little.

There are many other situations that could and should be tested...such as what happens during live tracking when a single core is being used for one track? What happens if you are playing down a large mix and trying to track at the same time? Etc.. What happens when you lower the buffer size lower and lower for each one, in the same situations as above? What happens if you load up the DAW with as many tracks and plugins as you can trying to max it out? Etc. I encourage others to continue creating more tests to add to the data points we have about them.

what I feel is that there is negative reaction to this data because somehow their beloved DAW doesn't seem to be in favorable light. The LogicPro users want to shout from the rooftops about this data, and reaper users want to discredit the data entirely. That is all silliness. Its just data. Its worth noting that LogicPro was the most efficient on the CPU with a 100 track orch mockup and it was definitely worth noting that Cubase 10.0.20 could not even complete the task until Steinberg fixed it in the next update. This is useful and important data, just as further tests mentioned above would also be good and useful data. I suspect most people don't care that much because for the most part, all of the DAW's will play just about any project thrown at them, regardless if they are using more or less CPU in order to do it. it might make a difference if you happen to try to put Diva into high quality mode or something, or might force you to use a larger buffer in some cases if your favorite DAW is not quite that efficient as it could be, but in the end, I think they can all get the work done and mostly people should choose the DAW that has the workflow they like. The DAW-makers should consider this data if they want to improve their products.

[chk071]

Apple software runs best on Mac OS? Surprise surprise.

[machinesworking]

You are speculating as much as anyone machinesworking and your speculation is not even based on simple reasoning. Please provide data. It would not surprise me at all if Reaper was the most CPU efficient on PC by the way, but it appears on mac it is not.

It's simple logic to say that in real life what matters is not how much CPU a DAW initially grabs, but how that DAW reacts to higher CPU thresholds.

It's also an observational fact that various DAWs will consume CPU at different rates when initially having plug ins instantiated in them. Effectively grabbing CPU beyond what is observed with dozens of that same plug in added in.

I'm playing hooky from real work today, so I'll take you up on it. Suffice to say though this has been a topic of discussion forever, and it's important to have tests that are reproducible. I've found this much harder to achieve than in the past for the simple reason that free third party plug ins these days run with ridiculously low CPU overhead. Crystal synth for instance is almost impossible to get to overload levels these days.,

The real debate though is what constitutes a real world test? I would say running at least four audio files looped along with the plug in tracks looped on a simple MIDI file is as accurate and reproducible as it gets.

BTW, DP is my favorite, Reaper beats it. If you're attempting to frame my observations as emotionally based, you're not accurate.

[Psuper]

Machines got it right, and he's elaborated on what I initially stated. I just know its not worth my time -- people believe what they want no matter what expertise and experience is involved, or lack thereof. More importantly, posts like those tend to be personal reasons in the guise of helpful.

[replicant X]

deleted

[machinesworking]

Machines got it right, and he's elaborated on what I initially stated. I just know its not worth my time -- people believe what they want no matter what expertise and experience is involved, or lack thereof. More importantly, posts like those tend to be personal reasons in the guise of helpful.

I don't think it's that disingenuous. It's also entirely possible that in dewdmans specific needs instance Logic performs better. He's using VEP, essentially forcing the DAW to not be able to use any prerendering strategies it has. Plus the VEP AU version is so specifically tailored to Logic that Vienna Instruments actually made a MAS plug in (DP's internal plug in protocol), specifically to address this.

Just performed the test again. Take Diva, with the factory "Initialize" preset, which is great for this, the reverb built in stresses my old Mac Pro etc. DP and Logic at 256 buffer, 120bpm, with the record unselected, running a simple mono note run can only do about 10 tracks without crippling levels of audio break ups and crackling. Reaper can run 14.

In past tests probably due to updates etc, DP and Logic would do better and Reaper slightly worse, like 11-12 for Logic and DP, and 13 for Reaper. Right now, with all three DAWs up to date, Reaper skunks the other two.

I can run the same test with the record enabled on the tracks, that will take out pre rendering and buffering, but in everyday use, with only one or two tracks record enabled for MIDI input etc. Reaper wins.

[Dewdman42]

You're assuming that I personally favor Logic and I don't neccessarily. I just reported the results for better worse. My test was completely unbiased, my intention was to understand more about what is better with or without VEP using all of the DAW's I own. And in the case of VEP, it was not always AU. For DP it was the MAS version of the plugin. For Cubase and StudioOne it was the VST3 version of the plugin. I have absolutely no axe to grind for or against any particular DAW.

As I have said repeatedly now, I welcome test data from either of you, or anyone else.