Proposed Benchmark testing for T2

[dealwithit]

Yeah, I guess you're right.
And, I think I meant "objective", not "subjective"....
That'll teach me to post with a cat on my desk (get down cat!).

But, in all seriousness, I'm interested in getting all logical and methodical on this, if only to prove to my own mind whether or not T can compete with Cubase in a benchmark like this (my gut feeling is that Cubase out-tracks T, but hey...).

[mandolarian]

No, I think you're onto something. Simple, consistent, repeatable tests is what made Elemetary school great. I'm all for it.

But to eliminate the remaining variables, we all would need to send you our computers for you to test in the cat cage.

[polaris20]

Yeah, I guess you're right.
And, I think I meant "objective", not "subjective"....
That'll teach me to post with a cat on my desk (get down cat!).

But, in all seriousness, I'm interested in getting all logical and methodical on this, if only to prove to my own mind whether or not T can compete with Cubase in a benchmark like this (my gut feeling is that Cubase out-tracks T, but hey...).

Sounds good to me; See if you can find some plug-ins that are also available in DX, as I'd like to compare it to CEP2.1.

Also, who wants to do the archive? Shall I do it? I guess I could just do some acoustic noodling for the benchmark, then archive it, and post it to my website.

[polaris20]

I'm home now; interestingly enough I get 12 tracks with the "announcement" clip, with my 1gig of RAM, Athlon 2500+, SATA drive. In other words, not much worse, if at all, than the Athlon 3000+ 64 at work. Hmmmm.

Also, keep in mind that when you add reverb, especially this many tracks, that's a stupidly high amount of processing, compare to other plugs.

For example (maybe not entirely apples to apples, but hey) the new Waves hardware device boasts the ability to run 6 instances of IR convolution reverbs. Now, granted those are fancier than the stock T2 reverbs, but still. To be running 12 tracks with reverb, delay, EQ, chorus, and a compressor, without going red, is good, IMO.

Also, my test I just did was running a total of 20 instances of the "announcement" clip, with 8 of them uneffected.

Upon further reflection, I'm actually quite pleased with this performance, since I haven't even begun playing with freezing, and I am not one to use this ridiculous amount of plug-ins per track.

If dealwithit is willing to hunt down some freeware universal standard plugs, I'd love to see how this translates across different platforms, DAWs, and architectures.

I am also really wanting to hear from the Mac folks, as I was considering reformatting my 2500+ desktop and disconnecting from the Internet entirely, to be used strictly for recording, and then using an iBook for scratch/remote/sequence recording and internet/office/graphics stuff.

This is getting really interesting, and without the participation of many people (and their opinions and input) it won't be a success.

I was prematurely down on the T2 performance, without really thinking about all the shit going on inside the box. THat was stupid of me, so I apologize.

I'd really like for this to become a sticky, for the longterm benefit of this forum. Mods?

EDIT!!!

Here's some plug-ins available for free, in VST and DX format.

http://www.digilogue.de/

I've used these before, and they're decent. Let me know what you think. This will allow the test to be done across multiple DAWs. The only downside is that it seems to be PC only.

[will.record.for.food]

But VST and DX also process differently. I remember in my good old Ntrack days, DX would take up more than twice the cpu of a VST plug (the blue line in particluar)

[bear]

Yeah, I don't think VST and DX would be the same. But with 3rd party vsts, comparisons could be made with sonar and cubase and even protools.

I have to say my 12 track limit doesn't bother me so much since it seems to be the about the same on other machines.

I totally agree with Polaris20 that 6*12 vsts where particularly reverb is very computationally intense is really not that bad....

Any MAC users out there up for the test?

And where is zeoy with his monstor overclocked machine....

[dealwithit]

I have to agree that any standardized test would have to be limited to VST plugins. No valuable comparison could be made between a test with VST's vs. a test with DX plugins (other than the obvious "which is more efficient" argument).
As for a good set of free standard VST plugins, look no further than Kjaerhus' Classic Collection (http://www.kjaerhusaudio.com/classic-series.php). This provides a good range of different plugs (compressor, reverb, EQ) that should give a good representation of what might be used in real-world music creation.

The only remaining item would be the audio content itself, which I think should be varied enough to provide a good cross-section of actual instruments (maybe a guitar loop, a bass loop, a drum loop and a piano loop). In other words, no silence, since the processing power required to add reverb (or other effects) to silence as opposed to an audio sample is questionable. This would allow the test to represent actual conditions.

The idea is that tracks would be added in a given order (drums/bass/guitar/piano) until Tracktion's CPU meter hits the red.

Just ideas, not sure if this test works in reality, but I'll be giving it a try. I have to check permissions on some of these loops to see if I can distribute an archive, and the plugin settings (most likely presets) would have to be standardized, but it seems possible.

Or am I making this too complicated?

[AngryFrog]

Hi,

Just your local lurker. Got 18 tracks to run. Had a CPU alert on the 19th.

Dell Inspiron 8600 laptop
1.7 gig Centrino
7200 RPM internal hard drive
1.5 gig of RAM
Windows XP SP1, fairly tweaked
Presonus Firebox set to 8.0ms latency

[djsubject]

11 Tracks before hitting red 12'th one hit red

amd sempreno 2400 1.67 gig 1gig ddr 2100 ram radion 9050 grafix card

subz

[djsubject]

i think the native vst's being used for testing should be the kjaerhus clasic (free) bundle

because most of us have them & any that dont realy should have them

Subz

[bk]

How about just using a sine wave test tone for the audio? I think processing a continuous sine wave would give more consistent results.

This is an interesting idea, but should definitely be standardized as much as possible to be meaningful.
I'll try to run my two systems tomorrow night and post the results, erm...that is, if we decide on a standardized version.

[ModuLR]

I don't think anyone is going to find anything highly unusual between hosts. This has been done in the past...(cough.. the Ambience test - tho it was more focused on exposing it's P4 issue). Ya know, a standard file.. a buncha people seeing how far they could go with it. At best, it's typically a +/- 1 track deal depending on the host with the exception being Logic Audio (even v5 for the PC), which can handle an unsually high track count before cutting out.

I dunno, at least that has been my experience... I'm sure if you search you can find such results around here, unless they've been tucked away in some archive. If anything, it should be proposed over in the host forum if the goal is to get *everyone* in on it.

[polaris20]

http://s62.yousendit.com/d.aspx?id=2CSB ... VL1ZI4JMQN

Here's an archive. I got 16 tracks with it, using the stock plugins.

THe plugins that were posted before locked up my system tighter than a drum.

[rockstar_not]

I don't want to bother checking my machine's performance in any other host. I would like a T/RMS only standard test of some kind.

Purpose: check the performance of my machine against comparable machines to see if I have it tweaked correctly.

Got my new DAW almost completely assembled and XP installed last night (the stock AMD socket 939 CPU fan is gonna get the heave-ho soon) and I want to check my machine's performance against others with similar setup.

I'm going to follow the XP tweak guidelines in the TASCAM guide and the XP Music Net http://www.musicxp.net/about_xp.php tips, except that I will leave the system restore settings in place until I get comfy with XP.

Why not do something utilizing the demo projects that come with T?

They are setup for efficiency most likely, but why couldn't one just use something like the train station noise from Gare du Nord, strip all filters except for perhaps one instance of T reverb, on that track, then just copy the track and filter over and over till the system poops?

Maybe I'm not thinking complex enough, I don't know.

-Scott

[polaris20]

The archive above is strictly with T plugs. The only difference is an audio recording I did. Otherwise, all T. I agree with you.

The other problem with using other plugs is that we can't compare them to Mac users, which is what I am really interested in.