Tracktion and CPU Optimization, part 2

[HansP]

It takes a team of high-level specialists. Let's wait a couple of years. The market will reward it.
There must be an optimizer like in Oracle SQL execution planning. It will rearrange the graph iteratively until it has the best efficiency.
(For a complex query, OLAP systems would have intermediate results on many levels as well. While database optimizers are cost-based, DAW will be latency-complexity-based. The highest parallel use of threads with the fastest reaction on the play button, for the longest plugin chain and deepest net i.e. synced interdependencies.)
It will be hint-based too: either we do real-time, or do editing (tweak EQ etc) or we want to render the whole project. Different strategy every time.

[jochicago]

The only way that could work well in real-time processing is if you had some sort of quantum computer that could look into the future to see events that haven't happened yet, then return those events back to the present for processing, so that you get your processed output 'now'.

While I do love the idea of a DAW that sees into the future of time, for this conversation I was thinking something much simpler.

I would use a simple predictive system that recalculates every time you perform a playback operation. So as you add plugins, it keeps a simple historical memory of how much processing each plugin & track takes. It also remembers your use of playback samples and overall CPU every time you press play. It uses this info to do a predictive calculation to configure an appropriate pre-render for your current project. As it sees you get closer to the limits of the machine, it makes a smart guess about the required delay and adds it automatically, and with every iteration it optimizes to keep the processing ahead of the limits but with the least required amount of time delay for the user.

Also, this doesn't have to happen during the user experience. The human user could be in the midst of choosing a plugin or sample from a lists, sending no audio to the output. The system takes advantage of this idle time to perform a background play to correct itself (much like the rendering engine in the export dialog starts exporting the file before you are done tweaking the settings).

So the DAW doesn't have to wait for you to press play to start learning the project's resource usage. Every now and then the DAW engages a short silent play in the background to keep an eye on tracks, plugins, CPU and samples, and keep itself ahead with optimal resource usage predictions.

To the user this would be pretty transparent, except that the DAW would feel several times more powerful and make much better use of the new 16+ multi-thread processors that are starting to become the norm.

[zzz00m]

The only way that could work well in real-time processing is if you had some sort of quantum computer that could look into the future to see events that haven't happened yet, then return those events back to the present for processing, so that you get your processed output 'now'.

While I do love the idea of a DAW that sees into the future of time, for this conversation I was thinking something much simpler.

I would use a simple predictive system that recalculates every time you perform a playback operation. So as you add plugins, it keeps a simple historical memory of how much processing each plugin & track takes. It also remembers your use of playback samples and overall CPU every time you press play. It uses this info to do a predictive calculation to configure an appropriate pre-render for your current project. As it sees you get closer to the limits of the machine, it makes a smart guess about the required delay and adds it automatically, and with every iteration it optimizes to keep the processing ahead of the limits but with the least required amount of time delay for the user.

Also, this doesn't have to happen during the user experience. The human user could be in the midst of choosing a plugin or sample from a lists, sending no audio to the output. The system takes advantage of this idle time to perform a background play to correct itself (much like the rendering engine in the export dialog starts exporting the file before you are done tweaking the settings).

So the DAW doesn't have to wait for you to press play to start learning the project's resource usage. Every now and then the DAW engages a short silent play in the background to keep an eye on tracks, plugins, CPU and samples, and keep itself ahead with optimal resource usage predictions.

To the user this would be pretty transparent, except that the DAW would feel several times more powerful and make much better use of the new 16+ multi-thread processors that are starting to become the norm.

Yes, that should work for every operation of the computer, EXCEPT for the real-time audio stream going to the buffers of the audio interface. That will always be a serial operation and nothing can change the laws of physics in that regard. That happens in 'real-time', and there is no way to predict what will happen with audio when it hasn't happened yet.

[HansP]

the fact that the serial operation is chopped up into buffers, and takes place in a segmented and repetitive way, makes it applicable for parallel operation, as soon as the first chop has completed its travel from start to end. then, many will follow the same path. but everything is happening at the same time, geometrically speaking. there is a buffer swap and a geometry of a bucket brigade.

[zzz00m]

[chico.co.uk]

the fact that the serial operation is chopped up into buffers, and takes place in a segmented and repetitive way, makes it applicable for parallel operation, as soon as the first chop has completed its travel from start to end. then, many will follow the same path. but everything is happening at the same time, geometrically speaking. there is a buffer swap and a geometry of a bucket brigade.

.... eh?

[zzz00m]

the fact that the serial operation is chopped up into buffers, and takes place in a segmented and repetitive way, makes it applicable for parallel operation, as soon as the first chop has completed its travel from start to end. then, many will follow the same path. but everything is happening at the same time, geometrically speaking. there is a buffer swap and a geometry of a bucket brigade.

.... eh?

Right?

[chico.co.uk]

That video you posted is really good, by the way. Well worth watching, thanks for that

[zzz00m]

That video you posted is really good, by the way. Well worth watching, thanks for that

+1

That is the best I have ever seen on the subject. Opened my eyes, for sure!

[dRowAudio]

There are two main problems with pre-rendering audio in DAWs.

The first is that because FX have cumulative effects (think a delay or reverb), simply changing anything in a track with these on means you have to ditch all of the existing pre-render for the track and re-render the whole thing. Couple this with the fact that you can have signal flows that go to different outputs, between tracks, through Racks (which can have shared input sources), auxes etc. you can very quickly end up having to ditch the entire pre-rendered audio from any change.

The second big problem is that the most resource hungry things in an Edit are plugins. A plugin can't be used for background rendering and live play at the same time so in order to render in the background, you'd need to create a new instance of every plugin which will consume more resources and CPU cycles.

You can also have live inputs to worry about.


With all this in mind, automatic pre-rending probably isn't going to be the silver bullet you might think it is.
It would be much better to spend our time an energy on optimising the audio graph's threading model and making it quick and easy to freeze tracks that you know you won't be working on for a while (we have thought about adding hints for this, a Mr Paperclip: "It looks like you haven't changed this track in a while, would you like to freeze to save resources?" anyone? We all know how well regarded Paperclip is

[terrynoakes]

It would be much better to spend our time an energy on optimising the audio graph's threading model and making it quick and easy to freeze tracks that you know you won't be working on for a while (we have thought about adding hints for this, a Mr Paperclip: "It looks like you haven't changed this track in a while, would you like to freeze to save resources?" anyone? We all know how well regarded Paperclip is [/quote]

Sounds like a good idea, so long as you could have the option to turn mr papeclip on/off

[zzz00m]

It would be much better to spend our time an energy on optimising the audio graph's threading model and making it quick and easy to freeze tracks that you know you won't be working on for a while (we have thought about adding hints for this, a Mr Paperclip: "It looks like you haven't changed this track in a while, would you like to freeze to save resources?" anyone? We all know how well regarded Paperclip is

Sounds like a good idea, so long as you could have the option to turn mr papeclip on/off

Cakewalk Sonar implemented the 'Freeze Tracks and Synths' feature, which can be turned on/off with the click of one 'Freeze' button. This would be a useful model to follow for this feature.

Freeze tracks and synths
The Freeze feature allows you to temporarily bounce your track, including soft synths and effects, to reduce the amount of CPU power needed. The Freeze feature also works for synths patched in the Synth Rack.

Freeze Track. Bounces the audio in the track to a new audio clip or clips, applies any effects, and disables the effects bin.

Freeze Synth. Audio from a soft synth is bounced and placed on the synth’s track. Output from the synth is disabled, as is the effects bin on the synth track.

https://www.cakewalk.com/Documentation? ... ng.23.html

[terrynoakes]

It would be much better to spend our time an energy on optimising the audio graph's threading model and making it quick and easy to freeze tracks that you know you won't be working on for a while (we have thought about adding hints for this, a Mr Paperclip: "It looks like you haven't changed this track in a while, would you like to freeze to save resources?" anyone? We all know how well regarded Paperclip is

Sounds like a good idea, so long as you could have the option to turn mr papeclip on/off

Cakewalk Sonar implemented the 'Freeze Tracks and Synths' feature, which can be turned on/off with the click of one 'Freeze' button. This would be a useful model to follow for this feature.

Freeze tracks and synths
The Freeze feature allows you to temporarily bounce your track, including soft synths and effects, to reduce the amount of CPU power needed. The Freeze feature also works for synths patched in the Synth Rack.

Freeze Track. Bounces the audio in the track to a new audio clip or clips, applies any effects, and disables the effects bin.

Freeze Synth. Audio from a soft synth is bounced and placed on the synth’s track. Output from the synth is disabled, as is the effects bin on the synth track.

https://www.cakewalk.com/Documentation? ... ng.23.html

This already available in waveform, I think the idea is for the software to give you a nudge, if it thinks performance would benefit freezing tracks.

[dRowAudio]

But we already have this feature in the form of Freeze Points? You can freeze up to any point in the signal chain?
You can even combine several tracks in to a single freeze file with the other freeze option?

[zzz00m]

But we already have this feature in the form of Freeze Points? You can freeze up to any point in the signal chain?
You can even combine several tracks in to a single freeze file with the other freeze option?

Maybe you need to start here?

From the current Waveform User Guide, Version 9:

This manual is significantly improved and expanded. We are working toward complete coverage of
Waveform’s rich feature set but there are still few gaps. These are some of the key things that aren’t
fully covered:
• Control Surface Mapping
• Waveform Sampler
• Video integration
• Melodyne Integration
• Project Templates
• Waveform Archive format
• Freeze Points
• CPU Usage Meter