Audio mixdown as "freeze"...

[pljones]

If you freeze a module, you're freezing the one module. If there are upstream modules that only feed to that module, they could be disabled. Equally, any module upstream of a module that produces no output (that only feeds to that module) could be disabled -- regardless of whether it's related to freeze. So that's a separate issue related to processing optimisation, in my view.

So stick with the simple point: freezing a module disables the module (causing it to produce no output) and inserts playback of the captured audio immediately after it in the signal chain, replacing the outputs. (Mmm, so a 32 output VSTi being frozen generates a 32 output freeze... And freezing Reason in full swing could be fun... OK, so you'd only need to handle the connected outputs...)

[TheGuysanIdiot]

Hi All,

Due to the way MULAB can use tracks/parts/targets and all the modular routing possibilities I also think that it is probably best to look at freezing modules first.

Most of the time I would not use freezing as I would want to control the module(s) in real-time.

Freezing will obviously render selected module(s) output to an audio file and then process off those module(s).

The whole point of freezing is to reduce the load of the CPU on specific CPU hungry module(s).

How Jo would attempt this is beyond me.

All I can suggest is that Jo works on a few test versions and let the users see what is practical.

This is going to be a hard nut to crack but worth it in the long run.

I am sure many users out there as still on single CPU machines and some kind of freezing method to free up CPU would be very welcome indeed.

As for multicore machine users they too would also benefit from freezing and then in time multicore support.

I think both freezing and multicore solutions will be tricky to implement into MULAB but will be well worth the effort.

OZ

[MuTools]

Nice thoughts OZ.

You're right to put Freezing and MultiCore support into balance to eachother.

Not sure which of both is the very most important one.

(and indeed, it would be best of all if we would not need freezing!)

The more machines with a multicore processor out there, the more the balance tends to MC support, i think.

Anyway, lets not forget a third CPU-reducer: Optimize the code of the modular system and a bunch of DSP algorithms. In fact that's something i want to/should R&D first.

[TheGuysanIdiot]

Indeed a bit of a chicken and egg situation.

From a pure sales and marketing point of view imagine being able to advertise MULAB as a multicore music app.

If a user buys a new machine today it will most likely be a multicore one.

If you buy a second user machine off an auction site it too will probably be multicore also.

Single CPU machines are very much legacy machines and will need replacing eventually as the parts wear out.

So the future I guess is really multicore.

So it may be worth optimising existing code of the modular system as you say. This is something that you are very experienced in and should be able to do relatively easily. It will free up CPU.

I for one do not freeze anything out of choice and if I want to mixdown in MULAB and add to the existing session the facility is there already.

So IMHO multicore seems the most important feature to work on and add to MULAB. It will probably be a programming nightmare for you to begin with but once you get going I am sure you will be able to turn MULAB into a multithreading monster.

Anyway just food for thought I am sure others will have alternative view points on this subject.

OZ

[Trancit]

Nice thoughts OZ.

You're right to put Freezing and MultiCore support into balance to eachother.

Not sure which of both is the very most important one.

(and indeed, it would be best of all if we would not need freezing!)

The more machines with a multicore processor out there, the more the balance tends to MC support, i think.

Anyway, lets not forget a third CPU-reducer: Optimize the code of the modular system and a bunch of DSP algorithms.In fact that's something i want to R&D first.

From my point of view, Multi-Core support is of course No. 1...

The more I think about freeze, the more I tend to a complete different approach:

Because of the complex routings, it will be very hard to implement this feature, that all users with their different workflows can benefit from...
Not to mention, that it would take ages only to consider all the possibilities...

Why not making it a way easier:

My last idea is using the audio file recorder...
After reading the other postings, for me it's getting clearer, that an automatic freeze is too complicate.
But a manual freeze approach (this would be quite similar to FL Studio) would be quite easy to implement.

To user has to set up the audio file recorder(s) manually (with the very welldone "Record from" it is even semi automatic)...or a little button on every module...clicking the button, autoroute a audio file recorder to the modules output...

After routing, there could be a menu entry (rightclick menu, or edit menu...something, which is reachable from every window): "Freeze"

All "selected" Audio File Recorder(s) (perhaps, there should be a button (disabled by default), which set this audio file recorder up for freezing) render now their inputs offline (important)

Now there could be a new window popping up, which shows all the modules involved in this "freezing process", offering options to

1. either leave processing on (for modules, which have different inputs)
2. or toogling processing off (for "normal" synths)
3. or unload plugin to free up RAM (for big samples or RAM hungry plugins like Spectrasonics stuff)

The only point, which I am not sure about, is, where to route the new rendered audiofiles...?

Definetely not the way it is now...it has to be routed before the Master and I think it would be the best to leave the downstream (I don't know, is this correct...I mean the routing after the "frozen" module) intact...perhaps the audio file recorder could get a output connector and act as an "insert effect"...so the following routing could be left as it was before...


So, what do you think about this???

The advantages for me would be:

- quite easy to implement (from a non programmers view ), because it uses in part already available things...
- very flexible, the user decide, where to pick up the signal, depending own his own routings
- significant easier than it is now
- it fits the whole complexity
- not complete automatic, but it brings up all the benfits from "Freeze"

For me, only disadvantage:

- it is not complete automatic...but after thinking about this problem a few days...would a complete automatic freeze in MU.LAB, with all it's possibilities, really realisable????


Trancit

[pljones]

Nice explanation, Trancit. An "add freeze/record" menu entry on every module, perhaps?

There are still complications. If you have a 32-out VSTi with four outs connected, you really only want to auto-record/freeze those four outputs. So "Add freeze/record" might set up four inputs, four recordings and four outputs onward to the existing connections. Hitting freeze runs the VSTi's outputs into the recordings and frees up the VSTi processing, with subsequent playback from the recordings via its already set up outputs. Great!

But if you then add another connection to the VSTi... should this automatically get tracked by the freeze/record system? Or would it invalidate what was set up? Or..? Hmmm..... You're suggesting the "easy way out" is to leave it up to the user to sort out... Hmm... I think the system could "know" that a module was enabled for freeze and follow changes to its outputs.

In the Modular Area, though, I wouldn't want to see all this extra complexity. I'd like freeze-enabled modules highlighted somehow - but all the extra complexity behind the scenes should ... be behind the scenes.

[Trancit]

But if you then add another connection to the VSTi... should this automatically get tracked by the freeze/record system? Or would it invalidate what was set up? Or..? Hmmm..... You're suggesting the "easy way out" is to leave it up to the user to sort out... Hmm... I think the system could "know" that a module was enabled for freeze and follow changes to its outputs.

In any other host you cannot make changes to a frozen plugin (which processing is toogled offline) and so should it be here...
If the processing is not toogled offline, perhaps it is possible in the plugin wrapper of MU.LAB to turn off "frozen" outputs of a multi out plugin...e.g. you have frozen outputs 1-4 and don't "hear" them anymore but you can still use 5-32...

In the Modular Area, though, I wouldn't want to see all this extra complexity. I'd like freeze-enabled modules highlighted somehow - but all the extra complexity behind the scenes should ... be behind the scenes.

How many modules do you like to freeze in a project???
I think, if in one project 3-4 modules are frozen would be the "normal" case...
Beside this, a very handy feature in addition to freeze and for modules, producing sound only from time to time, is the "Smart disabling" feature of FL Studio, which toogles processing off after 5 secs inactivity and toogles processing on on the fly, if a signal is received...often freeze is not necessary for those modules and this happens behind the scenes...if a plugin makes trouble with this feature, you can turn smart disabling off...

But perhaps, it would be possible to make frozen plugins, which processing is toogled off invisble as long as you select the "Freeze audio file recoder" and choose unfreeze...
But I think, this is more the fine tuning...

First we should find the right idea...


Trancit

[cytone]

This is much more complicated than I thought it would be. Here's another idea though. A kind of transparent freeze that happens at the individual sequence level.

So how this would work is you select one or more sequences then right mouse button -> freeze



The freezing process would need to take into account the tail for the sequence as shown.

You then have the flexibility to copy and move around the sequences in the composer. You don't see the waveforms but just the sequences with an indication that they are frozen.
If you edit a sequence and change the notes then it can just re-freeze that sequence.



I haven't really though this through a lot so it might not work

[robenestobenz]

This is much more complicated than I thought it would be. Here's another idea though. A kind of transparent freeze that happens at the individual sequence level.

So how this would work is you select one or more sequences then right mouse button -> freeze



The freezing process would need to take into account the tail for the sequence as shown.

You then have the flexibility to copy and move around the sequences in the composer. You don't see the waveforms but just the sequences with an indication that they are frozen.
If you edit a sequence and change the notes then it can just re-freeze that sequence.



I haven't really though this through a lot so it might not work

this doesn't address the complications in determining what modules to disable downstream, but it would be very handy to have in terms of saving vsti processing, which is in itself quite a big saving in some cases. E.g. NI Battery 3 is often about 10% of my CPU in many projects.

[cytone]

this doesn't address the complications in determining what modules to disable downstream, but it would be very handy to have in terms of saving vsti processing, which is in itself quite a big saving in some cases. E.g. NI Battery 3 is often about 10% of my CPU in many projects.

One approach here may be to use smart bypass for the VST's. This works well in Renoise where it is called "Auto Suspend". The plugin is shut down when it is no longer producing sound. A few plugins might not like this, such an an effect with an internal sequencer running. In Renoise this is handled per plugin. If you disable Auto Suspend for one of the FX VST it is stored in the config so whenever you load it again it remembers this setting.

The top of the plugin window could change to this to show it is in standby



An advantage of not totally disabling the plugin is that it is always available for use again. So you can still play it from the keyboard and if you edit a frozen part it is immediately reverted back to a normal part.

[robenestobenz]

One approach here may be to use smart bypass for the VST's. This works well in Renoise where it is called "Auto Suspend". The plugin is shut down when it is no longer producing sound.

Other than plugs which have some other functionality than responding to notes (like your sequencer example), I'd hope most VSTis are well coded enough that the CPU usage when idle should be very insignificant. That's been my experience so far, anyway. Did you find otherwise in Renoise?

But per track/part freeze for VSTi sequences would be awesome. Very simple to use, it's absolutely clear to the user WHAT they are freezing, and it would require less work from Mulab to determine what should be disabled.

[cytone]

Yes, most vst's use hardly any CPU when idle. One that does though is Firebird. If I load 6 instances with nothing playing my CPU is around 50%. With auto suspend that would be reduced to almost nothing.

[TheGuysanIdiot]

Hi All,

This is just a few more general comments from me which relates to CPU saving.

I have used some third party Synthedit modules that the developers state use some kind of sleep/timer device that monitors whether a specific sub control within the module is being used if not it puts into a sleep mode. I have read similar stuff in the manuals of commercial synths. I was wondering if MULAB's own synths/effects use sleep mode kind of stuff. (I do not program this is just what I think I have read in the past).

I have been testing out the Audio Recorder MULAB module and using the Session Modular Area and putting them in different parts of the audio chain. I know this is very manual and not automatic but once the output of the VSTi etc has been rendered to wav I then toggle off the VSTi processes. There are many routing possibilities, you can use an Audio Recorder per VSTi or route many VSTi etc to one Audio Recorder etc etc etc.

Until there is an automatic freeze method in MULAB assuming there will be one. I would recommend to anyone that they experiment with the Audio Recorder module.

As I said before multi-core is probably the way to go without freezing stuff. But in the meantime get to know the Audio Recorder module.

And yes maybe the Audio Recorder technology could be used in a simple automatic freezing system.

Anyway just wanted to say what a great tool the Audio Recorder module is when used in the Session Modular Area.

OZ

[liquidsound]

Maybe this is too superficial thinking but if we wait for Code Opt. and Multicore by that time the need for freezing would be narrowed down to less extreme choices and "Part" freezing could be just enough then.... you know what I am trying to say here...

[MuTools]

I also think we should be careful with investing time in a freezing mecanism. Code optimalizations and multi-core support are more important, imho.

But besides these considerations, i'm thinking of adding some extra support for 'simple manual freezing' i.e. add a "Render Into Audio Part" context function for modules.

This function works like 'Mixdown' but then only the output of that specific module (further called "module X") is rendered to an audiofile, and put into an audio part. Then that audio part is routed to where the audio out of module X is connected to. If there are multiple connections, MU.LAB will automatically insert a 'Patch Point' between module X and its target audio inputs, and then the rendered audio part is routed to that patch point.

To streamline the freezing process, module X will (optionally) automatically be turned off, and it would also be nice that when module X is rendered another time (because some things have changed) then the new audio is rendered into that same audio part (and audio file) so the new rendered result is simply replacing the previous version.

These are my thoughts so far. I could not yet research this deeply from the technical side, and maybe i'm still missing something from the user side, so to be continued. Please let me know your toughts. Cheers!