Apple's neural engine cores and u-he

[MaF]

Since there are several neural engine cores in Apple Silicon chips, is there a meaningful way to take advantage of them for audio purposes?

[Urs]

Probably, but I don't know much about that.

[Tj Shredder]

Well, one could write code to teach a synth to get close to a sound you feed into an audio input by just moving all of its parameters... Make Diva sound like a Piano... We tried it for 50 years and a neural net will probably beat us... Takes only 5 years for training the net... ; - )

[medienhexer]

Well, one could write code to teach a synth to get close to a sound you feed into an audio input by just moving all of its parameters... Make Diva sound like a Piano... We tried it for 50 years and a neural net will probably beat us... Takes only 5 years for training the net... ; - )

I‘m pretty sure u-he would still have to do all the work. It’s not like you can take the Neural Engine out for a beer and tell it: „well, Netty, I was thinking. How about I send you audio and an overview of my synth‘s parameters. You try and copy it and I‘ll tell you when you’re close.“

Typically, you have to solve the problem of how to resynthesize first and then train the neural network offline first until the approximated solution is satisfactory and you can throw new problems at it. It may be a little easier with additive synthesis? I don’t know but can imagine that setting up the volume of partials may be an easier method.

My understanding is that Apple have included processing hardware which will accelerate the typical AI workloads better than general purpose CPU registers with much less energy drain. That hardware may or may not accelerate other audio operations. In any case, developers need either low level access to the hardware or a high level framework which offers audio processes. It still would not know how to solve a problem using Diva.

[Tj Shredder]

As all parameters of a synth are exposed to a DAW, it should be possible to train a network to create sounds for any VST. But this wouldn't be anything U-He would have to care of. It could be a training app which hosts the plugin, and then spits out presets... I doubt this would be very interesting beside scientific curiosity. The amount of work to create and train such an application is probably not worth it... The neural processors in ARM chips are needed for voice recognition and pictures...
It would be more interesting to have automatic accompanying music creation or an AI buddy for rehearsing and training of your human skills...
I wouldn't mind an AI Aebersold app to play along...

[moscom_electronics]

Some NI amp plug-ins seem to use trained neural networks to obtain the best model of a given amp, if I remember correctly. If that is true, it is possible to imagine that a neural network could be trained to obtain the most accurate response of a given piece of electronic gear so that part of model tuning is already done. I am way out of my expertise to be honest

[Ploki]

the way neural networks are used for plugins right now is pretty much "here's a box, figure out what it does" so a human doesn't have to do it.
I don't think neural cores would help with that

[Unaspected]

A neural network could lend itself to filtering and similar processes - any process that requires lots of multiplications - especially in parallel - polyphase filtering (up and down sampling). Probably the most interesting thing would be if the neural processing were fully analogue - I don't know whether this is the case with Apple's NPU.

[ghettosynth]

Oooh shiny. But seriously though, what fun it would be to have to make your Apple solution work cross platform, no? That sounds like a great use of time instead of getting Zebra finished so that Urs can get his ass in gear and get back to the Berlin Modular project.

[Ploki]

Or maybe the randomisation functionality in Hive

[Urs]

Or maybe the randomisation functionality in Hive

(don't)

[ghettosynth]

Or maybe the randomisation functionality in Hive

I know right? That's the one thing holding me back from picking it up!

[Ploki]

Or maybe the randomisation functionality in Hive

(don't)

I always find shit that’s not supposed to be there when i start skinning and digging in the skin editor