What's a Hartmann Neuron "model"?

[-Fluky-]

So I read about the Hartmann Neuron synthesizer and its "models", which sounded like they had something to do with FFT (or "resynthesis"), but then there's some "classification algorithm" that "classifies" the input sound into some model (there are many models).

From: http://neuron.prosoniq.com/resynthesis.html:

The Neuron Audio System uses a type of resynthesis where the synthesis itself is adapted to the sound. The basis function(s) used to resynthesize this sound are determined by evaluating its underlying model. It is this concept that allows not only a faithful reproduction but also a vast variety of modifications of the basic instrument.

To synthesize the sound, the underlying model of the sound generator has to be estimated from and adapted to the sonic qualities of the original sampled sound. The Neuron System analyzes both the type of vibrating medium as well as the resonant corpus used to shape the sound. This process is computationally very demanding, which is why it is done at the analysis stage. A pattern recognition process based on Artificial Neural Networks is used to create a model from the sound. If the sound cannot be uniquely represented by a single model (which may be the case for drum loops and complete musical pieces) it is constructed from a time variable morph between different models representing the sound at that instant.

After the pattern recognition stage, the basic model is subsequently refined and its parameters are adjusted in a way to reproduce the original sound as accurately as possible. The accuracy is determined by the „Complexity parameter" inside the ModelMaker application. Complex models are built from a set of simpler models, up to the limit set by the complexity constraint. Basic properties such as the size of the resonant body, material and esasticity (determined and accessed through the so-called Parameter set) can be influenced and combined from other models during the resynthesis.

But what's a model in the Neuron? Any guesses?

A naive guess would be that it uses a database of some wavelets or just some waveforms and then performs autocorrelation or similar to windowed chunks of the input audio and matches the individual windows taken from the input sound waveforms in the database, e.g one waveform per window. Then it uses this waveform in the resynthesis of that window. And this may lead to improved sound quality.

But what's this

The Neuron System analyzes both the type of vibrating medium as well as the resonant corpus used to shape the sound.

[Music Engineer]

The Neuron System analyzes both the type of vibrating medium as well as the resonant corpus used to shape the sound.

this hints to some kind of source-filter modeling. standard dsp techniques for this are based on linear prediction and cepstral analysis, or - more generally - spectral envelope estimation.

generally, the only thing that i have read about neural networks for sound (re)synthesis was the dissertation of axel röbel. there, it roughly goes like this: you use the universal-function-approximator property of neural networks to create a nonlinear predictor for one output sample as function of some set of previous output samples (and - possibly - an additional time-input variable to get a time-variant model). it's actually about modelling a nonlinear recursive dynamic system by using a flexible enough class of nonlinear models, which - in this case - is a neural network and then training your model on the data, which - in this case - is an audio file. the input vector to the function-approximator is always a set of past sample values (and possibly, a time variable) and the desired target-output is the next sample. then, you train a neural network using the audio file(s). in the dissertation, the same technique was also successfully applied to model well known nonlinear dynamic systems such as the lorentz attractor. here's a paper that summarizes some findings of it:

http://papers.nips.cc/paper/1276-neural ... ignals.pdf

but since the hartmann text mentions pattern recognition and classification, i'm not so sure, if they are really doing anything remotely similar there. classification/pattern-recognition is actually another application domain of neural networks that's (as far as i know) a bit distinct from function approximation and prediction. ...although, you can somehow also interpret the approximated function output of the network also as approximated probabilities of class membership....or something (it's long ago, that i learned about this stuff).