The Biggest Problem With EVERY Vocal Creation Program

[Examigan]

I thought Vortifex answered it for you.

The songs I made synth and drum tracks for on other’s albums used real vocalists.

[wagtunes]

I thought Vortifex answered it for you.

The songs I made synth and drum tracks for on other’s albums used real vocalists.

Sure, lots of people use real vocalists or sing themselves. But some people can't sing and can't afford to hire a singer every time they want to record a song. They rely on programs like these. So it would just be nice if they had a little more variety.

It's odd that this is such a contentious subject for people. With synths, they cater to just about every genre you can think of from ambient to hip hop to heavy metal and nobody has a problem with the variety you can get from synths. But mention trying to get variety out of a vocal plugin and everybody has a cow.

I don't get the utter disdain for vocal programs.

[DaveClark]

I don't care if it's Vocaloid, Synthesizer V, Omnivocal or whatever. They all suffer from one thing.

Generic pop voices.

You won't find any Rod Stewarts or Geddy Lees or anything with any real grit and character.

So here's my question. Why?

Is it

1. They don't have the technology?

2. The market doesn't want these voices?

Both?

I'm asking those of you who are most familiar with the tech and the industry. That isn't me. All I know is I go through all these voices and they all sound bland.

Why?

I'm not familar with the particular industry, but have experience with physical modelling over decades. I asked Google AI the following question, and it replied as follows:

Dave: I'm more interested in what the current limitations are for synthesis (not AI cloning) in order to achieve truly authentic artistic expression. One of my guesses would be that we don't really have a thorough understanding of how the mechanics of the thoat, the structure of the vocal chords, pressure from the lungs and diaphram, etc. are related in a predictable way to the various forms of artistic expression

Google AI: Your intuition is backed by significant research: the primary barrier to "authentic" synthesis is that current models often treat the voice as a set of disconnected components rather than a dynamic, coupled system. While we have the math for individual parts, we struggle to model the reflexive, fluid-structure interactions that a human singer uses to convey emotion.

Current technical limitations in achieving truly authentic expression through synthesis (distinct from AI cloning) include:

1. The "Source-Filter" Oversimplification
Most synthesizers use the Source-Filter Model, which treats the vocal folds (source) and the throat/mouth (filter) as independent systems.

In Humans: The vocal tract's shape actually pushes back on the vocal folds, changing how they vibrate (acoustic back-pressure).


2. Lack of Biomechanical Reflexivity
Authentic expression often comes from reflexive muscle movements that aren't consciously controlled by the singer but are triggered by emotion.

Coarticulation: In humans, the way you pronounce a "t" is influenced by the vowel coming after it. Simulating these smooth, context-dependent transitions—especially under the high lung pressure of singing—remains a major hurdle.
Complex Tissue Interaction: We are just beginning to model how "secondary" tissues like the false vocal folds (ventricular folds) interact with the true vocal folds to create grit, warmth, or specific textures.

3. Predictability vs. Artistic Spontaneity
As you noted, we don't yet have a predictable map for how lung pressure and diaphragm support translate into specific artistic "intent".

Control Parameters: Even the best physical models (like VocalTractLab) require hundreds of parameters (glottal aperture, lung pressure, tongue position) to be adjusted simultaneously.
Artistic "Ghost": Without a "lived experience" to draw from, synthesizers cannot decide when a breath should be shaky or when a note should transition from a "glottal attack" (hard start) to a "balanced onset" (smooth start) for emotional effect.

4. Computational Bottlenecks
Achieving high-fidelity physical modeling in real-time is incredibly taxing.

Mathematical Complexity: Solving the partial differential equations required to simulate airflow in a 3D moving vocal tract often leads to digital instability or requires massive processing power that isn't yet viable for standard consumer hardware.


Dave: This last point is a huge barrier with which I am very familar... I worked in support of computational physics programs of this type at a major semiconductor manufacturer.

[wagtunes]

Hey Dave, thanks for that. Fascinating reading.

[4_the masses]

If you are looking for something with character, use a human. Why does AI have to replace everything ?

[wagtunes]

If you are looking for something with character, use a human. Why does AI have to replace everything ?

1. I'm not talking about AI. I'm talking about plugins.

2. Why do synths have to replace orchestras, pianos, harmonicas, etc.? I could go on and on.

Why is it okay for somebody to go buy a violins sample library and sound exactly like the real thing but it's not okay for vocals?

The double standard that we have is astounding.

[Examigan]

You're not talking about AI? But don't most of those kind of plugins use AI?

[wagtunes]

You're not talking about AI? But don't most of those kind of plugins use AI?

Some do, some don't. And that's only recently. Vocaloid is still synthetic vocals though it does have an AI option. But if you want to sound like Korean K-Pop, you still can.

But that isn't the point of any of this.

[wagtunes]

When I think AI, I think of something like Audimee where you have to sing your vocal and then upload it to their site to get the AI version. With these other plugins, you draw each note and pick a singer to sing it. Yes, there are exceptions but in general, that's how it works.

[whyterabbyt]

Tried slinging them through one of thae voice changer applications?

https://store.sonarworks.com/products/s ... nsion-pack

I have SoundID but I'm not talking about AI vocals.

If you have Sound ID as you say, then you'd know its not AI-generated vocals, its an AI algorithm for voice alteration. Hence my very specific use of the words 'voice changer'.

[Super Piano Hater 64]

When I think AI, I think of something like Audimee where you have to sing your vocal and then upload it to their site to get the AI version. With these other plugins, you draw each note and pick a singer to sing it. Yes, there are exceptions but in general, that's how it works.

Quick question about the Vocaloid example. I've only used older versions with fully manual input. My understanding of the "AI" features in V6 was that their whole purpose is automatic transcription of an audio sample of a human vocalist. In other words, it's less like Suno and more like Dragon plus Melodyne, and the output is still produced by the same frequency domain synthesis algorithms as in older versions of the software. Is this accurate, or am I missing something important?

EDIT: To answer my own question, it seems that the new "AI" engine did introduce changes to the synthesis architecture, but based on what information is public it's not terribly clear what those changes actually are. The new engine was already undergoing testing circa 2011, so it predates the famous attention/transformer neural net architecture used in all modern generative AI.

[wagtunes]

When I think AI, I think of something like Audimee where you have to sing your vocal and then upload it to their site to get the AI version. With these other plugins, you draw each note and pick a singer to sing it. Yes, there are exceptions but in general, that's how it works.

Quick question about the Vocaloid example. I've only used older versions with fully manual input. My understanding of the "AI" features in V6 was that their whole purpose is automatic transcription of an audio sample of a human vocalist. In other words, it's less like Suno and more like Dragon plus Melodyne, and the output is still produced by the same frequency domain synthesis algorithms as in older versions of the software. Is this accurate, or am I missing something important?

EDIT: To answer my own question, it seems that the new "AI" engine did introduce changes to the synthesis architecture, but based on what information is public it's not terribly clear what those changes actually are. The new engine was already undergoing testing circa 2011, so it predates the famous attention/transformer neural net architecture used in all modern generative AI.

Honestly, I'm not sure. I don't pretend to be an expert on the technology. I just wanted to know why they can't make these voices have a little more character to them. That's all.

[wagtunes]

Tried slinging them through one of thae voice changer applications?

https://store.sonarworks.com/products/s ... nsion-pack

I have SoundID but I'm not talking about AI vocals.

If you have Sound ID as you say, then you'd know its not AI-generated vocals, its an AI algorithm for voice alteration. Hence my very specific use of the words 'voice changer'.

All I know is with SoundID I sing the song. I then turn on analyze and SoundID then replaces my vocal with the singer I chose. Vocaloid, on the other hand, you draw notes and pick the singer you want to sing those notes.

As far as the tech involved to do these two things, I couldn't care less. I just want to know why I can't get voices that have a little more character to them. And yes, with Audimee and SoundIDm you can (as shown by your post of the rock voices) but with these other ones, they all sound the same.

[Super Piano Hater 64]

Honestly, I'm not sure. I don't pretend to be an expert on the technology. I just wanted to know why they can't make these voices have a little more character to them. That's all.

My understanding is that they tried and failed. Some versions (+ voice banks) support the "growl" parameter, which produces downright awful results (seriously, it's terrible) and a few other parameters that are only barely more useful. The basic architecture of the system is just really limited in range compared to a live human voice. The single largest factor in the size of a voice bank is the number of samples available for transitioning between consecutive phonemes. For any transition where such a sample can't be found, Vocaloid falls back on a method of algorithmic interpolation that doesn't do any kind of sophisticated modeling of the physics of the human vocal tract.

As I alluded to earlier, this is my understanding of the V3/V4-era tech, so I don't know what specific improvements they might have made in V6. Now that I'm getting into it, I guess their fallback algorithm for phoneme transitions is one of the biggest weaknesses of the software, and an obvious area where they ought to be trying to improve the engine's internals, so maybe that's where they put some of the "AI."

[whyterabbyt]

All I know is with SoundID I sing the song. I then turn on analyze and SoundID then replaces my vocal with the singer I chose.

What, with the exact same intonation, timing and accent?

Hmmm, makes sense... they absolutely definitely do the extra 100,000% percent work to 'replace' the original with entirely new generated audio, rather than just processing the original audio.
And they refer to it as doing 'transformation' and 'morphing' because words like 'creation' or 'generation' just arent AI enough.
Yeah, that makes sense.

But hey, you dont want to even try a suggestion because something something AI. So it wasnt really about solving anything, just whinging.

It's odd that this is such a contentious subject for people.

hmmm, irony.

But mention trying to get variety out of a vocal plugin and everybody has a cow.

I don't get the utter disdain for vocal programs.

well, just consider how you're reacting to the notion of an AI algorithm, and apply that.

Vocaloid, on the other hand, you draw notes and pick the singer you want to sing those notes.

I just want to know why I can't get voices that have a little more character to them

Because what Vortifex already told you.
Because the companies are predominantly japanese, and aim at a market that isnt you.