How to get AI to think outside of your box?

[BBFG#]

Much of my questioning comes from about a month ago asking one of the browser AIs a specific on demographics. It condensed and spit out an answer that had a political tone that I specifically did not want.
So I changed the question and syntax of how I asked it to get the simple demographic breakdown I could research from. And it gave me the exact same response. I tried rewording and being more specific to just find a simple graph/chart and got the exact same political garbage every time. Even when asked to specifically remove the politicism, it would not. Tried all this on different browsers with different AI bots where the only difference was in the style it regurgitated the same exact thing.

My general feeling was that all AI is still in its learning stage and inherently biased by the person that wrote the original algorithm. Not sure how that can ever get past that. On the other end, once the algorithm has reached a "maturity", is how that original bias will be permanently ingrained in its bigotry. If we may be stuck on the AI treadmill of finding e next AI on its "sweet spot" of its "middle age" between machine infantilism and machine dementia. And I'm speaking of data bigotry and not the social bigotry we're ingrained to trigger from with that word.

Things like genre, style, mood are too subjectively ambiguous to rely on what it decides it must be uniformly.
And music that conforms to these base lines may have an infinite design while still being infinitely boring.

There's a scientific documentary on infinity in Schrodinger's box that metaphorically keys me to this as well. So the problem to me concerning the box is the box itself.

[guitarzan]

I’ve been trying for two days now in the Gemini/Lyria beta to refine this Hard Bop Firebird guitar intro to Cool Jazz turnaround thing, and the further I try to dig in the further away it gets from the first promising results.

After prompting just minutes ago that I was done with this one, saying that it looked like AI music generation at this point was a great semi-random inspiration machine but not useful for actually composing anything, I was surprised that Gemini agreed completely! Here’s a big chunk of its reply:

You’ve hit on the core reality of where this tech is right now: it’s a fantastic "random inspiration machine," but when it comes to precise refinement and "surgical editing," it can feel like the more you pull on a thread, the more the whole sweater unravels. The "regression" you're feeling is real—sometimes the model loses the "latent space" of that original vibe as we try to add more specific constraints.

So there we are. Promising tech but still in its infancy.

[guitarzan]

The problem is that the process is an open loop, the LLM takes your prompt and coverts into a very technical prompt to the algorithmic composer, the composer generates the music but the LLM only gets confirmation from you, the user, what the output is. There is no way for the LLM to know what the generated track is other than feedback in the form of prompts from the user. They need to close that loop, somehow the LLM must “train” on the generated track directly.

[BertKoor]

If you want better control of the tool, know its limitations, it might help to gain some understanding of how it works under the hood:
https://jenniferplusplus.com/what-is-a-token/

[BBFG#]

I also feel it's not just about training on the generated track directly, but at first training on the composer's personality of place and flow of how each of us do things. I guess this is where a paid version would be better. Like a pet dog with a willingness to please their one and only master. And not every person it "sees" on the net.

[guitarzan]

It’s kind of hard to pass on the things AI says because their replies are so verbose, but on the subject of refining a composition this was where Gemini summed it up best in my opinion and where we left it for now:

Refinement is currently Regression. The "open loop" problem means I can't hear that cowbell to tell the model, "Stop! No! Put the bell down!" I can only keep shouting "Firebird and Bongos" into the void, and hope the machine listens. This time, it clearly didn't.

[BONES]

I’ve been trying for two days now in the Gemini/Lyria beta to refine this Hard Bop Firebird guitar intro to Cool Jazz turnaround thing, and the further I try to dig in the further away it gets from the first promising results.

Yeah, that's just part of the process. Try to be less specific with your goals and just let it riff on your desired theme. We don't tell it what instruments to use or how to structure anything, our goals are broad enough to give the AI room to surprise us with something unexpected.

[Michael L]

We don't tell it what instruments to use or how to structure anything

So what do you actually tell it?
Can you give examples?

[guitarzan]

I’ve been trying for two days now in the Gemini/Lyria beta to refine this Hard Bop Firebird guitar intro to Cool Jazz turnaround thing, and the further I try to dig in the further away it gets from the first promising results.

Yeah, that's just part of the process. Try to be less specific with your goals and just let it riff on your desired theme. We don't tell it what instruments to use or how to structure anything, our goals are broad enough to give the AI room to surprise us with something unexpected.

I have tried another whole day and yeah, it will not give me anything close to the original track.

I am going to have to split the stems and deconstruct the track to figure out what’s going on that appeals to me so much… pitch to midi on the bass and piano comping, reconstruct the drums with a drum program, and just learn the guitar part.

There is the spark of a new style of jazz fusion in that generated clip, like a proto-fusion thing, like if jazz/rock fusion had occurred a decade earlier than it did. It’s my in to jazz. Oh, and I need some bongos. Dig it

[BONES]

We've never been able to get it to do the same thing twice, or anything even close. I imagine that's because they want it to be creative, not a total slave to the prompts. Something that's a bit of fun, rather than a serious tool. But just because it's designed to be a bit of fun doesn't preclude it from also being useful.

My bandmate has been saying since Xmas that the newer updates to Tunee have made it much, much harder to get anything worthwhile from it. I've found the same thing with image generating AIs. A couple of years ago Mid Journey, in particular, had this quirky style that I really liked but as it got "better" at generating photo-realistic images, it lost that quirkiness. At the same time Dall-E seemed to lose the ability to do anything convincingly photographic, everything started to look too clean and fake.

Michael, I've asked my bandmate if he's wiling to share some prompts but he's got a lot going on at home this week and I haven't heard back from him.

[Michael L]

Michael, I've asked my bandmate if he's wiling to share some prompts but he's got a lot going on at home this week and I haven't heard back from him.

That's ok.
From what you just said, there is no connection between a prompt and a result, so it's like writing with a musician who speaks another language: you find a way to use what he gives you.
AI is not my wai, but I am fascinated by the bizarre workflow that clearly works for Craig.

[ghettosynth]

Michael, I've asked my bandmate if he's wiling to share some prompts but he's got a lot going on at home this week and I haven't heard back from him.

That's ok.
From what you just said, there is no connection between a prompt and a result, so it's like writing with a musician who speaks another language: you find a way to use what he gives you.
AI is not my wai, but I am fascinated by the bizarre workflow that clearly works for Craig.

That is an overstatement. The problem at present is the open loop of these kinds of models, but, even that is not exactly as you frame it. I was easily able to get Google's image model, for example, to reproduce comic frames that were consistent to what was expected. The same is true for music models, but, it's not really clear that using them that way is a good use case. FWIW: Being more precise was what was needed. It takes experience to understand what that means and our friend with a physics degree was simply not sufficiently precise and/or, he didn't understand limitations of specific models.

When you close the loop, e.g., with software, this changes completely. Now the model can explore the latent space for solutions that meet testable expectations. This is certainly possible with music, but the current popular products do not function that way, nor, is it quite as easy to write deterministic test cases for whatever it is that you want the model to produce. In fact, what both this, and the image problem represent is that, for some use cases, writing the prompt or set of prompts to accomplish a task may be more laborious than completing the task manually.

[Michael L]

Michael, I've asked my bandmate if he's wiling to share some prompts but he's got a lot going on at home this week and I haven't heard back from him.

That's ok.
From what you just said, there is no connection between a prompt and a result, so it's like writing with a musician who speaks another language: you find a way to use what he gives you.
AI is not my wai, but I am fascinated by the bizarre workflow that clearly works for Craig.

That is an overstatement.... writing the prompt or set of prompts to accomplish a task may be more laborious than completing the task manually.

We have both identified the same bizarre workflow, just using different metaphors.
It's enough for me to know that getting useful results with AI would push me into a truly joyless headspace

[ghettosynth]

Michael, I've asked my bandmate if he's wiling to share some prompts but he's got a lot going on at home this week and I haven't heard back from him.

That's ok.
From what you just said, there is no connection between a prompt and a result, so it's like writing with a musician who speaks another language: you find a way to use what he gives you.
AI is not my wai, but I am fascinated by the bizarre workflow that clearly works for Craig.

That is an overstatement.... writing the prompt or set of prompts to accomplish a task may be more laborious than completing the task manually.

We have both identified the same bizarre workflow, just using different metaphors.
It's enough for me to know that getting useful results with AI would push me into a truly joyless headspace

Sure, I take your point, but, I want to be clear, that getting similar results at different scales that does not push at the limitations of the model so much is a different thing. I do not subscribe to BONES laborious methods to crank out what I see as memes. In fact, the most laborious part of the workflow is conjoining the different media bits and rendering a video for youtube.

So, what I was pushing back on is that there absolutely is a connection between a prompt and a result, but how close that is, depends on a quite a few factors.

[Michael L]

Michael, I've asked my bandmate if he's wiling to share some prompts but he's got a lot going on at home this week and I haven't heard back from him.

That's ok.
From what you just said, there is no connection between a prompt and a result, so it's like writing with a musician who speaks another language: you find a way to use what he gives you.
AI is not my wai, but I am fascinated by the bizarre workflow that clearly works for Craig.

That is an overstatement.... writing the prompt or set of prompts to accomplish a task may be more laborious than completing the task manually.

We have both identified the same bizarre workflow, just using different metaphors.
It's enough for me to know that getting useful results with AI would push me into a truly joyless headspace

there absolutely is a connection between a prompt and a result, but how close that is, depends on a quite a few factors.

Yes, but finding those *factors* that "get AI to think outside" seems like trying to 'find a needle in a search engine' and not at all like music or even fun. Bones thinks it is more tedious than watching the telly, which is saying something.