Anyone interested in building an online AI music DAW for implementing on a FaaS cloud vendor?

[kivadour]

Some random thoughts...

Serverless / FaaS needs some proper thinking about the overall architecture.

I guess it's OK if the AI aspect sits in a serverless environment. Give it a prompt, it thinks for a while and it gives back an answer.

But the data flow of a traditional DAW, it's bonkers to put that on the cloud as nano-services.

I have to do a couple-day session with a SmartGPT connected to a present-day multi-API-search engine on this to bullseye SOTA audio processing. SmartGPT may even offer "nucleus sampling" instances where we discover that a major corp is applying signal processing in other....unlikely... applications that can then be effectively assimilated for our future web213.0, WiFi-7 (whatever) online AI music DAW.
I have a theory where we can apply an existing prediction AI to our audio buffering scenario, where latency and audio interpolation are successfully solved. Something about orthogonal signals in microwave towers and Fourier theory. Macro--->micro.

Adopting Ross Perot's maxim, we can look to existing online music services, albeit not as AI as we will be, such as audio tool, Soundclouds stellar buffering/disseminating practices being practically located in Berlin and other more songwriting-centric online audio processing services not solely techno. (Human vocal producers.)

An aside: WebGPU could help in the latency dept. (or not) but I want, otherwise, unprecedented audio interpolation for all and well within a budget model.

We'd have to be the WebGPU poster child for that, if so.

[kivadour]

Also musicians are rather protective about their work. What if this platform they used to create things goes bankrupt. They lost the projects they were working on, and even with a backup they cannot continue. Vendor lock-in. The market hates it.

I'd like to propose a completely free offline music DAW clone of the real thing.

Users can continue to produce on their home machines until about 10 Yeezys from now, when virtually everything will be encrypted and online. Physical home machines will be effectively obsolete.

In present day, music producers of our music AI will be okay with having to continue their work online, where our suite of optimum AI music tools are exclusively offered.

Economics AI will assist in the economics of this.

It will provide a payout gradation to original members and new "music AI heads" alike.

Rewarding both "the right brain dreamers and the left brain doer's".

I neither picture myself buying a yacht with this endeavor nor sponsoring an impossible crypto blockchain micro salary that's unsustainable for everyone's blood, sweat or tears devoted to this project.

Thanks for your astute and serious/humorous contemplation of my OP.

Hafta get to others replies now...

[BertKoor]

Yeah, keep on dreaming...

[WatchTheGuitar]

https://youtu.be/93JArmMY3n4

[vurt]

is it ready yet?

[BertKoor]

No, the list of buzzwords to tick hasn't been finalized yet.

[kivadour]

Because Stanford excels at business historically, it was kind of neat to hear their statistics reputation of Stanford in the business formulations.

Its a shame it doesnt excel at proper use of human language.

Please get ChatGPT to rephrase these posts, as I can hardly understand WTF is your point in all of this.

BTW, recent news show that local open-source models are catching up with ChatGPT 4 (which is not even not released out to wider audience yet) for a fraction of the cost, and can run locally on a decent GPU.
https://www.semianalysis.com/p/google-w ... Wx5kFbg0ZE
Think about it, as GPUs are in general underutilized by DAW users. Then, for music specifically we may do well with even smaller specialized model, trained on a curated data set.

Emerging AI is needing names we can use because it's developing fast.

"GPT" is a combo of neural networks and Open AIs chatGPT isn't the only one.

You're right.

Speaking of names, running a chatGPT clone "locally" is not really completely offline or private and it if it is then what we have is an X-bit quantized version.

And likely slow because of further compression techniques.

We're in an econometrics/tokenization bottleneck where folks build for AI and then discover they hit their monthly or weekly or daily threshold with their particular AI vendor.

It's hardly a way to build.

Let alone something great.

I'm presently looking for the right chain of prompts where your AI auto-tells you what your cost is at inserted points in your chats.

There's prompts that make GPT compensate for it's known weakness in the area of math itself so the same must be true for tokenization.

Otherwise there can be a specific 'tokenizer' AI that monitors your work on the fly.

[kivadour]

An important premise: I'm not interested in anything remotely close to AI daw or software as service.

With that said, I don't want to be *that* person, but there's something I can't really work out about this thread. You said

I'm looking to shift the music producer experience from "think like a computer" in your DAW... to "dream in color."

Do you already have any idea of how the user will interact with the software, or is it just your vision?

It seems quite a spanse to not really be interested in using the software ultimately and then offering a plethora of (good) questions about the look, shape and feel of the final product.

Before I answer your questions, I want to pass on that the project really needs momentum.

And not just any project software momentum.

Due to the unprecedented mass utilization of AI that mankind is in... I see you thinking big.

But not big enough...or "not small enough."

I'm sure you've read about the effects of Ai's "convergent evolution" or "parallel innovation" which connote several things:

1. LLMs that very recently cost millions to "train" are now available under $1000
2. Likewise, agents and "wrappers" are able to elicit what you specify (given you're comfortable with prompt engineering) in MANY streams from just ONE LLM...let alone calling it from multiple boutique LLMs that satisfy your specific AI goal....all without being charged to run the entire LLM and it's requisite energy costs.
3. As an aside: put on your "reverse engineer AI" hat.

Now considering the SPEED of AI to process the OLD SIZE and so on and so on.... it should then become apparent that people are "asking AI... to ask AI... to ask AI'.

Here's a good reference: https://learnprompting.org/docs/applied ... ld_chatgpt

So take the phenomenon of this iteration, and at the very least you will succeed at picturing a "leveraging" that yields a fusion of accuracy and speed that supersede the original question of the domain you want to solve itself.

With this momentum, you change the accuracy and speed of your entire start up.


So we're in it.

The AI thing.

But by no means is it limited to a corporation.

I'm well aware of established DAW-makers offline and online.

However, as I wrote before it will always be about the idea and how the idea gets finished and much less about the age, size or "huge tracks of land" of your music software company.

To only briefly answer your first question:

Simply compare the visual arts we have now: MidJourney, DALL-E, Stable Dif.,... and offer the same ease of contextual rapport but in our aural arts.


It's not much of a leap.


Depending on the "music AI heads" gathered, we will talk about interfaces, for instance.

Sure, "It's got a SOTA/AI 'dark mode'"... but what will users keep coming back for time and time again?

Suggestion: Experiment with WavTool music AI because it actually is AI + music married together in one.

Seek out others like it after you learn its limitations.

But at the very least, I bet your ideas take root MUCH faster when you are actually using a music DAW....with AI... in some way, shape, or form.


I'll be back to answer other serious and humorous questions alike.

[vurt]

are you an ai yourself?

[counterparty]

I think the best way forward would be for you to start building it. If people see you've got something good going on they will be much more likely to join you.

Starting to actually build things is also an excellent way to work through the details of what it is you're building (because with writing software it inevitably it turns out there are a lot of things you didn't understand or realise before starting to write the code makes it obvious).

[BertKoor]

Start building something is indeed the best advice. But I have my suspicions as to why this has not started yet.

And only now I noticed this thread is in the DIY forum. Not the DSP section where Real Programmers (not quiche eaters) hang out.
There is a pinned thread titled README - For non-programmers with great ideas. Mandatory reading material!
<TL;DR> :

Ideas are cheap. Implementations are expensive.

[kivadour]

Start building something is indeed the best advice. But I have my suspicions as to why this has not started yet.

And only now I noticed this thread is in the DIY forum. Not the DSP section where Real Programmers (not quiche eaters) hang out.
There is a pinned thread titled README - For non-programmers with great ideas. Mandatory reading material!
<TL;DR> :

Ideas are cheap. Implementations are expensive.

Well I'm basically giving the idea works away for free so anyone can start who has viewed this thread.

To date only in the most recent, 1/3rd of replies to this thread, I've gotten sincere + serious responses. Not to mention not one willing to DM. Needless to say, I got my Ghandi moment for the first 2/3 of responses.

Also, like lots of other celebrities (I'm not. I assure you.) who have come out about OCD, I may appear to be repetitive in my answers. I apologize for my run-on sentences.

Anyway, until someone DMs me, I will simply do my due diligence and add to the quality of the base of KVRaudio.com user history.

Thank you for contributing to this thread.

[kivadour]

Some random thoughts...

Serverless / FaaS needs some proper thinking about the overall architecture.

I guess it's OK if the AI aspect sits in a serverless environment. Give it a prompt, it thinks for a while and it gives back an answer.

But the data flow of a traditional DAW, it's bonkers to put that on the cloud as nano-services. Here's why.

Audio gets processed in small buffers of anywhere between 32 and 512 samples, depending on how much latency the user is willing to experience. Larger buffers are more efficient but latency is high as well. Smaller buffer sizes have much more processing overhead costs. Each full buffer of audio needs to flow through the whole processing chain.

Synth -> zero or many effects -> mixer channel -> effects -> mixer bus -> mastering effects.

Do this for 30 tracks in parallel with 6 components each, on a buffer size of 128 samples at 48 kHz that's 180 serverless function calls 375 times per second. Nearly 70.000 for a second, on a track of 3:30 that's 14 million calls. It usually takes a musician some hundred attempts before even remotely satisfied. Now look up the cloud computing costs.

Also musicians are rather protective about their work. What if this platform they used to create things goes bankrupt. They lost the projects they were working on, and even with a backup they cannot continue. Vendor lock-in. The market hates it.

I think this recent link can loosen the concerns about the admitted large gobs of audio data processed that would traditionally post high costs on the online AI music DAW:

https://www.deepmind.com/blog/alphadev- ... algorithms


Also, this paper was written two years ago: https://arxiv.org/pdf/2208.08100.pdf
The paper improves A.I. evals...as in A.I. self-eval...of code itself.

I am building my own personalized AI with an emphasis on websearch like 'GPT-4 with browsing' and discoveries like these will pinpoint the "domain dilemmas" much faster and with exponential accuracy.

Especially with my multi search engine endpoint/API.

[kivadour]

I've been thinking about possibilities and I envisioned this:

Autonomous AI agent tightly integrated with DAW

- Takes user input as a task: "Create me a 5-minute syntwave track in C minor"

So this is like two things going on.


1. Download a free copy of WavTool out of Vancouver. Though in training-wheels now, it can give strong hints at just this sort of thing.

2. Someone who's actually organized (prompt engineering and/or multi-agent systems) a way to teach an A.I. on music theory. Sure, ChatGPT was trained on the internet but any person(s) can unveil her/his "music theory A.I.", specifically aimed at the domain of music theory.

[kivadour]

I've been thinking about possibilities and I envisioned this:

Autonomous AI agent tightly integrated with DAW

- Proceeds executing the task autonomosuly, but still allowing user intervention or interruption, like this: https://agentgpt.reworkd.ai/pl

8 weeks ago when visual artist job-loss woes and stunning context aware AI art was getting traction, an award-winning artist was interviewed about the scale of negativity the artist received from the fear-mongering passed around the web, and he stated that...of the contemporaries/peers he witnessed putting out A.I. art that received notoriety..... these same artists were already known on the internet for accomplishing kickass art before A.I. Well before the last 24 months.

If you're comfortable within your D.A.W. already, then you'll simply excel at using an A.I. music DAW.

So yeah!

There'll be a lot of intervention from the human music producer still.

It will just be a shift as we realize that the mechanics of songwriting in the DAW has been transformed.