AI plugin uses neural networks to generate infinite drum samples - feedback welcome

[Guenon]

This is very promising. Props to you, and everyone else working on similar projects. In the future, would it be possible (or... totally out of the question) to allow for a separate component/mode for the end user to train this software with a large and completely custom set of training sound files? Even if it takes a very long time to get good training results? If things like this were possible, the potential experimental use, for all kinds of very interesting oneshots and variations thereof... seems very very intriguing. (Also from a sound design perspective, which is a field I work in quite a bit.)

[Rahodees]

Once beta/early-access are over I'm assuming we'll have to pay for the release version?

[Rahodees]

This is very promising. Props to you, and everyone else working on similar projects. In the future, would it be possible (or... totally out of the question) to allow for a separate component/mode for the end user to train this software with a large and completely custom set of training sound files? Even if it takes a very long time to get good training results?

Obviously OP will be more knowledgable about this than me but fwiw my understanding is on the typical end user computer, it would be a matter of _weeks_ for software like this to do its thing. In the labs and startups etc they use banks and banks of high end graphics cards to host the AI algorithms.

[Guenon]

my understanding is on the typical end user computer, it would be a matter of _weeks_ for software like this to do its thing

I know, that's why I said "even if it takes a very long time" . Offering the functionality would still open up very interesting avenues for those willing to experiment.

[dionenoid]

Seems that Steinberg Backbone just added something similar in 1.5 : https://youtu.be/qLFo2udvWfA

[SLiC]

Like busses!

Artificial drum intelligence by Sony CSL - Backbone 1.5 now comes with DrumGAN, an innovative drum generator developed by Sony Computer Science Laboratories, Inc. (Sony CSL). Powered by artificial intelligence (AI), it creates new kick drum, snare drum and cymbal samples in a very fast, intuitive way. It also analyzes samples, can replicate them and create countless variations. DrumGAN is the perfect addition to Backbone for crafting new drum sounds entirely from scratch, while staying in your favorite sample editing environment.

[SeeingInMidi]

Woaw, that analyze button is very impressive in backbone 1.5. This will allow sample makers to "bootleg" samples from other sample packs without the worries of other manufacturers being able to ID them. Very excited for both projects.

[ai_music_guy]

The sound generation does generate a lot of unusual sounds, not many are actual kicks and snares. But as a sound generator, it does work well.

- Panning per cell is a must for a drum plug-in.
- So is multi-out-put.
- The ability to reverse samples would be nice
- Undo is not working at all. If it were working, it would be nice to have a history drop-down list per cell to select one of the last ten iterations.
- a global generate button (like the dice in poly plex). In addition, lock individual cells to keep them in their state.
- A live mode for the generative music folks. Generate to samples in the background and have them change after x number of bars in each selected cell.

- Panning per cell is a must for a drum plug-in. [great idea]
- So is multi-out-put. [already on the to do list, coming soon after the initial launch]
- The ability to reverse samples would be nice [great idea]
- Undo is not working at all. If it were working, it would be nice to have a history drop-down list per cell to select one of the last ten iterations. [sorry, this is our fault. the current beta build just introduced the UI for these features, the next one will implement them. sorry for the confusion]
- a global generate button (like the dice in poly plex). In addition, lock individual cells to keep them in their state. [LOVE THIS]
- A live mode for the generative music folks. Generate to samples in the background and have them change after x number of bars in each selected cell. [already on the to do list! "humanize" or "jitter" or something]

Thanks for the feedback!

[ai_music_guy]

Tried this out, and absolutely incredible for experimental music. A lot of these are sounds that I haven't heard in anything before. +1

That's exactly what I'm interested in with this new approach to creating sounds, getting novel and unheard sounds.

Or on the other hand, to get tiny variations of the same sound to feed the round robin function of my sampler.

I can't wait to try it!

Thanks! This exact feature is already on our to do list for right after the launch release

[ai_music_guy]

Tried it out for a bit in Reaper, didn`t crash or anything like that. It`s really good for midrange sounds, loved the claps, snares are also pretty good. Kicks and hihats not so much. Overall has very noisy/grungy character, especially can be heard in decay and that really hurts how hihats sound.
Would be nice to have a pitch control. Maybe also pitch envelope slider, going from center to left to pitch down/ center right to pitch up.
Probably something that`s already known, but playhead in sample window doesn`t retrigger in time with midi note. Also for my needs it would be enough just having 1 pad, no need for 16.
Overall I like it, already rendered a few wavs for my one-shot collection so thank you for making this and good luck in developing it further.

This is really great and fair feedback. We'll look at it as a team and get most of this onto the to-do list. Thank you for the time and the well-wishes. We hope to make something dope that is actually helpful.

[ai_music_guy]

two things after testing

Needs drag and drop audio
reduce the amount of sample rate/bits......it makes almost every sample sound as though they have unwanted transients. The bit effect is cool, but something is happening with the sample rate that has a high ping

1000% needs drag and drop, we're working on it

Re: the sample rate/bit effect - yes, I know what you're talking about and you're right. We are evolving are neural network architecture and our training scheme and will chip away at this stuff. Since we generate the sounds from our servers, the sounds will just automatically get better as we keep training and updating the model

[HOTF]

Sounds Really interesting. Will demo it this wwek a give you feedback.

[ai_music_guy]

Seems that Steinberg Backbone just added something similar in 1.5 : https://youtu.be/qLFo2udvWfA

Very interesting, thanks for sharing! I think they picked a sensible minimum set of features and I understand the tradeoffs they're making given they have an existing product - it all needs to stay cohesive. And it's also clear they're focused on realistic, traditional drums.

We take a different approach more at the fringe with fewer (no) boundaries. You can use our tech to create traditional sounding drums, sure, but you can also get funky, trippy, weird one-shots that add amazing texture to your song. And discover sounds that have never been heard by anyone, ever. And our ambitions go far beyond drums.

I have to checkout Backbone, though, it looks legit. I welcome some choice and competition in the generative sound space

[Alexander137]

Interesting.
Until it generates the desired type of drums, as I understand it, the kick was generated approximately 20 times.
But it looks promising.
I would like more control options, such as changing the pitch, panning or choosing the length of the generated drum.
(sorry for my English)

[jules99]

- A live mode for the generative music folks. Generate to samples in the background and have them change after x number of bars in each selected cell.

- A live mode for the generative music folks. Generate to samples in the background and have them change after x number of bars in each selected cell. [already on the to do list! "humanize" or "jitter" or something]

Thanks for the feedback!

Just to clarify, because I think you might have misunderstood. A live mode for generative purposes would mean that you load a set of generated samples in the plug-in and start sequencing them Or have a sequencer trigger them. Meanwhile, the plug-in would, in the background generate new samples for each cell. And you could set the number of times a sample is played until a sample is replaced. Or the number of bars until all samples change. Or just some.

But I'm not 100 percent sure if this would work since being connected to a server is not always feasible for live situations.