I want to code my own music-making software. What subjects, books and websites should I study?

[inhahe]

Hi, I want to write some software to help me make music. I have a lot of SoundFont2 instruments and I want to write the code to render notes from them and apply all of the SF2-defined effects to them, and also code for various other effects like phaser, flanger, chorus, reverb, echo, time stretching, pitch modification, etc.

I may also want to code my own instruments using various synthesis techniques including physical modeling synthesis, granular synthesis, wave terrain synthesis, etc. - even graintable synthesis if I can figure out how it works (I think currently it's only one product does it and it's a commercial product).

I may also want to provide support for other kinds of instruments and plugins, like Kontakt, AU, VST/VSTi, DX/DXi, Propellerhead Rack Extensions, and others. (I don't know if I'd be able to support Rack Extensions because according to https://www.propellerheads.se/reason/rack-extensions they integrate with all of Reason)

I'll probably also want to understand DSP really well and invent my own kinds of effects, but I don't know much math beyond algebra, geometry and trigonometry (and I may have forgotten most of the latter two too)..

My question is, what books should I read and what areas of math and DSP should I learn for these purposes?

Also, are there any online repositories of code or pseudocode I can use for any of these purposes (either as parts of larger open-source programs or, preferably, as stand-alone algorithms)? Or any other websites I should visit for specs of formats I'm interested in?

Thanks..

[ablaut]

I would suggest you start with Reaktor, Max/MSP, and/or PureData. Then look at SuperCollider and CSound. If you still want to code your own plugins after that, you should learn C++ and dive into the details of DSP. I know JUCE is a good open source framework, but further than that I'm out of my depth.

[inhahe]

JUCE looks really useful, thanks. Do you know, or does anybody else know, if there are any extensive repositories of /quality/ (commercial -music worthy) instruments available for Max/MSP, PureData, SuperCollider, etc.?

Also, I've been looking into ChucK recently (I've briefly looked into most of those other sound programming environments at other times in the past), how do you think ChucK compares to Reaktor, Max/MSP, PureData, SuperCollider, and CSound?

[bltpyro]

https://www.coursera.org/learn/audio-signal-processing on coursera is a free course that is pretty good to start coding. They use python which you probably wouldn't use for a released product, but is great for learning and testing algorithms.

[BertKoor]

Do you know, or does anybody else know, if there are any extensive repositories of /quality/ (commercial -music worthy) instruments available for Max/MSP, PureData, SuperCollider, etc.?

If you want to make an application that makes use of existing instruments (and not like we probably all assumed you wanted to create an instrument yourself), it sounds a bit like you're creating some sort of a DAW. Plenty of free instruments exist in VST format.

So then the first step is to create a VST host. There's an open source example of that floating around somewhere, think it's by Herman Seib. From there you can extend it and implement your own ideas.

Good luck !!!

[edit] After reading your first message again, it sounds like your ambitions are rather high! If you want to run, you first have to learn how to walk. That means you first have to learn how to crawl. So I'd advice you to start really simple. Take small steps, do it incrementally. Something like this might be a roadmap:

1. Play a WAV file to default system soundcard
2. Play a WAV file to ASIO driver
3. Play a sample from a SF2 file
4. Repitch a sample, implement loop points
5. Implement all other aspects of SF2 (adsr envelopes, loop points, etc etc)

But then I'm thinking why reinvent that wheel if perfectly fine SF2 players already exist as a VST. Maybe you better concentrate on building a VST host. And that's not an easy task. I repeat: your ambitions are very high. If you're not a programmer already I'd forget that dream.

[inhahe]

Thanks for the responses, bltpyro and BertKoor. I realized after I posted that I could probably just use VST instruments and that some of the sound programming environments I mentioned might support VST. I looked it up and apparently some of them do. I also realized that I had found a VST implementation of SoundFont2 rendering so I could probably use that for all the SoundFont2 instruments I have.

@BertKoor I am a programmer already, but you're right that it's really ambitious. Most of those plans are tentative, maybe dreams. But what I want to do at the very least is programmatically control instruments and have a large collection of instruments to use, such as my Digital Sound Factory examples collection, and probably do some custom sound creation or maybe instrument creation too. The ability to control certain aspects of my music as it's playing using some input device or another may also prove to be necessary.