Imho it's mainly about finding a sweet spot of capabilities vs speed-to-get-results. If what you want to achieve is already possible with a pre-wired synth and you are essentially twenty knob-turns away from your desired result - go with it.
Next level would be modulars like VM, VCV, Tassman etc. and semi-modulars like Bazille, ACE etc. with high level modules and -imho most important - full audio rate modulation capabilities. For me this is less about "casting chaos" but more about the ability to modulate everything with controllers that provide inputs at almost (or literally) audio rate (100 Hz+ to several kHz). (Eigenharp, Morph, Linnstrument, Continuum, Soundplane - you name it). No "control rate parameter slewing" in the way with these audio rate capable modulars!
Reaktor is a little bit of a middle ground. Lots of useful high level modules combined with the capability to go deeper. Haven't worked much with Reaktor yet, but from what I tried for the low level stuff Reaktor was imho less convenient than Max&Co which offer better patching and abstractions.
And for Max, PD etc. I'm a little undecided. It's even further down the rabbit whole. And probably great for prototyping. But wondering whether a text based language with some good libraries isn't easier to get structure into the project, which might initially mean some overhead but could pay off fast when the project is supposed to get big enough. So there is probably a sweet spot area that justifies Max, PD&Co and a threshold above which it might be better to use a text based language.
That's why I like VM and VCV - they provide a nice combination of (according to the distinction above) "level 2" high abstraction modules - and writing own modules in a convenient way with wide-spread text based languages.
Saying "I can do everything" with an open ended programming language (which even Reaktor is at the end) is imho a little bit cheating. Sure, you can build everything that's not there yet. But you also *have* to build everything that's not there and you cannot find a ready to use library or template for (for Reaktor there's fortunately quite a lot in the user library - imho one of it's biggest pros).
But if nobody has already done the work for you then it's like a pile of papers and a pencil. Sure, no need to go to a gallery or buy a book, you can paint all these pictures yourself and write any book you can think of. And if you are a fast painter you might even not need any movies. But...
