Apple's habit of breaking plugins
-
- KVRAF
- Topic Starter
- 3183 posts since 31 Dec, 2004 from People's Republic of Minnesota
Sorry, not an audio developer, but I'm posing this question for audio developers. Why does every release of Mac OSs break plugins (primarily audio units)? I know the "break" is usually the result of validation problems. I guess I have a better, second question. Are the changes that Apple makes to their OS actually drastic enough to merit new validation requirements? Or are they arbitrary, unnecessary decisions that are made without concern for developers and/or users? Thanks.
-
- KVRer
- 25 posts since 12 Jan, 2008
Great question. A lot of people will probably have differing opinions on this, but for an OS developer there's really a trade-off between compatibility with existing products and improving the system for the end user. Since currently the main competition is Microsoft (of course, until the year of the linux desktop), I'll use them as a comparison point.
The way this trade-off happens in practice is quite interesting. Often, what Apple sees as undesirable behavior is something that an existing product requires to work. This can happen for three reasons:
1) The developer didn't read the documentation, instead just experimented until something worked. Unfortunately, this resulted in the developer depending on some behavior from the OS that the Apple programmed by accident. When it comes time for an OS upgrade, Apple may not even imagine that someone is relying upon their accident, so they might correct it. In general, I think this is an unfortunate circumstance, because the developer didn't do anything "wrong", but ultimately it's the way it should work. The developer did something expedient, and now they have to pay the piper. Microsoft's philosophy on this is actually completely the opposite; in general, they will leave in broken code to keep the developer happy, even if it negatively impacts users.
2) The developer did read the documentation, but didn't pay attention to the "deprecated" warnings put in by the documentation writer, and therefore depended on deprecated behavior. Now, when an OS upgrade come and the behavior changes, the developer really has no reason to think it should work. One special circumstance here is, what if the developer needs functionality that has been deprecated without replacement? Well, then they are in bad shape. Apple rarely deprecates things without replacement, but it does happen. There was a big problem with this during the 64 bit transition, especially with video APIs. Microsoft routinely deprecates APIs without replacement, particularly in the audio and music spheres. However, because of their developer focus (rather than user focus), they rarely actually remove those APIs once they are deprecated. This becomes a bit of a vicious cycle - few developers take Microsoft deprecation warnings seriously, thus further disincentivizing Microsoft from removing these crufty APIs.
3) The developer read the documentation, used proper APIs, and then Apple swept the rug up from under them, removing the APIs they needed. This is a place where I would put partial blame on Apple. This happens quite rarely, but to be fair probably significantly more than Microsoft. As a general rule, Apple gives a year or so of warning. In some cases this is enough, in some cases not. Some developers resent being forced to do extra work by OS vendors, so no amount of warning is enough for them. The times I remember, as a developer, being a bit shocked was the x86 transition, and the removal of the Carbon API from 64-bit. I think it's really hard to see the x86 transition as a bad thing for the user, so that was definitely a case where forcing developers to do some extra work was appropriate. I'm still not sure about the Carbon thing, though, that one stung.
4) There was no appropriate documentation, so the developer had no idea what to do. This is something that all OS vendors struggle with, as operating systems are incredibly complex things that are hard to document. Add on to that the fact that Apple and Microsoft have trade secrets and proprietary information embedded in the OS that they *don't want* to document, and you have a recipe for developer frustration. Overall, I'd say the Apple documentation is better than its competitors, partially because the kernel (lowest layer) of the OS actually is open source and fully documented. But as I said, this is something that's kind of an unsolved problem.
Anyways, pretty hard to answer such an abstract question concretely, but there's my best shot. I think most of the time these breakages could have been prevented by a bit more diligent work up front by the developers. Although, if your program works, there's not much incentive to change it until an new OS comes along and breaks it, so I see the developer side too.
As to your specific question of 10.11, my understanding is that most plug-in breakages happen because of security features added at a very low level of the OS, that prevented plug-ins from doing sort of sketchy things that they were doing. The idea here is by locking the system down, fewer security threats will slip in. The downside is that law-abiding citizens have to take their shoes off at the airport. Why do these security features lock down Audio Plug-ins during validation? No idea, doesn't make sense to me, but I'm not a security expert. If I had to guess, the Apple guys ran out of time and it was easier to have the security lock-downs apply to auval than to code a special exception.
One thing I would love to see is better conformance tests provided by Apple, so we could detect the kinds of problems that might show up in a future OS release, even if they aren't bugs yet. Currently the only way to test conformance is to carefully cross examine your code with the Apple-provided documentation, a time consuming and tedious process.
The way this trade-off happens in practice is quite interesting. Often, what Apple sees as undesirable behavior is something that an existing product requires to work. This can happen for three reasons:
1) The developer didn't read the documentation, instead just experimented until something worked. Unfortunately, this resulted in the developer depending on some behavior from the OS that the Apple programmed by accident. When it comes time for an OS upgrade, Apple may not even imagine that someone is relying upon their accident, so they might correct it. In general, I think this is an unfortunate circumstance, because the developer didn't do anything "wrong", but ultimately it's the way it should work. The developer did something expedient, and now they have to pay the piper. Microsoft's philosophy on this is actually completely the opposite; in general, they will leave in broken code to keep the developer happy, even if it negatively impacts users.
2) The developer did read the documentation, but didn't pay attention to the "deprecated" warnings put in by the documentation writer, and therefore depended on deprecated behavior. Now, when an OS upgrade come and the behavior changes, the developer really has no reason to think it should work. One special circumstance here is, what if the developer needs functionality that has been deprecated without replacement? Well, then they are in bad shape. Apple rarely deprecates things without replacement, but it does happen. There was a big problem with this during the 64 bit transition, especially with video APIs. Microsoft routinely deprecates APIs without replacement, particularly in the audio and music spheres. However, because of their developer focus (rather than user focus), they rarely actually remove those APIs once they are deprecated. This becomes a bit of a vicious cycle - few developers take Microsoft deprecation warnings seriously, thus further disincentivizing Microsoft from removing these crufty APIs.
3) The developer read the documentation, used proper APIs, and then Apple swept the rug up from under them, removing the APIs they needed. This is a place where I would put partial blame on Apple. This happens quite rarely, but to be fair probably significantly more than Microsoft. As a general rule, Apple gives a year or so of warning. In some cases this is enough, in some cases not. Some developers resent being forced to do extra work by OS vendors, so no amount of warning is enough for them. The times I remember, as a developer, being a bit shocked was the x86 transition, and the removal of the Carbon API from 64-bit. I think it's really hard to see the x86 transition as a bad thing for the user, so that was definitely a case where forcing developers to do some extra work was appropriate. I'm still not sure about the Carbon thing, though, that one stung.
4) There was no appropriate documentation, so the developer had no idea what to do. This is something that all OS vendors struggle with, as operating systems are incredibly complex things that are hard to document. Add on to that the fact that Apple and Microsoft have trade secrets and proprietary information embedded in the OS that they *don't want* to document, and you have a recipe for developer frustration. Overall, I'd say the Apple documentation is better than its competitors, partially because the kernel (lowest layer) of the OS actually is open source and fully documented. But as I said, this is something that's kind of an unsolved problem.
Anyways, pretty hard to answer such an abstract question concretely, but there's my best shot. I think most of the time these breakages could have been prevented by a bit more diligent work up front by the developers. Although, if your program works, there's not much incentive to change it until an new OS comes along and breaks it, so I see the developer side too.
As to your specific question of 10.11, my understanding is that most plug-in breakages happen because of security features added at a very low level of the OS, that prevented plug-ins from doing sort of sketchy things that they were doing. The idea here is by locking the system down, fewer security threats will slip in. The downside is that law-abiding citizens have to take their shoes off at the airport. Why do these security features lock down Audio Plug-ins during validation? No idea, doesn't make sense to me, but I'm not a security expert. If I had to guess, the Apple guys ran out of time and it was easier to have the security lock-downs apply to auval than to code a special exception.
One thing I would love to see is better conformance tests provided by Apple, so we could detect the kinds of problems that might show up in a future OS release, even if they aren't bugs yet. Currently the only way to test conformance is to carefully cross examine your code with the Apple-provided documentation, a time consuming and tedious process.
-
- KVRAF
- 2275 posts since 4 Dec, 2011 from Brasília, Brazil
there were significant breaks since 10.7?I don't think so.
The Audio Unit standart is receiving a great update on 10.11. A nice thing is a kind of sandbox model (multiprocess a la chrome), so, if a plugin crashes, it will crash only the plugin, and not the whole host.
The Audio Unit standart is receiving a great update on 10.11. A nice thing is a kind of sandbox model (multiprocess a la chrome), so, if a plugin crashes, it will crash only the plugin, and not the whole host.
My soundcloud: https://soundcloud.com/waltercruz
-
- KVRAF
- 11093 posts since 16 Mar, 2003 from Porto - Portugal
Every OS release had issues, usually with plug-ins, and sometimes also with major applications like hosts, requiring specific updates (this happened with Cubase, for example). So, the question is more not "if", but "where" were significant breaks.waltercruz wrote:there were significant breaks since 10.7?I don't think so.
Really? I didn't notice anything that noticeably improved the AU behaviour. If they are only concerned with crashes, that is thinking low, IMO. Besides, the whole AU validation process sucks, IMO. The way things are, I'd rather have the option to jump over it, and go through the crashing process, eliminating the offending plug-ins. Because having to wait about half-hour for the host to start every time there is an OS update and the plug-ins need to be validated again isn't exactly my idea of "great update" for an OS.waltercruz wrote: The Audio Unit standart is receiving a great update on 10.11. A nice thing is a kind of sandbox model (multiprocess a la chrome), so, if a plugin crashes, it will crash only the plugin, and not the whole host.
And during the Beta, Apple broke USB (again - they had already done that during the Beta of Yosemite too). Seems like Apple still doesn't deal very well with USB. They fixed it already, but there was a release where I simply could not use the USB dongles, or the áudio interface.
Fernando (FMR)
-
- KVRAF
- 2275 posts since 4 Dec, 2011 from Brasília, Brazil
I think that the multi-process model will still show his face somewhere soon..
http://devstreaming.apple.com/videos/ww ... s.pdf?dl=1
http://devstreaming.apple.com/videos/ww ... s.pdf?dl=1
My soundcloud: https://soundcloud.com/waltercruz
-
- KVRAF
- Topic Starter
- 3183 posts since 31 Dec, 2004 from People's Republic of Minnesota
Thanks for all that info, guys. Really appreciate it. 
I was under the impression that Apple consciously did it just to beef up security requirements. Those explanations make a lot of sense.
I was under the impression that Apple consciously did it just to beef up security requirements. Those explanations make a lot of sense.
-
- KVRAF
- 2275 posts since 4 Dec, 2011 from Brasília, Brazil
ah, just remembered that the 10.10.2 update broke z3ta and other cakewalk products.
My soundcloud: https://soundcloud.com/waltercruz
-
Aletheian-Alex Aletheian-Alex https://www.kvraudio.com/forum/memberlist.php?mode=viewprofile&u=175428
- KVRist
- 38 posts since 6 Mar, 2008 from Philadelphia
Very true. logicalhippo nailed it pretty well, I'll just add my $0.02Anyways, pretty hard to answer such an abstract question concretely, but there's my best shot. I think most of the time these breakages could have been prevented by a bit more diligent work up front by the developers.
I have not run into a 'broken' Apple mechanism without an alternative, but it is true that you have to stay on your toes more with Apple since they blast through changes pretty quickly, BUT much of what Apple 'breaks' had been warned about for a long time before the change. If you stay close to the metal, don't depend on sketchy 3rd party libraries, and pay attention to developer notes and warnings, then it is not *too* hard to stay mostly current and compatible.
BUT that all takes additional time, money, and effort; many devs prefer using higher-level 3rd party tools to cut down on time, but that adds another link of potential breakage in your build toolchain. Apple telegraph big changes far in advance typically, and then provide legacy-support mechanisms for several years afterward, so the problem is usually old build-chains with band-aids that finally give out, and slow-to-change 3rd party libs.
A common example would be: a dev wants to build cross-platform with only one codebase, so they turn to something like VSTGUI, which does not keep in step with Apple's changes, so if VSTGUI breaks, their plug-ins break. But it is a freebie: paid libs like JUCE are much better at staying current.
More specific to the actually OP question. The big game-changers that have caused headaches have been: carbon/C-to-cocoa/ObjC, PowerPC-to-Intel, and 32bit-to-64bit native, and killing off Interface Builder pallets. More recently, there are smaller things related to security like sandboxing, code-signing/gatekeeper, etc which can cause headaches for presets, registration mechanisms, loading binaries, etc. And then there are the 'here derp' issues like consolidating libraries, moving header locations, moving directories, etc, but those are not too difficult to deal with... just annoying.
The move to AudioUnit v3, and eventually swift and metal will inevitably cause headaches as well, but honestly I am not too worried as I have had far far fewer problems with AudioUnit/CoreAudio changes from Apple than I have with VST 2.0/2.4/3.0/3.5 changes from Steinberg.
-
Zaphod (giancarlo) Zaphod (giancarlo) https://www.kvraudio.com/forum/memberlist.php?mode=viewprofile&u=111268
- KVRAF
- 2596 posts since 23 Jun, 2006
programmers don't need an other language, an other framework, an other format
Deprecating continuously things don't work: if things will break, people will abandon the platform, sequencer, or format. Customers don't care if the programmer did not read very well specifications. Customers don't care the reason. Customers judge a product just from the user perspective: does it work, or not. Is reaper any better? Yes? Let's try it. Is Windows any better? Let's move. And so on.
And if things change too often, developers will look for cross platform frameworks, in order to maintain easily their code. When there is a single cross platform framework there is no gain any more, because products are really the same.
So everyone here is just wasting time; if someone will break things too often he will simply loose customers
It does not matter if it was announced an year before or not.
The myth of adding a killer feature does not fit with the market size and the typical developer team size.
Btw: when we were a smaller company we contributed to move a lot of customers to Windows because we were not able to handle all that mess. If things were different now those customers would probably use logic.
My 2 cents
Deprecating continuously things don't work: if things will break, people will abandon the platform, sequencer, or format. Customers don't care if the programmer did not read very well specifications. Customers don't care the reason. Customers judge a product just from the user perspective: does it work, or not. Is reaper any better? Yes? Let's try it. Is Windows any better? Let's move. And so on.
And if things change too often, developers will look for cross platform frameworks, in order to maintain easily their code. When there is a single cross platform framework there is no gain any more, because products are really the same.
So everyone here is just wasting time; if someone will break things too often he will simply loose customers
It does not matter if it was announced an year before or not.
The myth of adding a killer feature does not fit with the market size and the typical developer team size.
Btw: when we were a smaller company we contributed to move a lot of customers to Windows because we were not able to handle all that mess. If things were different now those customers would probably use logic.
My 2 cents
-
- KVRian
- 853 posts since 13 Mar, 2012
In short: because Apple gives a shit about maintaining backward compatibility.masterhiggins wrote:Sorry, not an audio developer, but I'm posing this question for audio developers. Why does every release of Mac OSs break plugins (primarily audio units)? I know the "break" is usually the result of validation problems. I guess I have a better, second question. Are the changes that Apple makes to their OS actually drastic enough to merit new validation requirements? Or are they arbitrary, unnecessary decisions that are made without concern for developers and/or users? Thanks.
Haven been working with Apple technologies for years now and it's ridiculous... it's always the same with every new iOS or OSX release - you have to spent to time to adapt your code to what has been changed, removed or re-designed, or it won't work with current iOS/OSX version anymore.
IMHO its the second one. Just to give an example: I have been working on iAP & CarPlay for quite some time now and there it is quite common that they change protocol specs and/or implementation details with every new iOS release. So basically they intentionally f**k you as a developer.. the nature of a CarPlay system is that is build into a Car. So what do you do if they decide to break the existing protocol? Calling all x1000 Cars back to update the firmware of the infotainment system? Just because Apple gives a shit? ahaha... -.-Are the changes that Apple makes to their OS actually drastic enough to merit new validation requirements? Or are they arbitrary, unnecessary decisions that are made without concern for developers and/or users? Thanks.
No they don't.Deprecating continuously things don't work: if things will break, people will abandon the platform, sequencer, or format.
Good marketing is way more powerful than any technical explanation.
If your plugin does not work it is YOUR fault from a customer perspective. They don't care about why it doesn't work. They see the shiny new marking campaign and the queue in-front of the Apple store and want to be part of the Apple lifestyle. So they buy it. If you plugin doesn't work with new system anymore, it is your fault and you need to fix it. So they will most likely drop your products if you don't update, rather than skipping the Apple device generation.
btw. and yeah.. I'm biased.. had to work on too much Apple shit for too long..
Last edited by PurpleSunray on Tue Oct 06, 2015 12:17 pm, edited 1 time in total.
~~ ॐ http://soundcloud.com/mfr ॐ ~~
-
- KVRist
- 155 posts since 24 Jul, 2012
No they don't.
Good marketing is way more powerful than any technical explanation.
If your plugin does not work it is YOUR fault from a customer perspective. They don't care about why it doesn't work. They see the shiny new marking campaign and the queue in-front of the Apple store and want to be part of the Apple lifestyle. So they buy it. If you plugin doesn't work with new system anymore, it is your fault and you need to fix it. So they will most likely drop your products if you don't update, rather than skipping the Apple device generation.
Skipped Apple after Lion release, and I'm very happy with this choice...four five friends (all audio professionals or musicians) did to...
Good marketing is way more powerful than any technical explanation.
If your plugin does not work it is YOUR fault from a customer perspective. They don't care about why it doesn't work. They see the shiny new marking campaign and the queue in-front of the Apple store and want to be part of the Apple lifestyle. So they buy it. If you plugin doesn't work with new system anymore, it is your fault and you need to fix it. So they will most likely drop your products if you don't update, rather than skipping the Apple device generation.
Skipped Apple after Lion release, and I'm very happy with this choice...four five friends (all audio professionals or musicians) did to...
-
- KVRian
- 853 posts since 13 Mar, 2012
I disagree.Zaphod (giancarlo) wrote:Customers judge a product just from the user perspective: does it work, or not. Is reaper any better? Yes? Let's try it. Is Windows any better? Let's move. And so on.
Pretty much all Apple users I know are 110% convinced about being on the best platform available. Apple stuff costs more than other, so it must be better.
They usually do not question their platform in a way like "my plugins to not work anymore, should I switch to windows?", but more like "my plugins to not work anymore, f**k that devs, going to buy my plugins from another company now".
Last edited by PurpleSunray on Tue Oct 06, 2015 12:52 pm, edited 1 time in total.
~~ ॐ http://soundcloud.com/mfr ॐ ~~
-
- KVRian
- 853 posts since 13 Mar, 2012
Than you probably fall into the tech-guy cornerChandran wrote: Skipped Apple after Lion release, and I'm very happy with this choice...four five friends (all audio professionals or musicians) did to...
To do the decisions to change to windows you need to understand why your plugins stop working with almost every new OSX release. Most users can't make that link - for them the plugin does not work. OSX has nothing to do with that, it works fine, just that damn plugin doesn't
Last edited by PurpleSunray on Tue Oct 06, 2015 12:20 pm, edited 1 time in total.
~~ ॐ http://soundcloud.com/mfr ॐ ~~
-
- KVRian
- 853 posts since 13 Mar, 2012
-
- KVRAF
- Topic Starter
- 3183 posts since 31 Dec, 2004 from People's Republic of Minnesota
If 95% of my plugins work perfectly and one dev's plugin doesn't, why would I question the platform? Doesn't it make more sense that it's dev's fault? That's my case with a certain dev. Actually, every Mac user I know has trouble with their plugins. Every other dev's plugins work perfectly.PurpleSunray wrote:I disagree.Zaphod (giancarlo) wrote:Customers judge a product just from the user perspective: does it work, or not. Is reaper any better? Yes? Let's try it. Is Windows any better? Let's move. And so on.
Pretty much all Apple users I know are 110% convinced about being on the best platform available. Apple stuff costs more than other, so it must be better.
They usually do not question their platform in a way like "my plugins to not work anymore, should I switch to windows?", but more like "my plugins to not work anymore, f**k that devs, going to buy my plugins from another company now".
