Catalina: Apple turns macOS into a closed platform; many audio-devs warned from the upgrade

[Markus Krause]

Technical facts:

Wow, a thread that attempts to start a rational discussion. With at least some reasonable responses (thanks Urs!). Great.


The Arstechnica Catalina review is a good starting point for some basic underlying information:

https://arstechnica.com/gadgets/2019/10 ... ca-review/


Some points:
- Apple started the transition from 32bit to 64bit in 2003.
- OpenGL was deprecated (not removed) in Mojave last year, but has been stuck on an old version since 2010 (Metal was introduced in 2014).
- kexts will be replaced by System Extensions and DriverKit in the future, the transition started with Catalina
- SwiftUI
- SIP changes
- new Firmlinks in APFS
- read only OS partition
- Notarisations

Thanks. I will add this info.
Can you please add some more details to the list so that i can simply copy & paste it to the first page? It would be nice if you could also add what positive/negatives it will have for developers/end-users. Please try to explain it as simple as possible, so that also a non-devs can understand it

[el-bo (formerly ebow)]

So far I know of no audio developer who's pulling out of the Apple platforms. I think I could leave this cumbersome discussion alone if...

Please don't. Your no-nonsense clarity and rationality is extremely important, in cases such as this

[parricide]

it's known that apple is for anyone who can afford it and none the less.

it is comparable to $1000 wines, or exclusive paintings

[el-bo (formerly ebow)]

it's known that apple is for anyone who can afford it and none the less.

It's relative, of course. And its important to make distinctions between price and value.

£4000 has bought me over 12-years-worth of Apple computing, including all paid updates from Logic Pro 7, until the current version (which has provided almost 6 years worth of free updates). My 7.5-year-old Macbook Pro can still run any software I can think of, save for games that need more than the internal V-ram, and is compatible with the latest iterations of LPX and MacOS, which puts paid to the normal claims of planned obsolescence.

So, at £333 per-year, I don't feel one would need to be particularly rich to maintain an Apple virtual-studio environment. And the value-proposition definitely seems to be there

[WordClock]

i own logic pro x also but i'm waiting for v11 for now and only made a little with it because I get fiddly with things like variated function. I have live 10 also and bought the suite because i like the custom sounds a lot. my favorite daw is reason but i've used cubase for simplicity purposes a lot just because i prefer wysiwig as they call it. it's all wysiwig with things like the nord modular g1 and g2 and reason so you can noodle really special.

i'm very disappointed with reason 11 in no way because 10 is so great but don't plan to expand to 11 unless they add midi out from vsti's to the rack. majority of the new thing/album i made on my youtube is reason plus vst technology instruments and a little effect unit usage but mostly reason effects. it's like a dystopian/industrial effects extravaganza that i don't mind sharing because i know i'll never meet a record exec or have much distribution because of legality issues.

my carreer could've been the ultimate but i warn anyone out there that i do know what's going on. the only reason i bought 5 daw's is because of my life's confusion and because i was able to grab studio one for less than 200 dollars and also live 10 for less than 300 dollars. reason is so so so recommended in spite of the knocking of it. it is indeed the PENULTIMATE daw if you know how to use it or don't. price/feature/ease of use based.

[stratology]

Can you please add some more details to the list so that i can simply copy & paste it to the first page?

OK, quotes and links from the Arstechnica review:


Full review:
https://arstechnica.com/gadgets/2019/10 ... ca-review/



32bit transition
https://arstechnica.com/gadgets/2019/10 ... view/3/#h1

Apple was careful not to surprise anyone with Catalina's removal of 32-bit support, and the company has been projecting its plans for over two years now. Most developers took the hint, and even software like Audacity and Zoom that was still 32-bit as I wrote the Mojave review last year is 64-bit now—though larger, more complicated apps might still be hung up on old 32-bit dependencies that need to be either updated or replaced. 32-bit apps will remain installed on your Mac, but their icons will be crossed out.
..
The impact of the 32-bit decision will be felt disproportionately. Regular users are unlikely to notice or care—or, at least, it will be reasonably easy to find replacement or updated versions of older apps that support 64-bit. Creative professionals, on the other hand—people who work with images, video, or audio, or folks who rely on expensive (and not always new) external equipment may run into problems with apps, or plugins, or drivers. Community-maintained open-source software is likely to lag behind software maintained by large companies. If you're holding on to older versions of these pro software packages before the whole professional software industry shifted to yearly subscriptions with rolling updates, those apps may not have 64-bit compatibility and certainly won't have it added in an update. And while I don't think macOS is the gaming platform of choice for many people, tons of older games (and, as of this writing, the regular Steam client) are 32-bit only and are extremely unlikely to be updated.

OpenGL
https://arstechnica.com/gadgets/2019/10 ... view/3/#h2

OpenGL and OpenCL were officially deprecated in Mojave last year, though that's a little misleading since it implies that Apple had been actively maintaining and updating its support for those standards. In Catalina, as in every macOS version going all the way back to Mavericks, the macOS OpenGL implementation is stuck at version 4.1 (2010), and the OpenCL version is stuck at 1.2 (2011). This means that apps that still rely on those APIs on macOS will continue to run, provided they've been updated to meet the 64-bit-only requirement. But you shouldn't be developing new Mac apps that rely on OpenGL or CL for anything important.

Apple also isn't directly supporting newer replacement standards like Vulkan. Instead, the company is opting to direct developers toward its own proprietary Metal API for both graphics and GPU computing. But that doesn't mean there's no hope for cross-platform developers—the MoltenVK translation layer, which maps Vulkan API calls to Metal ones, is actively being used in several prominent apps, and the results are promising. In both Dota 2 and the Dolphin GameCube and Wii emulator, MoltenVK often performs significantly better than OpenGL while saving developers the trouble of adding and maintaining Metal support. It's not totally without bugs (and as those Dota 2 benchmarks from Phoronix show, still not quite as fast as Vulkan running on Linux or Windows 10), but MoltenVK is still a big improvement over OpenGL.

Python, Pearl, Ruby
https://arstechnica.com/gadgets/2019/10 ... view/3/#h4

For years now, Apple has shipped runtimes for the Python, Perl, and Ruby programming languages to allow users to run scripts written in those languages without performing any additional installs. But according to Apple's release notes for developers, "future versions" of the operating system will stop including those runtimes by default (though they may still be able to download the runtimes on-demand, just as older versions of macOS could download the Rosetta PowerPC compatibility layer or Java when needed). Apple recommends that developers whose software relies on one or more of these runtimes ship their own runtimes alongside their apps.

Anyone who relies on the built-in versions Python, Perl, or Ruby can breeze through installation using Homebrew or other methods, and the versions that Apple ships usually sometimes trail the current stable releases by several versions. If you do choose to use the built-in versions, Catalina does include Python 3 by default, where older versions of the operating system only included Python 2.7 (still included, and still the default environment if you just run "python" from the Terminal, but Apple warns against using it).

Catalyst, SwiftUI
https://arstechnica.com/gadgets/2019/10 ... view/6/#h1
..also has a section with comments from developers

SwiftUI isn’t a complete replacement for AppKit or UIKit (or the tvOS version of UIKit, or the Apple Watch’s WatchKit). But it will give developers a way to write code one time and have that code displayed differently in different operating systems based on what looks and works best on each. A chunk of code that generates a drop-down menu or a radio bubble on macOS might generate a scrollable picker in iOS or iPadOS; a list of options in an iOS app might be transformed into a strip of Touch Bar controls on a Mac. With SwiftUI, developers can write platform-specific code on top of that connective tissue, optimizing their apps for each operating system’s strengths and weaknesses.

Sign In With Apple (single sign on)
https://arstechnica.com/gadgets/2019/10 ... view/9/#h2

Apple's new single-sign on (SSO) service isn't exclusive to Catalina or macOS or even Safari. Instead, it works fine on other versions of the operating system, in other browsers, and in other operating systems entirely. It's a bigger deal in iOS, where developers are generally required to implement it if they support any other SSO options in their apps.

The one feature that does appear to require Safari 13 running on macOS Catalina can use your Mac's TouchID sensor to authenticate when you use Sign in with Apple. That means you don't need to type in your password as you do on other browsers and platforms. This doesn't work in older macOS versions running Safari 13, and it is not an option on Catalina Macs without a TouchID sensor (a connected Apple Watch can usually be used to authenticate anywhere in macOS where TouchID is supported, but that isn't the case here).

System Extensions and DriverKit
https://arstechnica.com/gadgets/2019/10 ... iew/10/#h1
..the whole section is worth reading...

..
But because the kernel is such important software, kexts can cause big problems. A bad kext can cause a kernel panic, bringing your whole system crashing down. The kernel has relatively unfettered access to your hardware and to other software running on your system, so a malicious or buggy one can open you up to security vulnerabilities. And from a development perspective, writing kexts can be difficult and time-consuming, since the whole system needs to be rebooted when you have a problem and there’s not much room for error.

Apple’s goal with both System Extensions and DriverKit is to retain the ability to extend macOS’ capabilities—a key point of differentiation between macOS and iOS—while bringing that software out of the kernel and into user space where it can cause fewer problems. A bug can crash the extension but not the whole system; the System Extensions will be subject to the same security policies as other applications; and System Extensions don’t have the same level of unprotected access to the kernel, your hardware, and your apps as kernel extensions do.
..
Apple has defined three different kinds of System Extensions for Catalina: Network Extensions, Endpoint Security Extensions, and Driver Extensions
..
DriverKit: Fairly limited, for now
Apple eventually wants DriverKit to be the only way anyone writes device drivers for the Mac, totally replacing the venerable I/O Kit. Apple describes DriverKit as an “updated and modernized” version of I/O Kit, but at least in Catalina, it’s considerably more limited.
..
Kexts will continue to work as-is for now, but Apple ultimately wants all macOS drivers to make the jump to DriverKit.
..
The future of kexts
Catalina will begin the process of deprecating kexts entirely, but as it did when it dropped support for 32-bit software, Apple is trying to give developers an off-ramp, and the deprecation timeline may look different for different kinds of kexts.

Catalina’s read-only system volume
https://arstechnica.com/gadgets/2019/10 ... iew/11/#h1

System Integrity Protection (SIP), a feature added in macOS 10.11 El Capitan that disables write access for the /System, /bin, /usr/ and /sbin system folders by default. Catalina takes this a step further, creating a separate read-only APFS volume exclusively for system files—including not just the SIP-protected folders, but all system files and most of the preinstalled apps. A new Catalina install with a single user account has around 10GB of data on its system volume and another 4 or 5GB in the user data volume.

If you’re the kind of person who prefers or needs to disable SIP to work with system files and folders, you can still do that, but Catalina throws up another barrier. While SIP’s protections could be disabled permanently from your Mac’s recovery partition, disabling read-only access to the Catalina system partition is reset every time you reboot. SIP and the read-only system volume are related features, but they’re not the same—to access system files, you need to disable SIP from the recovery partition, and then you can mount the system volume as read/write from within Catalina.
..
..your current system volume is marked as a data volume, and system files are deleted, leaving your data intact. From there, a new system volume is created in your APFS container, and the majority of the Catalina system files are written there instead. That new volume is marked as read-only as soon as the installation is complete.
..
To the typical Catalina user who doesn’t notice or care about filesystems or partition maps, they won’t notice a difference.
..
And many of Apple’s changes to APFS in Catalina are done in service of hiding this volume split. Your Catalina system and data volumes aren’t quite the same as two typical APFS volumes—they’re part of something new called a Volume Group, which helps the operating system treat the two volumes in some of the same ways that Mojave and previous versions of macOS treat the unified system and data volume. Not only do the two volumes appear as one in the Finder, but they share things like FileVault encryption keys—the two volumes are encrypted (or decrypted) simultaneously and the same key unlocks them both.
APFS in Catalina supports something called “firmlinks,” as a way to maintain macOS’ existing directory structure despite everything now being spread across two volumes. Think of an alias in macOS or a shortcut in Windows—these are forms of symbolic links (or symlinks), small files that can be stored anywhere and which point you toward another file on another volume, partition, or disk. But where symlinks only go one way, firmlinks are bidirectional. Folders can appear in multiple places, but from the Finder’s perspective and the user’s perspective, neither directory is the “real” one and neither is the “shortcut.”
..
This is where firmlinks come in. Look at the contents of either /Applications or /System/Volumes/Data/Applications, and you’ll see that the only applications actually stored in that folder are the ones you’ve installed yourself, plus Safari (presumably to make the browser easier to update on its own without a major system update). But the Finder still shows the user their installed applications plus all of the regular macOS system apps in the same folder.
According to Apple, firmlinks only exist for directories, not individual files, and they only work within established volume groups like the one Catalina creates when you install it on your system.

Notarisations
https://arstechnica.com/gadgets/2019/10 ... iew/11/#h4

..
This feature was introduced as an optional extra step in Mojave.
..
The notarization process is a bit more involved. It requires developers who want to distribute outside the Mac App Store to submit apps to Apple for review by its Notary Service—this isn’t the same as the actual content review process used to allow apps into the Mac App Store but a shorter and more automated process that should only take a few minutes. The Notary Service checks to see whether the app contains malware; whether it uses the enhanced System Integrity Protection runtime from Mojave that protects running apps from being tampered with; and whether the apps and all their components are properly signed in the first place.

Notarization, like Developer ID, isn’t strictly necessary—non-notarized apps will still run on macOS, one crucial point on which macOS continues to differ from iOS and iPadOS. But to run (at least the first time) without triggering Gatekeeper and scary security warnings, all apps running on Catalina must be notarized. And starting in January of 2020, apps will need to meet new Catalina-specific notarization requirements (these were originally supposed to go into effect when the operating system was released, but Apple relaxed them just a bit to give devs more time).
..
notification fatigue is real

Bash > Z Shell
https://arstechnica.com/gadgets/2019/10 ... iew/12/#h9

Bash has been replaced by Z Shell, or zsh, as the default shell in the Terminal.

That's if from the Ars review.




From another source (targeted at sysadmins, but with some useful background):
Notarization on macOS Catalina and IT auditing
https://derflounder.wordpress.com/2019/ ... -auditing/

[Markus Krause]

Thanks for the detailed info. I have added it to the first post. I left out the paragraphs with python, bash, catalyst and sign in with apple, because it should not effect audio plugins. Please correct me if i should be wrong

[MacGyver]

my post was offtopic.

[stratology]

Thanks for the detailed info. I have added it to the first post. I left out the paragraphs with python, bash, catalyst and sign in with apple, because it should not effect audio plugins. Please correct me if i should be wrong

I left in Catalyst because of SwiftUI, which may be relevant to devs.

I thought Single Sign On (similar to 'sign in with Facehole' or 'sign in with Google' that some sites use) might be relevant for some websites of plugin manufacturers, where you can sign in to view and download your purchases.

But these are sideshows, I do agree with your decisions.

[stratology]

Maybe I'm being naive here, but my assumption has always been that developers who cannot attend WWDC will watch all relevant videos a week or two afterwards...

If you haven't:

All WWDC 2019 videos are here:
https://developer.apple.com/videos/wwdc2019/


Notarisations
https://developer.apple.com/videos/play/wwdc2019/703/

Advances in macOS Security
https://developer.apple.com/videos/play/wwdc2019/701/

System Extensions and DriverKit
https://developer.apple.com/videos/play/wwdc2019/702/

What's New in APFS
https://developer.apple.com/videos/play/wwdc2019/710/

Metal for Pro Apps
https://developer.apple.com/videos/play/wwdc2019/608/

AUv3 Extensions User Presets
https://developer.apple.com/videos/play/wwdc2019/509/

Modernizing your Audio App
https://developer.apple.com/videos/play/wwdc2019/508/

What's new in AVAudioEngine
https://developer.apple.com/videos/play/wwdc2019/510/


These are, obviously, rather technical, as devs are the target audience.

[andy-cytomic]

I have to pay $99 per year for a code signing certificate for Windows. Paying $99 to Apple for the Dev program gets me all the certificates I need for a year. If Apple insist upon having to sell through their App Store this is news to me. Signing binaries has been going on for a long time, hopefully with increased requirements it will actually help in terms of copy protection, instead of not really doing much in the past apart from giving end customers assurance the program is from a certain source.

[Urs]

If Apple insist upon having to sell through their App Store this is news to me.

They don't, but the speculation makes for great headlines.

[Angus_FX]

The App Store isn't going to be forced on anyone, but notarization nevertheless represents a major transfer of power: it gives Apple a right-of-approval (without any transparency, appeal process or independent regulation or arbitration) on any software that will run on the platform. How they exercise that right in practice is TBD. Right now, it's pretty lenient, but there's nothing to stop them tightening it up in future.

As copy protection this stuff is of rather limited use, because the checks aren't run every time (presumably because the performance cost is too high). It's still very easy for h/crackers to bypass.

The bigger concern from my point of view is the Entitlements/Intents security model and its poor compatibility with a DAW-and-plugins type of architecture.

The model exists for a good reason (which Urs has nicely summarised above, in reference to Cambridge Analytica and other such evildoers). The old UNIX-y security model ("Each user is wholly responsible for their stuff; system security exists to protect users from each other and itself") doesn't hold up so well in a world where user-data is almost more valuable to bad actors than anything else they might obtain by compromising root access to a system.

But there's a problem: Catalina's approach works at application level, that is, an application has to ask for the permissions it needs and remembers them. Yet when you have an application whose capabilities are built and extended dynamically by loading plug-ins, there is no way at the start to know what extra capabilities and permissions might be needed, and there's not really any way for plug-ins to inform the host, ahead of time, that they're going to need them. It's just a bad fit with the way pro music apps are built. And the new Security Bookmarks for filesystem access, which again are built with good intentions (to make sure the user wanted the system to be able to access file "X") privileges the system file control above and beyond any special-purpose filebrowser that plug-ins or DAWs might want to implement. The changes are well-intended certainly, but the side-effects for this world are much larger than might be expected.

[Urs]

I guess it would make sense to extend plug-in standards to deal with such issues. Plug-Ins have always been second-class citizen. For instance, it was (is?) pretty much impossible to simply use OpenMP for parallelisation. There was always the lingering danger and implications of sandboxing and the resulting privileges for file access and shared memory. There have always been possible issues with conflicting libraries and namespace collisions.

I'm think that maybe the approach of application based extensions is something worth embracing. Haven't dealt with it yet, though.

[FabienTDR]

I'm considering to impose subscription only licenses "exclusively" (sounds better) for mac users. While offering perpetual licenses only for windows. I think that's reasonable.

(personally, I'd prefer dropping Mac altogether, but it's still 30% of my audience)

The risk for any mac developer offering perpetual licenses is dangerously high. We'll likely stop. Changing licensing for mac only is a most pragmatic answer to it, and obviously mac users don't care too much about additional costs and paperwork anyway, according to Apple: just matter of the right wording ( ).

Ships sailing on the Apple sea predicable need more repairs. This has a price.