I’ve pushed a basic remote macOS filesystem project to GitHub. Its got all the bits and pieces necessary to work but I haven’t audited all parts of it for correctness. I’ll do that as I start implementing each area. What’s complete is all the project infrastructure to do filesystem and kernel development. I can’t stress enough the importance of taking care of your development environment to get the code, compile, deploy/test, debug cycle working as fast as possible. I can go from a fresh checkout to compiled, deployed, unit tested (yes you can unit test in the kernel), and remote filesystem mounted in under a minute. There’s a lot going on to make the above happen as you will see.
What’s In The Box
To get a working macOS filesystem you need three things: filesystem, kernel extension and NetFS plugin. Now the last one isn’t technically necessary but if you’re being a good steward of the platform you should include it.
Filesystem
This is what lives in the /Library/Filesystems folder. You can see the shipped filesystems by looking at /System/Library/Filesystems. For the most part there isn’t much code here. What you’ll find is the very important Info.plist describing the filesystem and
associated utilities such as mount, fsck, newfs, etc. Since Lustre is a remote filesystem we just have mount.
Extension
The core of how a filesystem is implemented in it’s kernel extension. This lives in the /Library/Extensions folder and is structured like any other kernel extension. There’s one important difference from most other kernel extensions though. In the Info.plist there’s this:
<key>OSBundleAllowUserLoad</key><true/>
If you plan to have a filesystem which may be mounted by normal (non-Admin) users, they need to be able to load the associated kernel extension. This key allows that to happen.
NetFS Plugin
If you’ve ever used ‘Connect To Server….’ from Finder, you’ve used a NetFS plugin. There isn’t much, if anything, documented about these. The best you can do is look at Apple’s implementation. Getting the Info.plist correct is the secret. The bundle lives in /Library/Filesystems/NetFSPlugins.
The Walking Tour
Schemes & Targets
To see how the aforementioned build products are made, let’s take a tour of the Lustre Xcode project. The
best place to start is with the schemes. There shouldn’t be too much surprised here. There’s a scheme for the filesystem, extension and NetFS plugin. Additionally there’s a scheme for a development helper target (Utility) and a scheme to build/deploy/test everything (All).
On the target side of things there’s a bit more. Here we see mount which is built as part of the Filesystem scheme. We also see two testing targets: Extension Test and Test Runner. They perform the in kernel unit testing and will be the source of a later blog post.
It’s important to note that some schemes have pre and post actions on them as shown below. 
You can see them by selecting ‘Edit Scheme` from the scheme drop down. I’ve chosen to put them in the scheme rather than as a build phase for two reasons. The first is the resulting generated files are used by multiple targets. The second is to make sure the generated files are actually recompiled. A Run Phase build step won’t necessarily do that. I think that’s an Xcode bug though it’s been that way for sometime.
While we’re talking about actions in schemes there’s one other thing to point out. In the archive task of the All scheme there’s a post-action that builds the installer. I won’t go into details as that will be for another blog post. Just know that if you have the All scheme selected and do an archive, you’ll also get a Lustre.pkg in the ${ARCHIVE_PATH}/Package folder.
Project Structure
Here’s how I structure my projects though you may prefer differently. That’s fine. I don’t think there’s just one way to do it. It just needs to make sense.
Starting off, I have a Common group which holds headers containing structures and constants used by multiple targets in the system. A good example is the mount args structure which needs to be shared between the extension and the mount utility.
All the code and support files for the filesystem and extension are in the Filesystem group. I don’t have many subgroups here yet but that will change as I start adding more and more code.
The NetFS group has the Finder NetFS plugin code and support files. There isn’t much there as it uses a class from the mount target for most of the work. The plugin needs some more work but there’s enough there to work.
Inside the Tools group I have code for the mount command. If this was a local filesystem I would also have code here for fsck, newfs and so on. If I have additional command line utilities that I would ship, they would go in here too.
The Test group is split into three parts, one for each target being tested. The Filesystem subgroup actually gets compiled in a separate kernel extension that’s dependent on the filesystem extension. I prefer this approach as it avoids accidentally shipping test code in your filesystem extension. The Filesystem Test Runner subgroup is a standard XCTest unit testing bundle with the Utility target as the test host. It has special XCTestCase subclass to do unit testing of kernel extensions. The NetFS subgroup has just one test case right now to test mount url parsing.
As mentioned above the archive task in the All scheme creates an installer package. The necessary files for that are located in the the Packages group. The installer is designed with three sub-components (Filesystem, Extension, and NetFS plugin). The whole process is controlled by a shell script called package.sh located here.
Any assets are located in the Resources group and any third party code or libraries are in the Vendor group.
Build and Deploy
I’ve said it before and I’ll say it again and again and again. Getting the cycle time down for coding, compiling, deploying, debugging is critical for a successful kernel extension. If this cycle takes too long, you start to dread making changes. There’s no way you’re going to just ‘try something’ for fun. As a result, technical debt starts increasing rapidly and it’s just a downward spiral from there. Not with this project though. Want to compile, it’s Cmd-B. Want to deploy, it’s Cmd-R. Want to test, it’s Cmd-T.
The secret sauce to all this is in the Utility target and should probably be renamed Awesomeness. The Utility target builds a Cocoa application called Lustre Utility. The Lustre Utility is really three applications in one and looking at it’s main.m shows this:
Lustre Utility is either a daemon, deployer or tester depending on the command line argument. For this blog post we’ll discuss the first two.
Unless you really enjoy pain, you need to do kernel development on another machine be it another physical machine or a virtual machine. I prefer the later but never ship without testing on the former. The necessary step for using virtual machine are:
- Start VMWare Fusion
- Start virtual machines
- Select correct snapshot
- Build project
- Copy over the filesystem, extension and NetFS plugin
- Load the extension
- Start testing/debugging
That’s a lot of work to do each time you want to try out some new code. Fortunately they can all be automated. VMWare provides an API called VIX to work with virtual machines. The Lustre Utility uses this API to start up virtual machines and select the right snapshot (selecting a snapshot is faster than rebooting a virtual machine to a clean state). The VIX API use to allow for file copying but recent macOS security changes result in this not working. To remedy this Lustre Utility is run in daemon mode on the virtual machine. As a daemon, it runs a distributed objects server with the following protocol:
Now distributed objects is a very old and not recommended framework. It’s absolutely simple though and perfect for something like this. With just a few lines of code you can get a very nice client/server setup going.
Back in the Xcode project when Cmd-R is hit, the Lustre Utility is started with the deploy option. This causes either a virtual machine to start up or connect to a physical machine. Once distributed object communication is established, files are copied over, the extension is loaded and the deploy running Lustre Utility exits.
Conclusion
A solid development environment where you can go from writing code to deploying and debugging quickly is key. Filesystem and kernel extension development is no different. With a some additional upfront work it’s quite doable as I’ve shown. If you’re interested in the details, please take a look at the source in GitHub.