Apple File System ================= The Apple File System (APFS) is the copy-on-write filesystem currently used on all Apple devices. This module provides a degree of experimental support on Linux. It's supposed to work with a range of kernel versions starting at 4.9 or before, but only a few of those have actually been tested. If you run into any problem, please send a report to <[email protected]>. To help test write support, a set of userland tools is also under development. The git tree can be retrieved from <git://github.com/eafer/apfsprogs.git>. Known limitations ================= This module is the result of reverse engineering and testing has been limited. You should expect data corruption. Please report any issues that you find. Apple has released other versions of the filesystem to the public before the current one. I would not expect them to be compatible with this module at all, but I am open to fixing that if requested. Many features are not yet supported: o Encryption. o Compression, though the compressed contents of a file can be read from the 'com.apple.decmpfs' and 'com.apple.ResourceFork' xattrs as long as they are under 64k. o Restoring to a snapshot. o Access control lists. This is not a priority. Build ===== In order to build a module out-of-tree, you will first need the Linux kernel headers. On Debian, you can get them by running (as root): apt-get install linux-headers-$(uname -r) Now you can just cd to the linux-apfs-oot directory and run make The resulting module is the apfs.ko file. Before you can use it you must insert it into the kernel, as well as its dependencies. Again as root: modprobe libcrc32c insmod apfs.ko Mount ===== Like all filesystems, apfs is mounted with mount [-o options] device dir where 'device' is the path to your device file or filesystem image, and 'dir' is the mount point. The following options are accepted: vol=n Volume number to mount. The default is volume 0. uid=n, gid=n Override on-disk inode ownership data with given uid/gid. cknodes Verify the checksum on all metadata nodes. Right now this has a severe performance cost, so it's not recommended. readwrite Enable the experimental write support. This WILL corrupt the container. So for instance, if you want to mount volume number 2, and you want the metadata to be checked, you should run (as root): mount -o cknodes,vol=2 device dir To unmount it, run umount dir Credits ======= Originally written by Ernesto A. Fernández <[email protected]>, with several contributions from Gabriel Krisman Bertazi <[email protected]>, Arnaud Ferraris <[email protected]> and Stan Skowronek <[email protected]>. For attribution details see the historical git tree at <git://github.com/eafer/linux-apfs.git>. Work was first based on reverse engineering done by others [1][2], and later on the (very incomplete) official specification [3]. Some parts of the code imitate the ext2 module, and to a lesser degree xfs, udf, gfs2 and hfsplus. [1] Hansen, K.H., Toolan, F., Decoding the APFS file system, Digital Investigation (2017), http://dx.doi.org/10.1016/j.diin.2017.07.003 [2] https://github.com/sgan81/apfs-fuse [3] https://developer.apple.com/support/apple-file-system/Apple-File-System-Reference.pdf
APFS module for linux, with experimental write support (out-of-tree repository)
Overview
Issues
-
Does not compile on Linux 4.14.237 and lower
While preparing to add this kernel module to NixOS, I tried to build it for all kernels in Nixpkgs and it failed for three kernel versions. The README states, that this is supposed to work with kernel version 4.9 or before but it fails to build on 4.4.273, 4.9.273 and 4.14.237. The error message for Linux 4.14.237 boils down to
/build/source/dir.c:652:2: error: implicit declaration of function 'discard_new_inode' 652 | discard_new_inode(inode); | ^~~~~~~~~~~~~~~~~
and the error message for Linux 4.4.273 and 4.9.273:
23 | SB_RDONLY = MS_RDONLY; | ^~~~~~~~~ /build/source/apfs.h:23:2: error: type defaults to 'int' in declaration of 'SB_RDONLY'
I don't really care about those kernels. You should decide if you want to fix this or not as I can of course just exclude building this module with those old kernels in Nixpkgs. Please either fix the compilation or update the README. As always, I'm happy to try to compile any patches you may develop.
-
Does not compile on Linux 5.14.1
The latest commit from the develop branch does not compile on Linux 5.14.1. Here is the relevant build log:
/build/source/inode.c: In function 'apfs_do_ioc_setflags': /build/source/inode.c:1807:8: error: implicit declaration of function 'vfs_ioc_setflags_prepare' [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wimplicit-function-declaration-Werror=implicit-function-declaration8;;] 1807 | err = vfs_ioc_setflags_prepare(inode, oldflags, newflags); | ^~~~~~~~~~~~~~~~~~~~~~~~ /build/source/inode.c: In function 'apfs_ioc_setflags': /build/source/inode.c:1844:30: error: passing argument 1 of 'inode_owner_or_capable' from incompatible pointer type [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wincompatible-pointer-types-Werror=incompatible-pointer-types8;;] 1844 | if (!inode_owner_or_capable(inode)) | ^~~~~ | | | struct inode * In file included from /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/buffer_head.h:12, from /build/source/inode.c:7: /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/fs.h:1888:52: note: expected 'struct user_namespace *' but argument is of type 'struct inode *' 1888 | bool inode_owner_or_capable(struct user_namespace *mnt_userns, | ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~ /build/source/inode.c:1844:7: error: too few arguments to function 'inode_owner_or_capable' 1844 | if (!inode_owner_or_capable(inode)) | ^~~~~~~~~~~~~~~~~~~~~~ In file included from /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/buffer_head.h:12, from /build/source/inode.c:7: /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/fs.h:1888:6: note: declared here 1888 | bool inode_owner_or_capable(struct user_namespace *mnt_userns, | ^~~~~~~~~~~~~~~~~~~~~~ cc1: some warnings being treated as errors
Let me know if I can provide any additional information.
-
Fails to compile on Linux 5.12.11
I'm trying to build this on NixOS with Linux 5.12.11 but the build fails. I don't think this issue has to do with NixOS as it compiled fine with an older kernel (5.11.x). Here is the build output from
nix-build
:this derivation will be built: /nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv building '/nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv'... unpacking sources unpacking source archive /nix/store/0br698lgx8dm5bss6n25ls4llfv2q0zj-source source root is source setting SOURCE_DATE_EPOCH to timestamp 1624192656 of file source/xfield.c patching sources applying patch /nix/store/20r89d87xwycl20cfcsv4d884qlcp0gq-9.patch patching file Makefile configuring no configure script, doing nothing building build flags: SHELL=/nix/store/kxj6cblcsd1qcbbxlmbswwrn89zcmgd6-bash-4.4-p23/bin/bash KERNELRELEASE=5.12.11 KERNEL_DIR=/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build INSTALL_MOD_PATH=\$\(out\) make -C /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build M=/build/source make[1]: Entering directory '/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build' CC [M] /build/source/btree.o CC [M] /build/source/compress.o CC [M] /build/source/dir.o CC [M] /build/source/extents.o CC [M] /build/source/file.o CC [M] /build/source/inode.o CC [M] /build/source/key.o CC [M] /build/source/message.o CC [M] /build/source/namei.o CC [M] /build/source/node.o CC [M] /build/source/object.o CC [M] /build/source/spaceman.o CC [M] /build/source/super.o /build/source/super.c: In function 'apfs_attach_nxi': /build/source/super.c:1128:9: error: too few arguments to function 'lookup_bdev' 1128 | bdev = lookup_bdev(dev_name); | ^~~~~~~~~~~ In file included from /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/backing-dev.h:15, from /build/source/super.c:6: /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/blkdev.h:1984:5: note: declared here 1984 | int lookup_bdev(const char *pathname, dev_t *dev); | ^~~~~~~~~~~ make[3]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/scripts/Makefile.build:271: /build/source/super.o] Error 1 make[2]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:1856: /build/source] Error 2 make[1]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:215: __sub-make] Error 2 make[1]: Leaving directory '/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build' make: *** [Makefile:16: default] Error 2 error: builder for '/nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv' failed with exit code 2; last 10 log lines: > In file included from /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/backing-dev.h:15, > from /build/source/super.c:6: > /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/blkdev.h:1984:5: note: declared here > 1984 | int lookup_bdev(const char *pathname, dev_t *dev); > | ^~~~~~~~~~~ > make[3]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/scripts/Makefile.build:271: /build/source/super.o] Error 1 > make[2]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:1856: /build/source] Error 2 > make[1]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:215: __sub-make] Error 2 > make[1]: Leaving directory '/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build' > make: *** [Makefile:16: default] Error 2 For full logs, run 'nix log /nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv'.
This looks like something has changed in the kernel to which linux-apfs needs to be adapted to.
-
Clarify license
Source files currently contain headers with
SPDX-License-Identifier: GPL-2.0
. However, as per https://spdx.org/licenses/, there is no such license identifier, I am assuming it was meant to be "GPL-2.0-only" or "GPL-2.0-or-later". Please clarify which version is used by this project. -
Feedback for NixOS test
The last missing piece for perfect NixOS integration is a NixOS module. This will allow awesome things like specifying
fileSystems."/mnt/apfs" = { device = "/dev/disk/by-partlabel/apfs"; fsType = "apfs"; }
in the configuration, which is all that is required to automatically mount the filesystem during boot, no need to install anything or configure anything else. As part of writing a NixOS module, it is good practice to write a test for a basic sanity check and to gain some confidence, that at least the basic stuff all works as expected. Here is how the test looks currently:
import ./make-test-python.nix ({ pkgs, ... }: { name = "apfs"; meta.maintainers = with pkgs.lib.maintainers; [ Luflosi ]; machine = { pkgs, ... }: { virtualisation.emptyDiskImages = [ 1024 ]; boot.supportedFilesystems = [ "apfs" ]; }; testScript = '' machine.wait_for_unit("basic.target") machine.succeed("mkdir /tmp/mnt") with subtest("mkapfs refuses to work with a label that is too long"): machine.fail( "mkapfs -L '000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F' /dev/vdb") with subtest("mkapfs works with the maximum label length"): machine.succeed("mkapfs -L '000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7' /dev/vdb") with subtest("Enable case sensitivity and normalization sensitivity"): machine.succeed( "mkapfs -s -z /dev/vdb", # Triggers a bug, see https://github.com/linux-apfs/linux-apfs-rw/issues/15 # "mount -o cknodes,readwrite /dev/vdb /tmp/mnt", "mount -o readwrite /dev/vdb /tmp/mnt", "echo 'Hello World 1' > /tmp/mnt/test.txt", "[ ! -f /tmp/mnt/TeSt.TxT ] || false", # Test case sensitivity "echo 'Hello World 1' | diff - /tmp/mnt/test.txt", "echo 'Hello World 2' > /tmp/mnt/\u0061\u0301.txt", "echo 'Hello World 2' | diff - /tmp/mnt/\u0061\u0301.txt", "[ ! -f /tmp/mnt/\u00e1.txt ] || false", # Test Unicode normalization sensitivity "umount /tmp/mnt", "apfsck /dev/vdb", ) with subtest("Disable case sensitivity and normalization sensitivity"): machine.succeed( "mkapfs /dev/vdb", "mount -o readwrite /dev/vdb /tmp/mnt", "echo 'Hello World Hello World Hello World' > /tmp/mnt/Test.txt", "echo -n 'Hello World' > /tmp/mnt/test.txt", "echo ' 1' >> /tmp/mnt/TEST.TXT", "umount /tmp/mnt", "apfsck /dev/vdb", "mount -o readwrite /dev/vdb /tmp/mnt", "echo 'Hello World 1' | diff - /tmp/mnt/TeSt.TxT", # Test case insensitivity "echo 'Hello World 2' > /tmp/mnt/\u0061\u0301.txt", "echo 'Hello World 2' | diff - /tmp/mnt/\u0061\u0301.txt", "echo 'Hello World 2' | diff - /tmp/mnt/\u00e1.txt", # Test Unicode normalization "umount /tmp/mnt", "apfsck /dev/vdb", ) ''; })
Most of the space in the file is taken up by a Python script. The strings passed to
machine.succeed()
are shell script one-liners that all need to succeed for the test to succeed. The test builds a qemu VM with an empty 1GB disk image, loads the APFS kernel module and then executes the tests. Here is the NixOS test for ZFS. It basically tests smb file sharing, encryption and forefully importing a pool on a machine with a different host ID, for two versions of ZFS and the Linux kernel. I'd like to hear your feedback, if you think this is an adequate amount of testing or if you would test more or test less? Should I maybe reboot the VM somewhere during the test? Maybe test long filenames? Callapfsck
more often? Create a whole bunch of files and see if any disappear? Something else entirely? -
/lib/modules/5.13.9-arch1-1/build: No such file or directory. Stop.
I am on Arch, so I simply downloaded
linux-headers
from pacman, cloned the repo,cd
'd into the repo, ranmake
and got this error:make -C /lib/modules/5.13.9-arch1-1/build M=/home/nate/linux-apfs-rw make[1]: *** /lib/modules/5.13.9-arch1-1/build: No such file or directory. Stop. make: *** [Makefile:15: default] Error 2
Did I miss something?
EDIT: in
/lib/modules
I have a folder called5.13.12-arch1-1
... Do I need that specific version for this to work? -
Label does not seem to work
The label of an APFS filesystem does not seem to show up anywhere. The steps I tried:
modprobe apfs
truncate -s 1G /tmp/test.img
losetup -f /tmp/test.img
mkapfs -L testing /dev/loop0
lsblk -f
does not show a label for loop0ls -la /dev/disk/by-label
does not show loop0- mounting the filesystem does not change the situation
I don't think this is a problem with mkapfs because an APFS filesystem created with macOS also does not show a label in Linux and copying /tmp/test.img to a macOS machine and mounting it does show the label.
This issue is probably very low on your TODO list. I just wanted to know if it is a known issue and if I did something wrong or if this is supposed to work.
-
apfs: fix failed compilation due to idmap mount changes in v5.12
Since kernel commit 549c7297717c ("fs: make helpers idmap mount aware") extended the exisiting inode method to let filesystems be aware of idmapped mounts, compiling apfs module with linux v5.12 failed:
make make -C /lib/modules/5.12.0-custom/build M=/root/rw make[1]: Entering directory '/root/linux' CC [M] /root/linux-apfs-rw/dir.o /root/linux-apfs-rw/dir.c:644:5: error: conflicting types for apfs_mknod 644 | int apfs_mknod(struct inode *dir, struct dentry *dentry, umode_t mode, dev_t rdev) | ^~~~~~~~~~ In file included from /root/linux-apfs-rw/dir.c:8: /root/linux-apfs-rw/apfs.h:642:12: note: previous declaration of apfs_mknod was here 642 | extern int apfs_mknod(struct user_namespace *mnt_userns, struct inode *dir, | ^~~~~~~~~~ /root/linux-apfs-rw/dir.c:649:5: error: conflicting types for apfs_mkdir 649 | int apfs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode) | ^~~~~~~~~~ In file included from /root/linux-apfs-rw/dir.c:8: /root/linux-apfs-rw/apfs.h:644:12: note: previous declaration of apfs_mkdir was here 644 | extern int apfs_mkdir(struct user_namespace *mnt_userns, struct inode *dir, | ^~~~~~~~~~ /root/linux-apfs-rw/dir.c:654:5: error: conflicting types for apfs_create 654 | int apfs_create(struct inode *dir, struct dentry *dentry, umode_t mode, | ^~~~~~~~~~~ In file included from /root/linux-apfs-rw/dir.c:8: /root/linux-apfs-rw/apfs.h:649:12: note: previous declaration of apfs_create was here 649 | extern int apfs_create(struct user_namespace *mnt_userns, struct inode *dir, | ^~~~~~~~~~~ /root/linux-apfs-rw/dir.c:1197:5: error: conflicting types for apfs_rename 1197 | int apfs_rename(struct inode *old_dir, struct dentry *old_dentry, | ^~~~~~~~~~~ In file included from /root/linux-apfs-rw/dir.c:8: /root/linux-apfs-rw/apfs.h:646:12: note: previous declaration of apfs_rename was here 646 | extern int apfs_rename(struct user_namespace *mnt_userns, struct inode *old_dir, | ^~~~~~~~~~~ make[2]: *** [scripts/Makefile.build:271: /root/linux-apfs-rw/dir.o] Error 1 make[1]: *** [Makefile:1851: /root/rw] Error 2 make[1]: Leaving directory '/root/linux' make: *** [Makefile:14: default] Error 2
Although keeping forward and backward compatibilities is painful, add awful preprocessors to let compilation pass.
Signed-off-by: Su Yue [email protected]
-
wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
Hey @eafer, I'm very very keen on using this!
[[email protected] ~]$ yay apfs 5 aur/linux-apfs-git 1:r6.2a598f1-1 (+2 0.68) (Installed: 1:r8.0e35d97-1) Experimental APFS kernel module (DKMS) 4 aur/apfsprogs-git r304.26f3461-1 (+3 0.68) (Installed: r331.8ecdb0b-1) Experimental APFS tools for linux 3 aur/linux-apfs-dkms-git 1:r5.277a34e-1 (+5 0.68) (Installed: 1:r8.0e35d97-1) Experimental APFS kernel module (DKMS) 2 aur/apfs-fuse-git r77.670e45e-1 (+9 0.79) (Installed: r92.ee71aa5-1) FUSE driver for APFS (Apple File System)
# dkms status linux-apfs, 0.1, 5.10.26-1-lts, x86_64: installed linux-apfs, 0.1, 5.11.10-arch1-1, x86_64: installed (original_module exists)
Install over your existing linux-apfs-dkms-git? https://github.com/linux-apfs/linux-apfs-rw/commit/cb4f9d2514ea0d4913ddc2ce3b4fe95258b22587
sudo pacman -Rns linux-apfs-dkms-git linux-apfs-git yay --getpkgbuild linux-apfs-dkms-git cd linux-apfs-dkms-git sed -i -e s/oot/rw/ PKGBUILD makepkg -si
I have a 50GB raw disk image, made with https://github.com/sickcodes/Docker-OSX
It is APFS + Unencrypted + Case Sensitive:
export IMAGE="${PWD}/mac_hdd_ng_nonqcow.img" sudo mkdir -p "/media/${USER}/macos" fdisk -l ${IMAGE}
Disk ./mac_hdd_ng_nonqcow.img: 50 GiB, 53687091200 bytes, 104857600 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 5C21B10B-9C3E-4AE9-8482-F4C321A8094D Device Start End Sectors Size Type ./mac_hdd_ng_nonqcow.img1 40 409639 409600 200M EFI System ./mac_hdd_ng_nonqcow.img2 409640 48174215 47764576 22.8G Apple APFS
apfs-fuse
works fine (read-only)sudo apfs-fuse -o allow_other "$IMAGE" "/media/${USER}/macos" ls "/media/${USER}/macos" # private-dir root
sudo touch "/media/${USER}/macos/root/ok" # touch: cannot touch '/media/user/macos/root/ok': Read-only file system sudo umount "/media/${USER}/macos"
When I try mount your new one, I get the bad superblock error :(
[[email protected] test]$ sudo mount -t apfs -o cknodes,vol=1 "$IMAGE" "/media/${USER}/macos" mount: /media/${USER}/macos: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
[[email protected] test]$ stat /dev/loop0 File: /dev/loop0 Size: 0 Blocks: 0 IO Block: 4096 block special file Device: 5h/5d Inode: 801 Links: 1 Device type: 7,0 Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 994/ disk) Access: 2021-04-04 00:22:50.183666683 +0000 Modify: 2021-04-04 00:22:50.183666683 +0000 Change: 2021-04-04 00:22:50.183666683 +0000 Birth: -
sudo losetup -d /dev/loop0
[[email protected] ~]$ sudo losetup -d /dev/loop0 losetup: /dev/loop0: detach failed: No such device or address
Since I can still apfs-fuse, I need to remove all and install ONLY this
-rw
repo instead of-oot
?Also, perhaps change https://github.com/linux-apfs/linux-apfs-rw/blob/master/dkms.conf to
PACKAGE_NAME="linux-apfs-rw" PACKAGE_VERSION="0.2" BUILT_MODULE_NAME[0]="apfsrw" DEST_MODULE_LOCATION[0]="/extra"
Or will you merge into linux-apfs soon?
Should I try the non-dkms version?
-
Overwriting files does not work
Executing
echo test > test.txt
twice on a readwrite mounted APFS filesystem prints "operation not supported: test.txt" and does not overwrite the file.Is this something you plan to support in the future? How much work would it be to implement this?
-
Add install target to Makefile
I will use this to make packaging the kernel module on NixOS easier. Executing this target will automatically compress
apfs.ko
and put it into the correct directory,$out/lib/modules/kernel-version-here/extra/
in the case of NixOS. I have not tested this on any other Linux distro. -
Failed to compile on RHEL 9 with Linux 5.14.0
When I was trying to compile this, it reports that there was an error in
super.c
.Compilation log
make -C /lib/modules/5.14.0-70.13.1.el9_0.x86_64/build M=/home/tienyu/linux-apfs-rw make[1]: Entering directory '/usr/src/kernels/5.14.0-70.13.1.el9_0.x86_64' CC [M] /home/tienyu/linux-apfs-rw/btree.o CC [M] /home/tienyu/linux-apfs-rw/compress.o CC [M] /home/tienyu/linux-apfs-rw/dir.o CC [M] /home/tienyu/linux-apfs-rw/extents.o CC [M] /home/tienyu/linux-apfs-rw/file.o CC [M] /home/tienyu/linux-apfs-rw/inode.o CC [M] /home/tienyu/linux-apfs-rw/key.o CC [M] /home/tienyu/linux-apfs-rw/message.o CC [M] /home/tienyu/linux-apfs-rw/namei.o CC [M] /home/tienyu/linux-apfs-rw/node.o CC [M] /home/tienyu/linux-apfs-rw/object.o CC [M] /home/tienyu/linux-apfs-rw/spaceman.o CC [M] /home/tienyu/linux-apfs-rw/super.o /home/tienyu/linux-apfs-rw/super.c: In function 'apfs_setup_bdi': /home/tienyu/linux-apfs-rw/super.c:1073:31: error: 'struct block_device' has no member named 'bd_bdi' 1073 | bdi_dev = nxi->nx_bdev->bd_bdi; | ^~ make[2]: *** [scripts/Makefile.build:271: /home/tienyu/linux-apfs-rw/super.o] Error 1 make[1]: *** [Makefile:1862: /home/tienyu/linux-apfs-rw] Error 2 make[1]: Leaving directory '/usr/src/kernels/5.14.0-70.13.1.el9_0.x86_64' make: *** [Makefile:16: default] Error 2
However I looked into
super.c
then and found a piece of code which detects kernel version:#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 15, 0) bdi_dev = nxi->nx_bdev->bd_bdi; #else bdi_dev = nxi->nx_bdev->bd_disk->bdi; #endif
I tried deleting the whole part and use
bdi_dev = nxi->nx_bdev->bd_disk->bdi;
(i.e. the one for Linux 5.15+) only and it compiled successfully. That's pretty wired since I'm on Red Hat Enterprise Linux 9 and my kernel version is 5.14.0.My kernel info (
uname -a
)Linux TienyusRHEL 5.14.0-70.13.1.el9_0.x86_64 #1 SMP PREEMPT Thu Apr 14 12:42:38 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux
Is there any way to work it out? Thank you for your work!
-
Incompatible with kernels >= 5.18
DKMS make.log for linux-apfs-rw-0.2 for kernel 5.18.3-arch1-1 (x86_64) Mon Jun 13 23:06:19 PDT 2022 make: Entering directory '/usr/lib/modules/5.18.3-arch1-1/build' CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/btree.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/compress.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/dir.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/extents.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/file.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/inode.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/message.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/key.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/namei.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/node.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/object.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/spaceman.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/super.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/symlink.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/transaction.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/unicode.o CC [M] /var/lib/dkms/linux-apfs-rw/0.2/build/xattr.o /var/lib/dkms/linux-apfs-rw/0.2/build/inode.c:524:10: error: ‘const struct address_space_operations’ has no member named ‘invalidatepage’; did you mean ‘invalidate_folio’? 524 | .invalidatepage = apfs_noop_invalidatepage, | ^~~~~~~~~~~~~~ | invalidate_folio /var/lib/dkms/linux-apfs-rw/0.2/build/inode.c:524:27: error: initialization of ‘sector_t (*)(struct address_space *, sector_t)’ {aka ‘long long unsigned int (*)(struct address_space *, long long unsigned int)’} from incompatible pointer type ‘void (*)(struct page *, unsigned int, unsigned int)’ [-Werror=incompatible-pointer-types] 524 | .invalidatepage = apfs_noop_invalidatepage, | ^~~~~~~~~~~~~~~~~~~~~~~~ /var/lib/dkms/linux-apfs-rw/0.2/build/inode.c:524:27: note: (near initialization for ‘apfs_aops.bmap’) cc1: some warnings being treated as errors make[1]: *** [scripts/Makefile.build:288: /var/lib/dkms/linux-apfs-rw/0.2/build/inode.o] Error 1 make[1]: *** Waiting for unfinished jobs.... make: *** [Makefile:1834: /var/lib/dkms/linux-apfs-rw/0.2/build] Error 2 make: Leaving directory '/usr/lib/modules/5.18.3-arch1-1/build'
Same git revision worked for 5.17.x kernels.
-
mount -t apfs device mountpoint
mounting apfs looks ok ...but the files inside the tree structure are almost empty I have mac's rootfs from a working virtual mac (I can play jessica rhaye singing bob dylan on youtube ; so it is really working) all the filenames in directory tree look quite normal ... but its contents are simply 0 I have just followed prescription make; modprobe libcrc... and insmod apfs.ko I am not aiming at modifying files in apfs ,but just willing to clone a apfs_mojave back into hfs_mojave using tar between the 2 filesytems under linux instead of using CCC carbone clone copy thank you
-
readwrite option not working
Thanks for this effort you are making by enabling apfs on Linux. I used your other tool to format an external hard disk, and everything seems to work. When I mount the newly formatted partition with:
sudo mount -o readwrite /dev/sdc1 Backup
And I try to create anything either with
sudo
or as a regular user; it does not let me do that.mkdir: cannot create directory ‘hola’: Read-only file system
PS. This is on Debian Unstable.
-
Mounting with -o cknodes,readwrite shows problem
While writing a simple test for NixOS, I experimented a bit and found a problem. This is what I did:
truncate -s 1G /tmp/tmp.img
losetup -f /tmp/tmp.img
mkapfs /dev/loop0
mkdir /mnt/tmp
mount -o cknodes,readwrite /dev/loop0 /mnt/tmp
touch /mnt/tmp/testfile
fails with "touch: cannot touch '/mnt/tmp/testfile': Bad message"- "APFS (86g): bad checksum for the space manager" and "APFS (86g): aborting transaction" appear in dmesg
Is this a known problem?
An Out-of-the-Box TensorRT-based Framework for High Performance Inference with C++/Python Support
An Out-of-the-Box TensorRT-based Framework for High Performance Inference with C++/Python Support
Instagram's experimental performance oriented greenfield implementation of Python.
Welcome to Skybison! Skybison is experimental performance-oriented greenfield implementation of Python 3.8. It contains a number of performance optimi
Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI
High-Performance-Computing-Experiments Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI 实验结
BayesOpt: A toolbox for bayesian optimization, experimental design and stochastic bandits.
BayesOpt: A Bayesian optimization library BayesOpt is an efficient implementation of the Bayesian optimization methodology for nonlinear optimization,
Experimental OpenCL SPIR-V to OpenCL C translator
spirv2clc spirv2clc is an experimental OpenCL SPIR-V to OpenCL C translator currently targeting OpenCL 1.2 support. It can generate OpenCL C code equi
Read and write rosbag on a platform without ROS installed, using MQTT for message delivery.
EasyRosBag x86_64: Test on Ubuntu18.04 arm64 : Test on Ubuntu21.04(raspberry Pi4) Introducton ROS(Robot Operation System) is too fat!!! We do not ins
A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.
Light Gradient Boosting Machine LightGBM is a gradient boosting framework that uses tree based learning algorithms. It is designed to be distributed a
ldd as a tree with an option to bundle dependencies into a single folder
libtree A tool that: ?? turns ldd into a tree ☝️ explains why shared libraries are found and why not ?? optionally deploys executables and dependencie
C++11 wrapper for the LMDB embedded B+ tree database library.
lmdb++: a C++11 wrapper for LMDB This is a comprehensive C++ wrapper for the LMDB embedded database library, offering both an error-checked procedural
Reactive Light Training Module used in fitness for developing agility and reaction speed.
Hello to you , Thanks for taking interest in this project. Use case of this project is to help people that want to improve their agility and reactio
A simple facial recognition script using OpenCV's FaceRecognizer module implemented in C++
Local Binary Patterns Histogram Recognizer A proyect that implements the LBPHRecognizer class of the OpenCV library to determine if a detected face co
The module for my life story archive that gives data and statistics for the family Kindle Fire.
By: Top README.md Read this article in a different language Sorted by: A-Z Sorting options unavailable ( af Afrikaans Afrikaans | sq Shqiptare Albania
A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.
Libonnx A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support. Getting Started The library's
Deep Learning API and Server in C++11 support for Caffe, Caffe2, PyTorch,TensorRT, Dlib, NCNN, Tensorflow, XGBoost and TSNE
Open Source Deep Learning Server & API DeepDetect (https://www.deepdetect.com/) is a machine learning API and server written in C++11. It makes state
A PoC for requesting HWIDs directly from hardware, skipping any potential hooks or OS support.
PCIBan A PoC for requesting HWIDs directly from hardware, skipping any potential hooks or OS support. This is probably very unsafe, not supporting edg
Support Yolov4/Yolov3/Centernet/Classify/Unet. use darknet/libtorch/pytorch to onnx to tensorrt
ONNX-TensorRT Yolov4/Yolov3/CenterNet/Classify/Unet Implementation Yolov4/Yolov3 centernet INTRODUCTION you have the trained model file from the darkn
Benchmark framework of 3D integrated CIM accelerators for popular DNN inference, support both monolithic and heterogeneous 3D integration
3D+NeuroSim V1.0 The DNN+NeuroSim framework was developed by Prof. Shimeng Yu's group (Georgia Institute of Technology). The model is made publicly av
ROS2 packages based on NVIDIA libArgus library for hardware-accelerated CSI camera support.
Isaac ROS Argus Camera This repository provides monocular and stereo nodes that enable ROS developers to use cameras connected to Jetson platforms ove
Code generation for automatic differentiation with GPU support.
Code generation for automatic differentiation with GPU support.