APFS module for linux, with experimental write support (out-of-tree repository)

Overview
Apple File System
=================

The Apple File System (APFS) is the copy-on-write filesystem currently used on
all Apple devices. This module provides a degree of experimental support on
Linux.

It's supposed to work with a range of kernel versions starting at 4.9 or before,
but only a few of those have actually been tested. If you run into any problem,
please send a report to <[email protected]>.

To help test write support, a set of userland tools is also under development.
The git tree can be retrieved from <git://github.com/eafer/apfsprogs.git>.

Known limitations
=================

This module is the result of reverse engineering and testing has been limited.
You should expect data corruption. Please report any issues that you find.

Apple has released other versions of the filesystem to the public before the
current one. I would not expect them to be compatible with this module at all,
but I am open to fixing that if requested.

Many features are not yet supported:

 o Encryption.
 o Compression, though the compressed contents of a file can be read from the
   'com.apple.decmpfs' and 'com.apple.ResourceFork' xattrs as long as they are
   under 64k.
 o Restoring to a snapshot.
 o Access control lists. This is not a priority.

Build
=====

In order to build a module out-of-tree, you will first need the Linux kernel
headers. On Debian, you can get them by running (as root):

  apt-get install linux-headers-$(uname -r)

Now you can just cd to the linux-apfs-oot directory and run

  make

The resulting module is the apfs.ko file. Before you can use it you must insert
it into the kernel, as well as its dependencies. Again as root:

  modprobe libcrc32c
  insmod apfs.ko

Mount
=====

Like all filesystems, apfs is mounted with

  mount [-o options] device dir

where 'device' is the path to your device file or filesystem image, and 'dir'
is the mount point. The following options are accepted:

  vol=n
	Volume number to mount. The default is volume 0.

  uid=n, gid=n
	Override on-disk inode ownership data with given uid/gid.

  cknodes
	Verify the checksum on all metadata nodes. Right now this has
	a severe performance cost, so it's not recommended.

  readwrite
	Enable the experimental write support. This WILL corrupt the container.

So for instance, if you want to mount volume number 2, and you want the metadata
to be checked, you should run (as root):

  mount -o cknodes,vol=2 device dir

To unmount it, run

  umount dir

Credits
=======

Originally written by Ernesto A. Fernández <[email protected]>,
with several contributions from Gabriel Krisman Bertazi <[email protected]>,
Arnaud Ferraris <[email protected]> and Stan Skowronek
<[email protected]>. For attribution details see the historical
git tree at <git://github.com/eafer/linux-apfs.git>.

Work was first based on reverse engineering done by others [1][2], and later
on the (very incomplete) official specification [3]. Some parts of the code
imitate the ext2 module, and to a lesser degree xfs, udf, gfs2 and hfsplus.

[1] Hansen, K.H., Toolan, F., Decoding the APFS file system, Digital
    Investigation (2017), http://dx.doi.org/10.1016/j.diin.2017.07.003
[2] https://github.com/sgan81/apfs-fuse
[3] https://developer.apple.com/support/apple-file-system/Apple-File-System-Reference.pdf
Issues
  • Does not compile on Linux 4.14.237 and lower

    Does not compile on Linux 4.14.237 and lower

    While preparing to add this kernel module to NixOS, I tried to build it for all kernels in Nixpkgs and it failed for three kernel versions. The README states, that this is supposed to work with kernel version 4.9 or before but it fails to build on 4.4.273, 4.9.273 and 4.14.237. The error message for Linux 4.14.237 boils down to

    /build/source/dir.c:652:2: error: implicit declaration of function 'discard_new_inode'
      652 |  discard_new_inode(inode);
          |  ^~~~~~~~~~~~~~~~~
    

    and the error message for Linux 4.4.273 and 4.9.273:

       23 |  SB_RDONLY = MS_RDONLY;
          |  ^~~~~~~~~
    /build/source/apfs.h:23:2: error: type defaults to 'int' in declaration of 'SB_RDONLY'
    

    I don't really care about those kernels. You should decide if you want to fix this or not as I can of course just exclude building this module with those old kernels in Nixpkgs. Please either fix the compilation or update the README. As always, I'm happy to try to compile any patches you may develop.

    opened by Luflosi 19
  • Does not compile on Linux 5.14.1

    Does not compile on Linux 5.14.1

    The latest commit from the develop branch does not compile on Linux 5.14.1. Here is the relevant build log:

    /build/source/inode.c: In function 'apfs_do_ioc_setflags':
    /build/source/inode.c:1807:8: error: implicit declaration of function 'vfs_ioc_setflags_prepare' [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wimplicit-function-declaration-Werror=implicit-function-declaration8;;]
     1807 |  err = vfs_ioc_setflags_prepare(inode, oldflags, newflags);
          |        ^~~~~~~~~~~~~~~~~~~~~~~~
    /build/source/inode.c: In function 'apfs_ioc_setflags':
    /build/source/inode.c:1844:30: error: passing argument 1 of 'inode_owner_or_capable' from incompatible pointer type [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wincompatible-pointer-types-Werror=incompatible-pointer-types8;;]
     1844 |  if (!inode_owner_or_capable(inode))
          |                              ^~~~~
          |                              |
          |                              struct inode *
    In file included from /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/buffer_head.h:12,
                     from /build/source/inode.c:7:
    /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/fs.h:1888:52: note: expected 'struct user_namespace *' but argument is of type 'struct inode *'
     1888 | bool inode_owner_or_capable(struct user_namespace *mnt_userns,
          |                             ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
    /build/source/inode.c:1844:7: error: too few arguments to function 'inode_owner_or_capable'
     1844 |  if (!inode_owner_or_capable(inode))
          |       ^~~~~~~~~~~~~~~~~~~~~~
    In file included from /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/buffer_head.h:12,
                     from /build/source/inode.c:7:
    /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/fs.h:1888:6: note: declared here
     1888 | bool inode_owner_or_capable(struct user_namespace *mnt_userns,
          |      ^~~~~~~~~~~~~~~~~~~~~~
    cc1: some warnings being treated as errors
    

    Let me know if I can provide any additional information.

    opened by Luflosi 8
  • Fails to compile on Linux 5.12.11

    Fails to compile on Linux 5.12.11

    I'm trying to build this on NixOS with Linux 5.12.11 but the build fails. I don't think this issue has to do with NixOS as it compiled fine with an older kernel (5.11.x). Here is the build output from nix-build:

    this derivation will be built:
      /nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv
    building '/nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv'...
    unpacking sources
    unpacking source archive /nix/store/0br698lgx8dm5bss6n25ls4llfv2q0zj-source
    source root is source
    setting SOURCE_DATE_EPOCH to timestamp 1624192656 of file source/xfield.c
    patching sources
    applying patch /nix/store/20r89d87xwycl20cfcsv4d884qlcp0gq-9.patch
    patching file Makefile
    configuring
    no configure script, doing nothing
    building
    build flags: SHELL=/nix/store/kxj6cblcsd1qcbbxlmbswwrn89zcmgd6-bash-4.4-p23/bin/bash KERNELRELEASE=5.12.11 KERNEL_DIR=/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build INSTALL_MOD_PATH=\$\(out\)
    make -C /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build M=/build/source
    make[1]: Entering directory '/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build'
      CC [M]  /build/source/btree.o
      CC [M]  /build/source/compress.o
      CC [M]  /build/source/dir.o
      CC [M]  /build/source/extents.o
      CC [M]  /build/source/file.o
      CC [M]  /build/source/inode.o
      CC [M]  /build/source/key.o
      CC [M]  /build/source/message.o
      CC [M]  /build/source/namei.o
      CC [M]  /build/source/node.o
      CC [M]  /build/source/object.o
      CC [M]  /build/source/spaceman.o
      CC [M]  /build/source/super.o
    /build/source/super.c: In function 'apfs_attach_nxi':
    /build/source/super.c:1128:9: error: too few arguments to function 'lookup_bdev'
     1128 |  bdev = lookup_bdev(dev_name);
          |         ^~~~~~~~~~~
    In file included from /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/backing-dev.h:15,
                     from /build/source/super.c:6:
    /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/blkdev.h:1984:5: note: declared here
     1984 | int lookup_bdev(const char *pathname, dev_t *dev);
          |     ^~~~~~~~~~~
    make[3]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/scripts/Makefile.build:271: /build/source/super.o] Error 1
    make[2]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:1856: /build/source] Error 2
    make[1]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:215: __sub-make] Error 2
    make[1]: Leaving directory '/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build'
    make: *** [Makefile:16: default] Error 2
    error: builder for '/nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv' failed with exit code 2;
           last 10 log lines:
           > In file included from /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/backing-dev.h:15,
           >                  from /build/source/super.c:6:
           > /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/blkdev.h:1984:5: note: declared here
           >  1984 | int lookup_bdev(const char *pathname, dev_t *dev);
           >       |     ^~~~~~~~~~~
           > make[3]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/scripts/Makefile.build:271: /build/source/super.o] Error 1
           > make[2]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:1856: /build/source] Error 2
           > make[1]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:215: __sub-make] Error 2
           > make[1]: Leaving directory '/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build'
           > make: *** [Makefile:16: default] Error 2
           For full logs, run 'nix log /nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv'.
    

    This looks like something has changed in the kernel to which linux-apfs needs to be adapted to.

    opened by Luflosi 5
  • Clarify license

    Clarify license

    Source files currently contain headers with SPDX-License-Identifier: GPL-2.0. However, as per https://spdx.org/licenses/, there is no such license identifier, I am assuming it was meant to be "GPL-2.0-only" or "GPL-2.0-or-later". Please clarify which version is used by this project.

    opened by Dekedro 3
  • Feedback for NixOS test

    Feedback for NixOS test

    The last missing piece for perfect NixOS integration is a NixOS module. This will allow awesome things like specifying

    fileSystems."/mnt/apfs" = {
      device = "/dev/disk/by-partlabel/apfs";
      fsType = "apfs";
    }
    

    in the configuration, which is all that is required to automatically mount the filesystem during boot, no need to install anything or configure anything else. As part of writing a NixOS module, it is good practice to write a test for a basic sanity check and to gain some confidence, that at least the basic stuff all works as expected. Here is how the test looks currently:

    import ./make-test-python.nix ({ pkgs, ... }: {
      name = "apfs";
      meta.maintainers = with pkgs.lib.maintainers; [ Luflosi ];
    
      machine = { pkgs, ... }: {
        virtualisation.emptyDiskImages = [ 1024 ];
    
        boot.supportedFilesystems = [ "apfs" ];
      };
    
      testScript = ''
        machine.wait_for_unit("basic.target")
        machine.succeed("mkdir /tmp/mnt")
    
        with subtest("mkapfs refuses to work with a label that is too long"):
          machine.fail( "mkapfs -L '000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F' /dev/vdb")
    
        with subtest("mkapfs works with the maximum label length"):
          machine.succeed("mkapfs -L '000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7' /dev/vdb")
    
        with subtest("Enable case sensitivity and normalization sensitivity"):
          machine.succeed(
              "mkapfs -s -z /dev/vdb",
              # Triggers a bug, see https://github.com/linux-apfs/linux-apfs-rw/issues/15
              # "mount -o cknodes,readwrite /dev/vdb /tmp/mnt",
              "mount -o readwrite /dev/vdb /tmp/mnt",
              "echo 'Hello World 1' > /tmp/mnt/test.txt",
              "[ ! -f /tmp/mnt/TeSt.TxT ] || false", # Test case sensitivity
              "echo 'Hello World 1' | diff - /tmp/mnt/test.txt",
              "echo 'Hello World 2' > /tmp/mnt/\u0061\u0301.txt",
              "echo 'Hello World 2' | diff - /tmp/mnt/\u0061\u0301.txt",
              "[ ! -f /tmp/mnt/\u00e1.txt ] || false", # Test Unicode normalization sensitivity
              "umount /tmp/mnt",
              "apfsck /dev/vdb",
          )
        with subtest("Disable case sensitivity and normalization sensitivity"):
          machine.succeed(
              "mkapfs /dev/vdb",
              "mount -o readwrite /dev/vdb /tmp/mnt",
              "echo 'Hello World Hello World Hello World' > /tmp/mnt/Test.txt",
              "echo -n 'Hello World' > /tmp/mnt/test.txt",
              "echo ' 1' >> /tmp/mnt/TEST.TXT",
              "umount /tmp/mnt",
              "apfsck /dev/vdb",
              "mount -o readwrite /dev/vdb /tmp/mnt",
              "echo 'Hello World 1' | diff - /tmp/mnt/TeSt.TxT", # Test case insensitivity
              "echo 'Hello World 2' > /tmp/mnt/\u0061\u0301.txt",
              "echo 'Hello World 2' | diff - /tmp/mnt/\u0061\u0301.txt",
              "echo 'Hello World 2' | diff - /tmp/mnt/\u00e1.txt", # Test Unicode normalization
              "umount /tmp/mnt",
              "apfsck /dev/vdb",
          )
      '';
    })
    

    Most of the space in the file is taken up by a Python script. The strings passed to machine.succeed() are shell script one-liners that all need to succeed for the test to succeed. The test builds a qemu VM with an empty 1GB disk image, loads the APFS kernel module and then executes the tests. Here is the NixOS test for ZFS. It basically tests smb file sharing, encryption and forefully importing a pool on a machine with a different host ID, for two versions of ZFS and the Linux kernel. I'd like to hear your feedback, if you think this is an adequate amount of testing or if you would test more or test less? Should I maybe reboot the VM somewhere during the test? Maybe test long filenames? Call apfsck more often? Create a whole bunch of files and see if any disappear? Something else entirely?

    opened by Luflosi 2
  • /lib/modules/5.13.9-arch1-1/build: No such file or directory.  Stop.

    /lib/modules/5.13.9-arch1-1/build: No such file or directory. Stop.

    I am on Arch, so I simply downloaded linux-headers from pacman, cloned the repo, cd'd into the repo, ran make and got this error:

    make -C /lib/modules/5.13.9-arch1-1/build M=/home/nate/linux-apfs-rw
    make[1]: *** /lib/modules/5.13.9-arch1-1/build: No such file or directory.  Stop.
    make: *** [Makefile:15: default] Error 2
    

    Did I miss something?

    EDIT: in /lib/modules I have a folder called 5.13.12-arch1-1... Do I need that specific version for this to work?

    opened by n8jadams 2
  • Label does not seem to work

    Label does not seem to work

    The label of an APFS filesystem does not seem to show up anywhere. The steps I tried:

    1. modprobe apfs
    2. truncate -s 1G /tmp/test.img
    3. losetup -f /tmp/test.img
    4. mkapfs -L testing /dev/loop0
    5. lsblk -f does not show a label for loop0
    6. ls -la /dev/disk/by-label does not show loop0
    7. mounting the filesystem does not change the situation

    I don't think this is a problem with mkapfs because an APFS filesystem created with macOS also does not show a label in Linux and copying /tmp/test.img to a macOS machine and mounting it does show the label.

    This issue is probably very low on your TODO list. I just wanted to know if it is a known issue and if I did something wrong or if this is supposed to work.

    opened by Luflosi 2
  • apfs: fix failed compilation due to idmap mount changes in v5.12

    apfs: fix failed compilation due to idmap mount changes in v5.12

    Since kernel commit 549c7297717c ("fs: make helpers idmap mount aware") extended the exisiting inode method to let filesystems be aware of idmapped mounts, compiling apfs module with linux v5.12 failed:

    make make -C /lib/modules/5.12.0-custom/build M=/root/rw make[1]: Entering directory '/root/linux' CC [M] /root/linux-apfs-rw/dir.o /root/linux-apfs-rw/dir.c:644:5: error: conflicting types for apfs_mknod 644 | int apfs_mknod(struct inode *dir, struct dentry *dentry, umode_t mode, dev_t rdev) | ^~~~~~~~~~ In file included from /root/linux-apfs-rw/dir.c:8: /root/linux-apfs-rw/apfs.h:642:12: note: previous declaration of apfs_mknod was here 642 | extern int apfs_mknod(struct user_namespace *mnt_userns, struct inode *dir, | ^~~~~~~~~~ /root/linux-apfs-rw/dir.c:649:5: error: conflicting types for apfs_mkdir 649 | int apfs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode) | ^~~~~~~~~~ In file included from /root/linux-apfs-rw/dir.c:8: /root/linux-apfs-rw/apfs.h:644:12: note: previous declaration of apfs_mkdir was here 644 | extern int apfs_mkdir(struct user_namespace *mnt_userns, struct inode *dir, | ^~~~~~~~~~ /root/linux-apfs-rw/dir.c:654:5: error: conflicting types for apfs_create 654 | int apfs_create(struct inode *dir, struct dentry *dentry, umode_t mode, | ^~~~~~~~~~~ In file included from /root/linux-apfs-rw/dir.c:8: /root/linux-apfs-rw/apfs.h:649:12: note: previous declaration of apfs_create was here 649 | extern int apfs_create(struct user_namespace *mnt_userns, struct inode *dir, | ^~~~~~~~~~~ /root/linux-apfs-rw/dir.c:1197:5: error: conflicting types for apfs_rename 1197 | int apfs_rename(struct inode *old_dir, struct dentry *old_dentry, | ^~~~~~~~~~~ In file included from /root/linux-apfs-rw/dir.c:8: /root/linux-apfs-rw/apfs.h:646:12: note: previous declaration of apfs_rename was here 646 | extern int apfs_rename(struct user_namespace *mnt_userns, struct inode *old_dir, | ^~~~~~~~~~~ make[2]: *** [scripts/Makefile.build:271: /root/linux-apfs-rw/dir.o] Error 1 make[1]: *** [Makefile:1851: /root/rw] Error 2 make[1]: Leaving directory '/root/linux' make: *** [Makefile:14: default] Error 2

    Although keeping forward and backward compatibilities is painful, add awful preprocessors to let compilation pass.

    Signed-off-by: Su Yue [email protected]

    opened by Damenly 2
  • wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.

    wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.

    Hey @eafer, I'm very very keen on using this!

    [[email protected] ~]$ yay apfs
    5 aur/linux-apfs-git 1:r6.2a598f1-1 (+2 0.68) (Installed: 1:r8.0e35d97-1)
        Experimental APFS kernel module (DKMS)
    4 aur/apfsprogs-git r304.26f3461-1 (+3 0.68) (Installed: r331.8ecdb0b-1)
        Experimental APFS tools for linux
    3 aur/linux-apfs-dkms-git 1:r5.277a34e-1 (+5 0.68) (Installed: 1:r8.0e35d97-1)
        Experimental APFS kernel module (DKMS)
    2 aur/apfs-fuse-git r77.670e45e-1 (+9 0.79) (Installed: r92.ee71aa5-1)
        FUSE driver for APFS (Apple File System)
    
    # dkms status
    linux-apfs, 0.1, 5.10.26-1-lts, x86_64: installed
    linux-apfs, 0.1, 5.11.10-arch1-1, x86_64: installed (original_module exists)
    

    Install over your existing linux-apfs-dkms-git? https://github.com/linux-apfs/linux-apfs-rw/commit/cb4f9d2514ea0d4913ddc2ce3b4fe95258b22587

    sudo pacman -Rns linux-apfs-dkms-git linux-apfs-git
    yay --getpkgbuild linux-apfs-dkms-git
    cd linux-apfs-dkms-git
    sed -i -e s/oot/rw/ PKGBUILD
    makepkg -si
    

    I have a 50GB raw disk image, made with https://github.com/sickcodes/Docker-OSX

    It is APFS + Unencrypted + Case Sensitive:

    export IMAGE="${PWD}/mac_hdd_ng_nonqcow.img"
    sudo mkdir -p "/media/${USER}/macos"
    fdisk -l ${IMAGE}
    
    Disk ./mac_hdd_ng_nonqcow.img: 50 GiB, 53687091200 bytes, 104857600 sectors
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: gpt
    Disk identifier: 5C21B10B-9C3E-4AE9-8482-F4C321A8094D
    
    Device                                                   Start      End  Sectors  Size Type
    ./mac_hdd_ng_nonqcow.img1     40   409639   409600  200M EFI System
    ./mac_hdd_ng_nonqcow.img2 409640 48174215 47764576 22.8G Apple APFS
    

    apfs-fuse works fine (read-only)

    sudo apfs-fuse -o allow_other "$IMAGE" "/media/${USER}/macos"
    ls "/media/${USER}/macos"
    # private-dir  root
    
    sudo touch "/media/${USER}/macos/root/ok"
    # touch: cannot touch '/media/user/macos/root/ok': Read-only file system
    sudo umount "/media/${USER}/macos"
    

    When I try mount your new one, I get the bad superblock error :(

    [[email protected] test]$ sudo mount -t apfs -o cknodes,vol=1 "$IMAGE" "/media/${USER}/macos"
    mount: /media/${USER}/macos: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
    
    [[email protected] test]$ stat /dev/loop0 
      File: /dev/loop0
      Size: 0         	Blocks: 0          IO Block: 4096   block special file
    Device: 5h/5d	Inode: 801         Links: 1     Device type: 7,0
    Access: (0660/brw-rw----)  Uid: (    0/    root)   Gid: (  994/    disk)
    Access: 2021-04-04 00:22:50.183666683 +0000
    Modify: 2021-04-04 00:22:50.183666683 +0000
    Change: 2021-04-04 00:22:50.183666683 +0000
     Birth: -
    
    sudo losetup -d /dev/loop0
    
    [[email protected] ~]$ sudo losetup -d /dev/loop0
    losetup: /dev/loop0: detach failed: No such device or address
    

    Since I can still apfs-fuse, I need to remove all and install ONLY this -rw repo instead of -oot?

    Also, perhaps change https://github.com/linux-apfs/linux-apfs-rw/blob/master/dkms.conf to

    PACKAGE_NAME="linux-apfs-rw"
    PACKAGE_VERSION="0.2"
    
    BUILT_MODULE_NAME[0]="apfsrw"
    DEST_MODULE_LOCATION[0]="/extra"
    

    Or will you merge into linux-apfs soon?

    Should I try the non-dkms version?

    opened by sickcodes 2
  • Overwriting files does not work

    Overwriting files does not work

    Executing echo test > test.txt twice on a readwrite mounted APFS filesystem prints "operation not supported: test.txt" and does not overwrite the file.

    Is this something you plan to support in the future? How much work would it be to implement this?

    opened by Luflosi 1
  • Add install target to Makefile

    Add install target to Makefile

    I will use this to make packaging the kernel module on NixOS easier. Executing this target will automatically compress apfs.ko and put it into the correct directory, $out/lib/modules/kernel-version-here/extra/ in the case of NixOS. I have not tested this on any other Linux distro.

    opened by Luflosi 1
  • Failed to compile on RHEL 9 with Linux 5.14.0

    Failed to compile on RHEL 9 with Linux 5.14.0

    When I was trying to compile this, it reports that there was an error in super.c.

    Compilation log
    make -C /lib/modules/5.14.0-70.13.1.el9_0.x86_64/build M=/home/tienyu/linux-apfs-rw
    make[1]: Entering directory '/usr/src/kernels/5.14.0-70.13.1.el9_0.x86_64'
      CC [M]  /home/tienyu/linux-apfs-rw/btree.o
      CC [M]  /home/tienyu/linux-apfs-rw/compress.o
      CC [M]  /home/tienyu/linux-apfs-rw/dir.o
      CC [M]  /home/tienyu/linux-apfs-rw/extents.o
      CC [M]  /home/tienyu/linux-apfs-rw/file.o
      CC [M]  /home/tienyu/linux-apfs-rw/inode.o
      CC [M]  /home/tienyu/linux-apfs-rw/key.o
      CC [M]  /home/tienyu/linux-apfs-rw/message.o
      CC [M]  /home/tienyu/linux-apfs-rw/namei.o
      CC [M]  /home/tienyu/linux-apfs-rw/node.o
      CC [M]  /home/tienyu/linux-apfs-rw/object.o
      CC [M]  /home/tienyu/linux-apfs-rw/spaceman.o
      CC [M]  /home/tienyu/linux-apfs-rw/super.o
    /home/tienyu/linux-apfs-rw/super.c: In function 'apfs_setup_bdi':
    /home/tienyu/linux-apfs-rw/super.c:1073:31: error: 'struct block_device' has no member named 'bd_bdi'
     1073 |         bdi_dev = nxi->nx_bdev->bd_bdi;
          |                               ^~
    make[2]: *** [scripts/Makefile.build:271: /home/tienyu/linux-apfs-rw/super.o] Error 1
    make[1]: *** [Makefile:1862: /home/tienyu/linux-apfs-rw] Error 2
    make[1]: Leaving directory '/usr/src/kernels/5.14.0-70.13.1.el9_0.x86_64'
    make: *** [Makefile:16: default] Error 2
    

    However I looked into super.c then and found a piece of code which detects kernel version:

    #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 15, 0)
            bdi_dev = nxi->nx_bdev->bd_bdi;
    #else
            bdi_dev = nxi->nx_bdev->bd_disk->bdi;
    #endif
    

    I tried deleting the whole part and use bdi_dev = nxi->nx_bdev->bd_disk->bdi; (i.e. the one for Linux 5.15+) only and it compiled successfully. That's pretty wired since I'm on Red Hat Enterprise Linux 9 and my kernel version is 5.14.0.

    My kernel info (uname -a)
    Linux TienyusRHEL 5.14.0-70.13.1.el9_0.x86_64 #1 SMP PREEMPT Thu Apr 14 12:42:38 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux

    Is there any way to work it out? Thank you for your work!

    opened by lingrottin 0
  • Incompatible with kernels >= 5.18

    Incompatible with kernels >= 5.18

    DKMS make.log for linux-apfs-rw-0.2 for kernel 5.18.3-arch1-1 (x86_64)
    Mon Jun 13 23:06:19 PDT 2022
    make: Entering directory '/usr/lib/modules/5.18.3-arch1-1/build'
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/btree.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/compress.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/dir.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/extents.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/file.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/inode.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/message.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/key.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/namei.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/node.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/object.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/spaceman.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/super.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/symlink.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/transaction.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/unicode.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/xattr.o
    /var/lib/dkms/linux-apfs-rw/0.2/build/inode.c:524:10: error: ‘const struct address_space_operations’ has no member named ‘invalidatepage’; did you mean ‘invalidate_folio’?
      524 |         .invalidatepage = apfs_noop_invalidatepage,
          |          ^~~~~~~~~~~~~~
          |          invalidate_folio
    /var/lib/dkms/linux-apfs-rw/0.2/build/inode.c:524:27: error: initialization of ‘sector_t (*)(struct address_space *, sector_t)’ {aka ‘long long unsigned int (*)(struct address_space *, long long unsigned int)’} from incompatible pointer type ‘void (*)(struct page *, unsigned int,  unsigned int)’ [-Werror=incompatible-pointer-types]
      524 |         .invalidatepage = apfs_noop_invalidatepage,
          |                           ^~~~~~~~~~~~~~~~~~~~~~~~
    /var/lib/dkms/linux-apfs-rw/0.2/build/inode.c:524:27: note: (near initialization for ‘apfs_aops.bmap’)
    cc1: some warnings being treated as errors
    make[1]: *** [scripts/Makefile.build:288: /var/lib/dkms/linux-apfs-rw/0.2/build/inode.o] Error 1
    make[1]: *** Waiting for unfinished jobs....
    make: *** [Makefile:1834: /var/lib/dkms/linux-apfs-rw/0.2/build] Error 2
    make: Leaving directory '/usr/lib/modules/5.18.3-arch1-1/build'
    

    Same git revision worked for 5.17.x kernels.

    opened by melvyn2 0
  • mount -t apfs device mountpoint

    mount -t apfs device mountpoint

    mounting apfs looks ok ...but the files inside the tree structure are almost empty I have mac's rootfs from a working virtual mac (I can play jessica rhaye singing bob dylan on youtube ; so it is really working) all the filenames in directory tree look quite normal ... but its contents are simply 0 I have just followed prescription make; modprobe libcrc... and insmod apfs.ko I am not aiming at modifying files in apfs ,but just willing to clone a apfs_mojave back into hfs_mojave using tar between the 2 filesytems under linux instead of using CCC carbone clone copy thank you

    opened by hoan3504 1
  • readwrite option not working

    readwrite option not working

    Thanks for this effort you are making by enabling apfs on Linux. I used your other tool to format an external hard disk, and everything seems to work. When I mount the newly formatted partition with:

    sudo mount -o readwrite /dev/sdc1 Backup 
    

    And I try to create anything either with sudo or as a regular user; it does not let me do that.

    
    mkdir: cannot create directory ‘hola’: Read-only file system
    

    PS. This is on Debian Unstable.

    opened by muammar 10
  • Mounting with -o cknodes,readwrite shows problem

    Mounting with -o cknodes,readwrite shows problem

    While writing a simple test for NixOS, I experimented a bit and found a problem. This is what I did:

    1. truncate -s 1G /tmp/tmp.img
    2. losetup -f /tmp/tmp.img
    3. mkapfs /dev/loop0
    4. mkdir /mnt/tmp
    5. mount -o cknodes,readwrite /dev/loop0 /mnt/tmp
    6. touch /mnt/tmp/testfile fails with "touch: cannot touch '/mnt/tmp/testfile': Bad message"
    7. "APFS (86g): bad checksum for the space manager" and "APFS (86g): aborting transaction" appear in dmesg

    Is this a known problem?

    opened by Luflosi 1
Owner
APFS for Linux
APFS for Linux
An Out-of-the-Box TensorRT-based Framework for High Performance Inference with C++/Python Support

An Out-of-the-Box TensorRT-based Framework for High Performance Inference with C++/Python Support

手写AI 1k Jun 28, 2022
Instagram's experimental performance oriented greenfield implementation of Python.

Welcome to Skybison! Skybison is experimental performance-oriented greenfield implementation of Python 3.8. It contains a number of performance optimi

Meta Experimental 275 Jun 13, 2022
Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI

High-Performance-Computing-Experiments Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI 实验结

Jiang Lu 1 Nov 27, 2021
BayesOpt: A toolbox for bayesian optimization, experimental design and stochastic bandits.

BayesOpt: A Bayesian optimization library BayesOpt is an efficient implementation of the Bayesian optimization methodology for nonlinear optimization,

Ruben Martinez-Cantin 320 Jun 24, 2022
Experimental OpenCL SPIR-V to OpenCL C translator

spirv2clc spirv2clc is an experimental OpenCL SPIR-V to OpenCL C translator currently targeting OpenCL 1.2 support. It can generate OpenCL C code equi

Kévin Petit 17 Jun 8, 2022
Read and write rosbag on a platform without ROS installed, using MQTT for message delivery.

EasyRosBag x86_64: Test on Ubuntu18.04 arm64 : Test on Ubuntu21.04(raspberry Pi4) Introducton ROS(Robot Operation System) is too fat!!! We do not ins

afei 9 May 23, 2022
A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

Light Gradient Boosting Machine LightGBM is a gradient boosting framework that uses tree based learning algorithms. It is designed to be distributed a

Microsoft 13.9k Jun 30, 2022
ldd as a tree with an option to bundle dependencies into a single folder

libtree A tool that: ?? turns ldd into a tree ☝️ explains why shared libraries are found and why not ?? optionally deploys executables and dependencie

Harmen Stoppels 1.5k Jun 27, 2022
C++11 wrapper for the LMDB embedded B+ tree database library.

lmdb++: a C++11 wrapper for LMDB This is a comprehensive C++ wrapper for the LMDB embedded database library, offering both an error-checked procedural

D.R.Y. C++ 257 Jun 16, 2022
Reactive Light Training Module used in fitness for developing agility and reaction speed.

Hello to you , Thanks for taking interest in this project. Use case of this project is to help people that want to improve their agility and reactio

null 1 Oct 31, 2021
A simple facial recognition script using OpenCV's FaceRecognizer module implemented in C++

Local Binary Patterns Histogram Recognizer A proyect that implements the LBPHRecognizer class of the OpenCV library to determine if a detected face co

Pablo Agustín Ortega-Kral 0 Jan 18, 2022
The module for my life story archive that gives data and statistics for the family Kindle Fire.

By: Top README.md Read this article in a different language Sorted by: A-Z Sorting options unavailable ( af Afrikaans Afrikaans | sq Shqiptare Albania

Sean P. Myrick V19.1.7.2 1 Dec 23, 2021
A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.

Libonnx A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support. Getting Started The library's

xboot.org 400 Jun 27, 2022
Deep Learning API and Server in C++11 support for Caffe, Caffe2, PyTorch,TensorRT, Dlib, NCNN, Tensorflow, XGBoost and TSNE

Open Source Deep Learning Server & API DeepDetect (https://www.deepdetect.com/) is a machine learning API and server written in C++11. It makes state

JoliBrain 2.4k Jun 13, 2022
A PoC for requesting HWIDs directly from hardware, skipping any potential hooks or OS support.

PCIBan A PoC for requesting HWIDs directly from hardware, skipping any potential hooks or OS support. This is probably very unsafe, not supporting edg

null 48 Jun 15, 2022
Support Yolov4/Yolov3/Centernet/Classify/Unet. use darknet/libtorch/pytorch to onnx to tensorrt

ONNX-TensorRT Yolov4/Yolov3/CenterNet/Classify/Unet Implementation Yolov4/Yolov3 centernet INTRODUCTION you have the trained model file from the darkn

null 156 Jun 10, 2022
Benchmark framework of 3D integrated CIM accelerators for popular DNN inference, support both monolithic and heterogeneous 3D integration

3D+NeuroSim V1.0 The DNN+NeuroSim framework was developed by Prof. Shimeng Yu's group (Georgia Institute of Technology). The model is made publicly av

NeuroSim 10 Dec 21, 2021
ROS2 packages based on NVIDIA libArgus library for hardware-accelerated CSI camera support.

Isaac ROS Argus Camera This repository provides monocular and stereo nodes that enable ROS developers to use cameras connected to Jetson platforms ove

NVIDIA Isaac ROS 26 Jun 16, 2022
Code generation for automatic differentiation with GPU support.

Code generation for automatic differentiation with GPU support.

Eric Heiden 38 Jun 13, 2022