APFS module for linux, with experimental write support (out-of-tree repository)

Overview
Apple File System
=================

The Apple File System (APFS) is the copy-on-write filesystem currently used on
all Apple devices. This module provides a degree of experimental support on
Linux.

It's supposed to work with a range of kernel versions starting at 4.9 or before,
but only a few of those have actually been tested. If you run into any problem,
please send a report to <[email protected]>.

To help test write support, a set of userland tools is also under development.
The git tree can be retrieved from <git://github.com/eafer/apfsprogs.git>.

Known limitations
=================

This module is the result of reverse engineering and testing has been limited.
You should expect data corruption. Please report any issues that you find.

Apple has released other versions of the filesystem to the public before the
current one. I would not expect them to be compatible with this module at all,
but I am open to fixing that if requested.

Many features are not yet supported:

 o Encryption.
 o Compression, though the compressed contents of a file can be read from the
   'com.apple.decmpfs' and 'com.apple.ResourceFork' xattrs as long as they are
   under 64k.
 o Restoring to a snapshot.
 o Access control lists. This is not a priority.

Build
=====

In order to build a module out-of-tree, you will first need the Linux kernel
headers. On Debian, you can get them by running (as root):

  apt-get install linux-headers-$(uname -r)

Now you can just cd to the linux-apfs-oot directory and run

  make

The resulting module is the apfs.ko file. Before you can use it you must insert
it into the kernel, as well as its dependencies. Again as root:

  modprobe libcrc32c
  insmod apfs.ko

Mount
=====

Like all filesystems, apfs is mounted with

  mount [-o options] device dir

where 'device' is the path to your device file or filesystem image, and 'dir'
is the mount point. The following options are accepted:

  vol=n
	Volume number to mount. The default is volume 0.

  uid=n, gid=n
	Override on-disk inode ownership data with given uid/gid.

  cknodes
	Verify the checksum on all metadata nodes. Right now this has
	a severe performance cost, so it's not recommended.

  readwrite
	Enable the experimental write support. This WILL corrupt the container.

So for instance, if you want to mount volume number 2, and you want the metadata
to be checked, you should run (as root):

  mount -o cknodes,vol=2 device dir

To unmount it, run

  umount dir

Credits
=======

Originally written by Ernesto A. Fernández <[email protected]>,
with several contributions from Gabriel Krisman Bertazi <[email protected]>,
Arnaud Ferraris <[email protected]> and Stan Skowronek
<[email protected]>. For attribution details see the historical
git tree at <git://github.com/eafer/linux-apfs.git>.

Work was first based on reverse engineering done by others [1][2], and later
on the (very incomplete) official specification [3]. Some parts of the code
imitate the ext2 module, and to a lesser degree xfs, udf, gfs2 and hfsplus.

[1] Hansen, K.H., Toolan, F., Decoding the APFS file system, Digital
    Investigation (2017), http://dx.doi.org/10.1016/j.diin.2017.07.003
[2] https://github.com/sgan81/apfs-fuse
[3] https://developer.apple.com/support/apple-file-system/Apple-File-System-Reference.pdf
Comments
  • Does not compile on Linux 4.14.237 and lower

    Does not compile on Linux 4.14.237 and lower

    While preparing to add this kernel module to NixOS, I tried to build it for all kernels in Nixpkgs and it failed for three kernel versions. The README states, that this is supposed to work with kernel version 4.9 or before but it fails to build on 4.4.273, 4.9.273 and 4.14.237. The error message for Linux 4.14.237 boils down to

    /build/source/dir.c:652:2: error: implicit declaration of function 'discard_new_inode'
      652 |  discard_new_inode(inode);
          |  ^~~~~~~~~~~~~~~~~
    

    and the error message for Linux 4.4.273 and 4.9.273:

       23 |  SB_RDONLY = MS_RDONLY;
          |  ^~~~~~~~~
    /build/source/apfs.h:23:2: error: type defaults to 'int' in declaration of 'SB_RDONLY'
    

    I don't really care about those kernels. You should decide if you want to fix this or not as I can of course just exclude building this module with those old kernels in Nixpkgs. Please either fix the compilation or update the README. As always, I'm happy to try to compile any patches you may develop.

    opened by Luflosi 19
  • umount cause NULL dereference

    umount cause NULL dereference

    I have mounted apfs partition, created by Catalina. No problems, mount's fine, ls /mnt I can see files, after that I'm umount directory and got backtrace. I'm using 5.18.10-zen1-1-zen kernel on archlinux. dmesg.txt

    opened by zvova7890 9
  • Does not compile on Linux 5.14.1

    Does not compile on Linux 5.14.1

    The latest commit from the develop branch does not compile on Linux 5.14.1. Here is the relevant build log:

    /build/source/inode.c: In function 'apfs_do_ioc_setflags':
    /build/source/inode.c:1807:8: error: implicit declaration of function 'vfs_ioc_setflags_prepare' [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wimplicit-function-declaration-Werror=implicit-function-declaration8;;]
     1807 |  err = vfs_ioc_setflags_prepare(inode, oldflags, newflags);
          |        ^~~~~~~~~~~~~~~~~~~~~~~~
    /build/source/inode.c: In function 'apfs_ioc_setflags':
    /build/source/inode.c:1844:30: error: passing argument 1 of 'inode_owner_or_capable' from incompatible pointer type [8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wincompatible-pointer-types-Werror=incompatible-pointer-types8;;]
     1844 |  if (!inode_owner_or_capable(inode))
          |                              ^~~~~
          |                              |
          |                              struct inode *
    In file included from /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/buffer_head.h:12,
                     from /build/source/inode.c:7:
    /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/fs.h:1888:52: note: expected 'struct user_namespace *' but argument is of type 'struct inode *'
     1888 | bool inode_owner_or_capable(struct user_namespace *mnt_userns,
          |                             ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
    /build/source/inode.c:1844:7: error: too few arguments to function 'inode_owner_or_capable'
     1844 |  if (!inode_owner_or_capable(inode))
          |       ^~~~~~~~~~~~~~~~~~~~~~
    In file included from /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/buffer_head.h:12,
                     from /build/source/inode.c:7:
    /nix/store/iswyx074r0h6k9dlbf53mhwddwjb1ri8-linux-5.14.1-dev/lib/modules/5.14.1/source/include/linux/fs.h:1888:6: note: declared here
     1888 | bool inode_owner_or_capable(struct user_namespace *mnt_userns,
          |      ^~~~~~~~~~~~~~~~~~~~~~
    cc1: some warnings being treated as errors
    

    Let me know if I can provide any additional information.

    opened by Luflosi 8
  • Incompatible with kernels >= 5.19

    Incompatible with kernels >= 5.19

    Here is the build log of compiling the latest master version:

    make -C /nix/store/0rdnj4x5qa093cf5zi78knljwa05raw0-linux-5.19-dev/lib/modules/5.19.0/build M=/build/source
    make[1]: Entering directory '/nix/store/0rdnj4x5qa093cf5zi78knljwa05raw0-linux-5.19-dev/lib/modules/5.19.0/build'
      CC [M]  /build/source/btree.o
      CC [M]  /build/source/compress.o
      CC [M]  /build/source/dir.o
      CC [M]  /build/source/extents.o
      CC [M]  /build/source/file.o
    /build/source/file.c: In function 'apfs_file_mmap':
    /build/source/file.c:109:28: error: 'const struct address_space_operations' has no member named 'readpage'
      109 |         if (!mapping->a_ops->readpage)
          |                            ^~
    make[2]: *** [/nix/store/0rdnj4x5qa093cf5zi78knljwa05raw0-linux-5.19-dev/lib/modules/5.19.0/source/scripts/Makefile.build:249: /build/source/file.o] Error 1
    make[1]: *** [/nix/store/0rdnj4x5qa093cf5zi78knljwa05raw0-linux-5.19-dev/lib/modules/5.19.0/source/Makefile:1843: /build/source] Error 2
    make[1]: Leaving directory '/nix/store/0rdnj4x5qa093cf5zi78knljwa05raw0-linux-5.19-dev/lib/modules/5.19.0/build'
    make: *** [Makefile:16: default] Error 2
    

    Looks like someone wrote a patch (that I didn't test) for a slightly older version of this kernel module: https://bugs.launchpad.net/ubuntu/+source/linux-apfs-rw/+bug/1982115.

    opened by Luflosi 7
  • Fails to compile on Linux 5.12.11

    Fails to compile on Linux 5.12.11

    I'm trying to build this on NixOS with Linux 5.12.11 but the build fails. I don't think this issue has to do with NixOS as it compiled fine with an older kernel (5.11.x). Here is the build output from nix-build:

    this derivation will be built:
      /nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv
    building '/nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv'...
    unpacking sources
    unpacking source archive /nix/store/0br698lgx8dm5bss6n25ls4llfv2q0zj-source
    source root is source
    setting SOURCE_DATE_EPOCH to timestamp 1624192656 of file source/xfield.c
    patching sources
    applying patch /nix/store/20r89d87xwycl20cfcsv4d884qlcp0gq-9.patch
    patching file Makefile
    configuring
    no configure script, doing nothing
    building
    build flags: SHELL=/nix/store/kxj6cblcsd1qcbbxlmbswwrn89zcmgd6-bash-4.4-p23/bin/bash KERNELRELEASE=5.12.11 KERNEL_DIR=/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build INSTALL_MOD_PATH=\$\(out\)
    make -C /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build M=/build/source
    make[1]: Entering directory '/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build'
      CC [M]  /build/source/btree.o
      CC [M]  /build/source/compress.o
      CC [M]  /build/source/dir.o
      CC [M]  /build/source/extents.o
      CC [M]  /build/source/file.o
      CC [M]  /build/source/inode.o
      CC [M]  /build/source/key.o
      CC [M]  /build/source/message.o
      CC [M]  /build/source/namei.o
      CC [M]  /build/source/node.o
      CC [M]  /build/source/object.o
      CC [M]  /build/source/spaceman.o
      CC [M]  /build/source/super.o
    /build/source/super.c: In function 'apfs_attach_nxi':
    /build/source/super.c:1128:9: error: too few arguments to function 'lookup_bdev'
     1128 |  bdev = lookup_bdev(dev_name);
          |         ^~~~~~~~~~~
    In file included from /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/backing-dev.h:15,
                     from /build/source/super.c:6:
    /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/blkdev.h:1984:5: note: declared here
     1984 | int lookup_bdev(const char *pathname, dev_t *dev);
          |     ^~~~~~~~~~~
    make[3]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/scripts/Makefile.build:271: /build/source/super.o] Error 1
    make[2]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:1856: /build/source] Error 2
    make[1]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:215: __sub-make] Error 2
    make[1]: Leaving directory '/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build'
    make: *** [Makefile:16: default] Error 2
    error: builder for '/nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv' failed with exit code 2;
           last 10 log lines:
           > In file included from /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/backing-dev.h:15,
           >                  from /build/source/super.c:6:
           > /nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/include/linux/blkdev.h:1984:5: note: declared here
           >  1984 | int lookup_bdev(const char *pathname, dev_t *dev);
           >       |     ^~~~~~~~~~~
           > make[3]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/scripts/Makefile.build:271: /build/source/super.o] Error 1
           > make[2]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:1856: /build/source] Error 2
           > make[1]: *** [/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/source/Makefile:215: __sub-make] Error 2
           > make[1]: Leaving directory '/nix/store/23gik70gdmadraqxzas87fy40dvg13ph-linux-5.12.11-dev/lib/modules/5.12.11/build'
           > make: *** [Makefile:16: default] Error 2
           For full logs, run 'nix log /nix/store/mq0rxj1qff8ak2r08l8i9nw6n9af7axv-apfs-unstable-2020-08-06-5.12.11.drv'.
    

    This looks like something has changed in the kernel to which linux-apfs needs to be adapted to.

    opened by Luflosi 5
  • Failed to compile on RHEL 9 with Linux 5.14.0

    Failed to compile on RHEL 9 with Linux 5.14.0

    When I was trying to compile this, it reports that there was an error in super.c.

    Compilation log
    make -C /lib/modules/5.14.0-70.13.1.el9_0.x86_64/build M=/home/tienyu/linux-apfs-rw
    make[1]: Entering directory '/usr/src/kernels/5.14.0-70.13.1.el9_0.x86_64'
      CC [M]  /home/tienyu/linux-apfs-rw/btree.o
      CC [M]  /home/tienyu/linux-apfs-rw/compress.o
      CC [M]  /home/tienyu/linux-apfs-rw/dir.o
      CC [M]  /home/tienyu/linux-apfs-rw/extents.o
      CC [M]  /home/tienyu/linux-apfs-rw/file.o
      CC [M]  /home/tienyu/linux-apfs-rw/inode.o
      CC [M]  /home/tienyu/linux-apfs-rw/key.o
      CC [M]  /home/tienyu/linux-apfs-rw/message.o
      CC [M]  /home/tienyu/linux-apfs-rw/namei.o
      CC [M]  /home/tienyu/linux-apfs-rw/node.o
      CC [M]  /home/tienyu/linux-apfs-rw/object.o
      CC [M]  /home/tienyu/linux-apfs-rw/spaceman.o
      CC [M]  /home/tienyu/linux-apfs-rw/super.o
    /home/tienyu/linux-apfs-rw/super.c: In function 'apfs_setup_bdi':
    /home/tienyu/linux-apfs-rw/super.c:1073:31: error: 'struct block_device' has no member named 'bd_bdi'
     1073 |         bdi_dev = nxi->nx_bdev->bd_bdi;
          |                               ^~
    make[2]: *** [scripts/Makefile.build:271: /home/tienyu/linux-apfs-rw/super.o] Error 1
    make[1]: *** [Makefile:1862: /home/tienyu/linux-apfs-rw] Error 2
    make[1]: Leaving directory '/usr/src/kernels/5.14.0-70.13.1.el9_0.x86_64'
    make: *** [Makefile:16: default] Error 2
    

    However I looked into super.c then and found a piece of code which detects kernel version:

    #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 15, 0)
            bdi_dev = nxi->nx_bdev->bd_bdi;
    #else
            bdi_dev = nxi->nx_bdev->bd_disk->bdi;
    #endif
    

    I tried deleting the whole part and use bdi_dev = nxi->nx_bdev->bd_disk->bdi; (i.e. the one for Linux 5.15+) only and it compiled successfully. That's pretty wired since I'm on Red Hat Enterprise Linux 9 and my kernel version is 5.14.0.

    My kernel info (uname -a)
    Linux TienyusRHEL 5.14.0-70.13.1.el9_0.x86_64 #1 SMP PREEMPT Thu Apr 14 12:42:38 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux

    Is there any way to work it out? Thank you for your work!

    opened by lingrottin 4
  • Incompatible with kernels >= 5.18

    Incompatible with kernels >= 5.18

    DKMS make.log for linux-apfs-rw-0.2 for kernel 5.18.3-arch1-1 (x86_64)
    Mon Jun 13 23:06:19 PDT 2022
    make: Entering directory '/usr/lib/modules/5.18.3-arch1-1/build'
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/btree.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/compress.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/dir.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/extents.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/file.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/inode.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/message.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/key.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/namei.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/node.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/object.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/spaceman.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/super.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/symlink.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/transaction.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/unicode.o
      CC [M]  /var/lib/dkms/linux-apfs-rw/0.2/build/xattr.o
    /var/lib/dkms/linux-apfs-rw/0.2/build/inode.c:524:10: error: ‘const struct address_space_operations’ has no member named ‘invalidatepage’; did you mean ‘invalidate_folio’?
      524 |         .invalidatepage = apfs_noop_invalidatepage,
          |          ^~~~~~~~~~~~~~
          |          invalidate_folio
    /var/lib/dkms/linux-apfs-rw/0.2/build/inode.c:524:27: error: initialization of ‘sector_t (*)(struct address_space *, sector_t)’ {aka ‘long long unsigned int (*)(struct address_space *, long long unsigned int)’} from incompatible pointer type ‘void (*)(struct page *, unsigned int,  unsigned int)’ [-Werror=incompatible-pointer-types]
      524 |         .invalidatepage = apfs_noop_invalidatepage,
          |                           ^~~~~~~~~~~~~~~~~~~~~~~~
    /var/lib/dkms/linux-apfs-rw/0.2/build/inode.c:524:27: note: (near initialization for ‘apfs_aops.bmap’)
    cc1: some warnings being treated as errors
    make[1]: *** [scripts/Makefile.build:288: /var/lib/dkms/linux-apfs-rw/0.2/build/inode.o] Error 1
    make[1]: *** Waiting for unfinished jobs....
    make: *** [Makefile:1834: /var/lib/dkms/linux-apfs-rw/0.2/build] Error 2
    make: Leaving directory '/usr/lib/modules/5.18.3-arch1-1/build'
    

    Same git revision worked for 5.17.x kernels.

    opened by melvyn2 4
  • Clarify license

    Clarify license

    Source files currently contain headers with SPDX-License-Identifier: GPL-2.0. However, as per https://spdx.org/licenses/, there is no such license identifier, I am assuming it was meant to be "GPL-2.0-only" or "GPL-2.0-or-later". Please clarify which version is used by this project.

    opened by Dekedro 3
  • Feedback for NixOS test

    Feedback for NixOS test

    The last missing piece for perfect NixOS integration is a NixOS module. This will allow awesome things like specifying

    fileSystems."/mnt/apfs" = {
      device = "/dev/disk/by-partlabel/apfs";
      fsType = "apfs";
    }
    

    in the configuration, which is all that is required to automatically mount the filesystem during boot, no need to install anything or configure anything else. As part of writing a NixOS module, it is good practice to write a test for a basic sanity check and to gain some confidence, that at least the basic stuff all works as expected. Here is how the test looks currently:

    import ./make-test-python.nix ({ pkgs, ... }: {
      name = "apfs";
      meta.maintainers = with pkgs.lib.maintainers; [ Luflosi ];
    
      machine = { pkgs, ... }: {
        virtualisation.emptyDiskImages = [ 1024 ];
    
        boot.supportedFilesystems = [ "apfs" ];
      };
    
      testScript = ''
        machine.wait_for_unit("basic.target")
        machine.succeed("mkdir /tmp/mnt")
    
        with subtest("mkapfs refuses to work with a label that is too long"):
          machine.fail( "mkapfs -L '000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F' /dev/vdb")
    
        with subtest("mkapfs works with the maximum label length"):
          machine.succeed("mkapfs -L '000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7' /dev/vdb")
    
        with subtest("Enable case sensitivity and normalization sensitivity"):
          machine.succeed(
              "mkapfs -s -z /dev/vdb",
              # Triggers a bug, see https://github.com/linux-apfs/linux-apfs-rw/issues/15
              # "mount -o cknodes,readwrite /dev/vdb /tmp/mnt",
              "mount -o readwrite /dev/vdb /tmp/mnt",
              "echo 'Hello World 1' > /tmp/mnt/test.txt",
              "[ ! -f /tmp/mnt/TeSt.TxT ] || false", # Test case sensitivity
              "echo 'Hello World 1' | diff - /tmp/mnt/test.txt",
              "echo 'Hello World 2' > /tmp/mnt/\u0061\u0301.txt",
              "echo 'Hello World 2' | diff - /tmp/mnt/\u0061\u0301.txt",
              "[ ! -f /tmp/mnt/\u00e1.txt ] || false", # Test Unicode normalization sensitivity
              "umount /tmp/mnt",
              "apfsck /dev/vdb",
          )
        with subtest("Disable case sensitivity and normalization sensitivity"):
          machine.succeed(
              "mkapfs /dev/vdb",
              "mount -o readwrite /dev/vdb /tmp/mnt",
              "echo 'Hello World Hello World Hello World' > /tmp/mnt/Test.txt",
              "echo -n 'Hello World' > /tmp/mnt/test.txt",
              "echo ' 1' >> /tmp/mnt/TEST.TXT",
              "umount /tmp/mnt",
              "apfsck /dev/vdb",
              "mount -o readwrite /dev/vdb /tmp/mnt",
              "echo 'Hello World 1' | diff - /tmp/mnt/TeSt.TxT", # Test case insensitivity
              "echo 'Hello World 2' > /tmp/mnt/\u0061\u0301.txt",
              "echo 'Hello World 2' | diff - /tmp/mnt/\u0061\u0301.txt",
              "echo 'Hello World 2' | diff - /tmp/mnt/\u00e1.txt", # Test Unicode normalization
              "umount /tmp/mnt",
              "apfsck /dev/vdb",
          )
      '';
    })
    

    Most of the space in the file is taken up by a Python script. The strings passed to machine.succeed() are shell script one-liners that all need to succeed for the test to succeed. The test builds a qemu VM with an empty 1GB disk image, loads the APFS kernel module and then executes the tests. Here is the NixOS test for ZFS. It basically tests smb file sharing, encryption and forefully importing a pool on a machine with a different host ID, for two versions of ZFS and the Linux kernel. I'd like to hear your feedback, if you think this is an adequate amount of testing or if you would test more or test less? Should I maybe reboot the VM somewhere during the test? Maybe test long filenames? Call apfsck more often? Create a whole bunch of files and see if any disappear? Something else entirely?

    opened by Luflosi 2
  • /lib/modules/5.13.9-arch1-1/build: No such file or directory.  Stop.

    /lib/modules/5.13.9-arch1-1/build: No such file or directory. Stop.

    I am on Arch, so I simply downloaded linux-headers from pacman, cloned the repo, cd'd into the repo, ran make and got this error:

    make -C /lib/modules/5.13.9-arch1-1/build M=/home/nate/linux-apfs-rw
    make[1]: *** /lib/modules/5.13.9-arch1-1/build: No such file or directory.  Stop.
    make: *** [Makefile:15: default] Error 2
    

    Did I miss something?

    EDIT: in /lib/modules I have a folder called 5.13.12-arch1-1... Do I need that specific version for this to work?

    opened by n8jadams 2
  • Label does not seem to work

    Label does not seem to work

    The label of an APFS filesystem does not seem to show up anywhere. The steps I tried:

    1. modprobe apfs
    2. truncate -s 1G /tmp/test.img
    3. losetup -f /tmp/test.img
    4. mkapfs -L testing /dev/loop0
    5. lsblk -f does not show a label for loop0
    6. ls -la /dev/disk/by-label does not show loop0
    7. mounting the filesystem does not change the situation

    I don't think this is a problem with mkapfs because an APFS filesystem created with macOS also does not show a label in Linux and copying /tmp/test.img to a macOS machine and mounting it does show the label.

    This issue is probably very low on your TODO list. I just wanted to know if it is a known issue and if I did something wrong or if this is supposed to work.

    opened by Luflosi 2
  • Build errors with kver. 6.0.9

    Build errors with kver. 6.0.9

    Sorry for my bad franch, but i have this issue at modpost stage:

    ERROR: modpost: "__umoddi3" [fs/apfs/apfs.ko] undefined! ERROR: modpost: "__udivdi3" [fs/apfs/apfs.ko] undefined !

    And I dont know what do me do!

    inxi -Fxz: System: Kernel: 6.0.2-zen-060002-generic++ arch: i686 bits: 32 compiler: gcc v: 12.2.0 Desktop: N/A Distro: Debian GNU/Linux bookworm/sid Machine: Type: Laptop System: Acer product: TravelMate 5320 v: 0100 serial: Mobo: Acer model: Columbia v: Rev serial: BIOS: Phoenix v: 1.21 date: 10/05/2007 CPU: Info: dual core model: Intel Core2 Duo T7500 bits: 64 type: MCP arch: Core2 Merom rev: B cache: L1: 128 KiB L2: 4 MiB Speed (MHz): avg: 798 min/max: 800/2201 boost: enabled cores: 1: 798 2: 798 bogomips: 8779 Flags: ht lm nx pae sse sse2 sse3 ssse3 vmx Graphics: Device-1: Intel Mobile GM965/GL960 Integrated Graphics vendor: Acer Incorporated ALI driver: i915 v: kernel arch: Gen-4 bus-ID: 00:02.0 Display: x11 server: X.Org v: 1.21.1.4 with: Xwayland v: 22.1.5 driver: dri: crocus gpu: i915 note: X driver n/a resolution: 1280x800~60Hz API: OpenGL v: 2.1 Mesa 22.2.4 renderer: Mesa Intel 965GM (CL) direct render: Yes Audio: Device-1: Intel 82801H HD Audio vendor: Acer Incorporated ALI Realtek ALC268 codec driver: snd_hda_intel v: kernel bus-ID: 00:1b.0 Sound API: ALSA v: k6.0.2-zen-060002-generic++ running: yes Sound Server-1: PulseAudio v: 16.1 running: yes Sound Server-2: PipeWire v: 0.3.60 running: no Network: Device-1: Broadcom NetLink BCM5787M Gigabit Ethernet PCI Express vendor: Acer Incorporated ALI driver: tg3 v: kernel port: N/A bus-ID: 02:00.0 IF: enp2s0 state: up speed: 100 Mbps duplex: full mac: Device-2: Broadcom BCM4311 802.11b/g WLAN vendor: AMBIT Microsystem driver: b43-pci-bridge v: N/A bus-ID: 04:00.0 IF-ID-1: erspan0 state: down mac: IF-ID-2: gre0 state: down mac: IF-ID-3: gretap0 state: down mac: IF-ID-4: ip6tnl0 state: down mac: IF-ID-5: tunl0 state: down mac: IF-ID-6: wlan0 state: down mac: Bluetooth: Device-1: Cambridge Silicon Radio Bluetooth Dongle (HCI mode) type: USB driver: btusb v: 0.8 bus-ID: 5-1.4:4 Report: hciconfig ID: hci0 rfk-id: 1 state: up address: bt-v: 2.1 lmp-v: 4.0 Drives: Local Storage: total: 1.06 TiB used: 483.18 GiB (44.7%) ID-1: /dev/sda vendor: Hitachi model: HTS542516K9SA00 size: 149.05 GiB ID-2: /dev/sdb type: USB vendor: Western Digital model: WD50 00LPVX-80V0TT0 size: 465.76 GiB ID-3: /dev/sdc type: USB vendor: Western Digital model: WD5000LPVX-22V0TT0 size: 465.76 GiB Partition: ID-1: / size: 29.36 GiB used: 27.83 GiB (94.8%) fs: ext4 dev: /dev/sdc9 Swap: ID-1: swap-1 type: partition size: 4 GiB used: 38 MiB (0.9%) dev: /dev/sdc5 Sensors: System Temperatures: cpu: 51.0 C mobo: N/A Fan Speeds (RPM): N/A Info: Processes: 196 Uptime: 23m Memory: 1.95 GiB used: 1.53 GiB (78.5%) Init: SysVinit runlevel: 2 Compilers: gcc: 12.2.0 clang: 14.0.6-6 Packages: 6134 Shell: Bash v: 5.2.2 inxi: 3.3.23

    Early I saccifuly build etch kver with apfs as kernel module and thats work fine, but now ((__

    opened by sergiall 3
  • mount -t apfs device mountpoint

    mount -t apfs device mountpoint

    mounting apfs looks ok ...but the files inside the tree structure are almost empty I have mac's rootfs from a working virtual mac (I can play jessica rhaye singing bob dylan on youtube ; so it is really working) all the filenames in directory tree look quite normal ... but its contents are simply 0 I have just followed prescription make; modprobe libcrc... and insmod apfs.ko I am not aiming at modifying files in apfs ,but just willing to clone a apfs_mojave back into hfs_mojave using tar between the 2 filesytems under linux instead of using CCC carbone clone copy thank you

    opened by hoan3504 1
  • readwrite option not working

    readwrite option not working

    Thanks for this effort you are making by enabling apfs on Linux. I used your other tool to format an external hard disk, and everything seems to work. When I mount the newly formatted partition with:

    sudo mount -o readwrite /dev/sdc1 Backup 
    

    And I try to create anything either with sudo or as a regular user; it does not let me do that.

    
    mkdir: cannot create directory ‘hola’: Read-only file system
    

    PS. This is on Debian Unstable.

    opened by muammar 64
  • Mounting with -o cknodes,readwrite shows problem

    Mounting with -o cknodes,readwrite shows problem

    While writing a simple test for NixOS, I experimented a bit and found a problem. This is what I did:

    1. truncate -s 1G /tmp/tmp.img
    2. losetup -f /tmp/tmp.img
    3. mkapfs /dev/loop0
    4. mkdir /mnt/tmp
    5. mount -o cknodes,readwrite /dev/loop0 /mnt/tmp
    6. touch /mnt/tmp/testfile fails with "touch: cannot touch '/mnt/tmp/testfile': Bad message"
    7. "APFS (86g): bad checksum for the space manager" and "APFS (86g): aborting transaction" appear in dmesg

    Is this a known problem?

    opened by Luflosi 1
Owner
APFS for Linux
APFS for Linux
An Out-of-the-Box TensorRT-based Framework for High Performance Inference with C++/Python Support

An Out-of-the-Box TensorRT-based Framework for High Performance Inference with C++/Python Support

手写AI 1.5k Jan 5, 2023
Instagram's experimental performance oriented greenfield implementation of Python.

Welcome to Skybison! Skybison is experimental performance-oriented greenfield implementation of Python 3.8. It contains a number of performance optimi

Meta Experimental 288 Jan 3, 2023
Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI

High-Performance-Computing-Experiments Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI 实验结

Jiang Lu 1 Nov 27, 2021
BayesOpt: A toolbox for bayesian optimization, experimental design and stochastic bandits.

BayesOpt: A Bayesian optimization library BayesOpt is an efficient implementation of the Bayesian optimization methodology for nonlinear optimization,

Ruben Martinez-Cantin 341 Nov 29, 2022
Experimental OpenCL SPIR-V to OpenCL C translator

spirv2clc spirv2clc is an experimental OpenCL SPIR-V to OpenCL C translator currently targeting OpenCL 1.2 support. It can generate OpenCL C code equi

Kévin Petit 19 Oct 1, 2022
Read and write rosbag on a platform without ROS installed, using MQTT for message delivery.

EasyRosBag x86_64: Test on Ubuntu18.04 arm64 : Test on Ubuntu21.04(raspberry Pi4) Introducton ROS(Robot Operation System) is too fat!!! We do not ins

afei 9 May 23, 2022
A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

Light Gradient Boosting Machine LightGBM is a gradient boosting framework that uses tree based learning algorithms. It is designed to be distributed a

Microsoft 14.5k Jan 5, 2023
ldd as a tree with an option to bundle dependencies into a single folder

libtree A tool that: ?? turns ldd into a tree ☝️ explains why shared libraries are found and why not ?? optionally deploys executables and dependencie

Harmen Stoppels 1.6k Dec 31, 2022
C++11 wrapper for the LMDB embedded B+ tree database library.

lmdb++: a C++11 wrapper for LMDB This is a comprehensive C++ wrapper for the LMDB embedded database library, offering both an error-checked procedural

D.R.Y. C++ 263 Dec 27, 2022
Reactive Light Training Module used in fitness for developing agility and reaction speed.

Hello to you , Thanks for taking interest in this project. Use case of this project is to help people that want to improve their agility and reactio

null 6 Dec 14, 2022
A simple facial recognition script using OpenCV's FaceRecognizer module implemented in C++

Local Binary Patterns Histogram Recognizer A proyect that implements the LBPHRecognizer class of the OpenCV library to determine if a detected face co

Pablo Agustín Ortega-Kral 0 Jan 18, 2022
The module for my life story archive that gives data and statistics for the family Kindle Fire.

By: Top README.md Read this article in a different language Sorted by: A-Z Sorting options unavailable ( af Afrikaans Afrikaans | sq Shqiptare Albania

Sean P. Myrick V19.1.7.2 2 Oct 24, 2022
A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.

Libonnx A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support. Getting Started The library's

xboot.org 442 Dec 25, 2022
Deep Learning API and Server in C++11 support for Caffe, Caffe2, PyTorch,TensorRT, Dlib, NCNN, Tensorflow, XGBoost and TSNE

Open Source Deep Learning Server & API DeepDetect (https://www.deepdetect.com/) is a machine learning API and server written in C++11. It makes state

JoliBrain 2.4k Dec 30, 2022
A PoC for requesting HWIDs directly from hardware, skipping any potential hooks or OS support.

PCIBan A PoC for requesting HWIDs directly from hardware, skipping any potential hooks or OS support. This is probably very unsafe, not supporting edg

null 62 Dec 28, 2022
Support Yolov4/Yolov3/Centernet/Classify/Unet. use darknet/libtorch/pytorch to onnx to tensorrt

ONNX-TensorRT Yolov4/Yolov3/CenterNet/Classify/Unet Implementation Yolov4/Yolov3 centernet INTRODUCTION you have the trained model file from the darkn

null 172 Dec 29, 2022
Benchmark framework of 3D integrated CIM accelerators for popular DNN inference, support both monolithic and heterogeneous 3D integration

3D+NeuroSim V1.0 The DNN+NeuroSim framework was developed by Prof. Shimeng Yu's group (Georgia Institute of Technology). The model is made publicly av

NeuroSim 11 Dec 15, 2022
ROS2 packages based on NVIDIA libArgus library for hardware-accelerated CSI camera support.

Isaac ROS Argus Camera This repository provides monocular and stereo nodes that enable ROS developers to use cameras connected to Jetson platforms ove

NVIDIA Isaac ROS 35 Dec 10, 2022
Code generation for automatic differentiation with GPU support.

Code generation for automatic differentiation with GPU support.

Eric Heiden 41 Dec 5, 2022