Using the global monitor to generate an event on unlock of a spinlock does not work with the MMU disabled. This fixes a regression from 9c795fbdbf445d144d331ff2a19a0f42fe0fc190 and an incomplete fix from 816388325262e5c7dd681c8ac6392248b6ca9f0e.

Bisected and fix confirmed by Sven.

Signed-off-by: Janne Grunau [email protected]

Here's the log when I directly boot the Linux. Although the Linux can boot up, the log on the host side shows like this.

zec@ubuntu-zec:~/GitTools/m1n1/proxyclient/tools$ sudo M1N1DEVICE=/dev/ttyACM0 python3.9 linux.py -b 'earlycon console=SAC0,1500000 console=tty0 debug' asahi_Image.gz asahi.dtb initramfs-busybox.cpio.gz

TTY> CPU init... CPU: M1 Icestorm TTY> TTY> boot_args at 0x808e08000 TTY> revision: 2 TTY> version: 2 TTY> virt_base: 0xfffffe001b9c4000 TTY> phys_base: 0x8019c4000 TTY> mem_size: 0x1d7a20000 TTY> top_of_kdata: 0x808e0c000 TTY> video: TTY> base: 0x9df4e4000 TTY> display: 0x1 TTY> stride: 0x2800 TTY> width: 2560 TTY> height: 1600 TTY> depth: 0x1001e TTY> machine_type: 0 TTY> devtree: 0xfffffe001e508000 TTY> devtree_size: 0x5c000 TTY> cmdline:
TTY> boot_flags: 0x0 TTY> mem_size_act: 0x200000000 TTY> TTY> TTY> TTY> m1n1 v63e8d0d-dirty TTY> Copyright (C) 2021 The Asahi Linux Contributors TTY> Licensed under the MIT license TTY> TTY> Running in EL2 TTY> TTY> Heap base: 0x808e0c000 TTY> MMU: Initializing... TTY> MMU: SCTLR_EL1: 30100180 -> 30901085 TTY> MMU: running with MMU and caches enabled! TTY> fb init: 2560x1600 (30) [s=2560] @0x9df4e4000 TTY> fb console: max rows 46, max cols 64 TTY> fb: display logo TTY> Device info: TTY> Model: MacBookAir10,1 TTY> Target: J313 TTY> Board-ID: 0x26 TTY> Chip-ID: 0x8103 TTY> TTY> WDT registers @ 0x23d2b0000 TTY> WDT disabled TTY> pmgr: initialized, 268 devices found. TTY> Initialization complete. TTY> Checking for payloads... TTY> No more payloads at 0x8046b8000 TTY> No valid payload found TTY> USB0: initialized at 0x808e0c030 TTY> USB1: initialized at 0x808e0c440 TTY> Running proxy... m1n1 base: 0x8045c0000 Setting boot args: "earlycon console=ttySAC0,1500000 console=tty0 debug" Loading 6364959 bytes to 0x811840200..0x811e5211f... ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Loading DTB to 0x811e52140... Kernel_base: 0x812000000 Loading 2024993 initramfs bytes to 0x832000000... ........................................................................................................................................................................................................................................................ TTY> Starting secondary CPUs... TTY> Starting CPU 1 (0:1)... Started. TTY> Starting CPU 2 (0:2)... Started. TTY> Starting CPU 3 (0:3)... Started. TTY> Starting CPU 4 (1:0)... Started. TTY> Starting CPU 5 (1:1)... Started. TTY> Starting CPU 6 (1:2)... Started. TTY> Starting CPU 7 (1:3)... Started. TTY> FDT: bootargs = 'earlycon console=ttySAC0,1500000 console=tty0 debug' TTY> FDT: initrd at 0x809730000 size 0x93e824 TTY> FDT: framebuffer@9df4e4000 base 0x9df4e4000 size 0xfa0000 TTY> ADT: 64 bytes of random seed available TTY> FDT: KASLR seed initialized TTY> FDT: Passing 64 bytes of random seed TTY> FDT: DRAM at 0x800000000 size 0x200000000 TTY> FDT: Usable memory is 0x8019c4000..0x9d93e4000 (0x1d7a20000) TTY> FDT: CPU 1 MPIDR=0x1 release-addr=0x804690110 TTY> FDT: CPU 2 MPIDR=0x2 release-addr=0x804690150 TTY> FDT: CPU 3 MPIDR=0x3 release-addr=0x804690190 TTY> FDT: CPU 4 MPIDR=0x10100 release-addr=0x8046901d0 TTY> FDT: CPU 5 MPIDR=0x10101 release-addr=0x804690210 TTY> FDT: CPU 6 MPIDR=0x10102 release-addr=0x804690250 TTY> FDT: CPU 7 MPIDR=0x10103 release-addr=0x804690290 TTY> FDT prepared at 0x80a070000 Uncompressing gz ... 15096320 Decompress OK... Ready to boot DAIF: c0 TTY> tunable: unable to find ADT node /arm-io/apcie/pci-bridge1. TTY> pcie: Error applying apcie-config-tunables for /arm-io/apcie/pci-bridge1 TTY> Preparing to boot kernel at 0x812000000 with fdt at 0x80a070000 --- TTY mode | Quit: CTRL+] | Menu: CTRL+T --- Preparing to run next stage at 0x812000000... Exception in thread rx: Traceback (most recent call last): File "/usr/local/bin/lib/python3.9/threading.py", line 973, in _bootstrap_inner --- Exit TTY mode --- self.run() File "/usr/local/bin/lib/python3.9/threading.py", line 910, in run self._target(*self._args, **self._kwargs) File "/usr/local/bin/lib/python3.9/site-packages/serial/tools/miniterm.py", line 499, in reader data = self.serial.read(self.serial.in_waiting or 1) File "/usr/local/bin/lib/python3.9/site-packages/serial/serialposix.py", line 595, in read raise SerialException( serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

It seems that some mistakes happen.

Then I want to boot the Linux on m1n1 hypervisor. The log shows like this.

zec@ubuntu-zec:~/GitTools/m1n1/proxyclient/tools$ sudo M1N1DEVICE=/dev/ttyACM1 python3.9 run_guest.py -S m1n1-payload.macho TTY> CPU init... CPU: M1 Icestorm TTY> TTY> boot_args at 0x808e18000 TTY> revision: 2 TTY> version: 2 TTY> virt_base: 0xfffffe00119c4000 TTY> phys_base: 0x8019c4000 TTY> mem_size: 0x1d7a20000 TTY> top_of_kdata: 0x808e1c000 TTY> video: TTY> base: 0x9df4e4000 TTY> display: 0x1 TTY> stride: 0x2800 TTY> width: 2560 TTY> height: 1600 TTY> depth: 0x1001e TTY> machine_type: 0 TTY> devtree: 0xfffffe0014518000 TTY> devtree_size: 0x5c000 TTY> cmdline:
TTY> boot_flags: 0x0 TTY> mem_size_act: 0x200000000 TTY> TTY> TTY> TTY> m1n1 v63e8d0d-dirty TTY> Copyright (C) 2021 The Asahi Linux Contributors TTY> Licensed under the MIT license TTY> TTY> Running in EL2 TTY> TTY> Heap base: 0x808e1c000 TTY> MMU: Initializing... TTY> MMU: SCTLR_EL1: 30100180 -> 30901085 TTY> MMU: running with MMU and caches enabled! TTY> fb init: 2560x1600 (30) [s=2560] @0x9df4e4000 TTY> fb console: max rows 46, max cols 64 TTY> fb: display logo TTY> Device info: TTY> Model: MacBookAir10,1 TTY> Target: J313 TTY> Board-ID: 0x26 TTY> Chip-ID: 0x8103 TTY> TTY> WDT registers @ 0x23d2b0000 TTY> WDT disabled TTY> pmgr: initialized, 268 devices found. TTY> Initialization complete. TTY> Checking for payloads... TTY> No more payloads at 0x8046c8000 TTY> No valid payload found TTY> USB0: initialized at 0x808e1c030 TTY> USB1: initialized at 0x808e1c440 TTY> Running proxy... Fetching ADT (0x0005C000 bytes)... Disable iodev IODEV.USB1 Initializing hypervisor over iodev IODEV.USB0 TTY> Starting secondary CPUs... TTY> Starting CPU 1 (0:1)... Started. TTY> Starting CPU 2 (0:2)... Started. TTY> Starting CPU 3 (0:3)... Started. TTY> Starting CPU 4 (1:0)... Started. TTY> Starting CPU 5 (1:1)... Started. TTY> Starting CPU 6 (1:2)... Started. TTY> Starting CPU 7 (1:3)... Started. Removing ADT node /arm-io/dart-usb0 Removing ADT node /arm-io/atc-phy0 Removing ADT node /arm-io/usb-drd0 Removing ADT node /arm-io/acio0 Removing ADT node /arm-io/acio-cpu0 Removing ADT node /arm-io/dart-acio0 Removing ADT node /arm-io/apciec0 Removing ADT node /arm-io/dart-apciec0 Removing ADT node /arm-io/apciec0-piodma Removing ADT node /arm-io/i2c0/hpmBusManager/hpm0 Removing ADT node /arm-io/atc0-dpxbar Removing ADT node /arm-io/atc0-dpphy Removing ADT node /arm-io/atc0-dpin0 Removing ADT node /arm-io/atc0-dpin1 Removing ADT node /arm-io/atc-phy0 LOAD: _HDR 16384 bytes from 0 to 0 LOAD: TEXT 131072 bytes from 4000 to 4000 LOAD: RODA 32768 bytes from 24000 to 24000 LOAD: DATA 393216 bytes from 2c000 to 2c000 ZERO: 1884160 bytes from 0x8c000 to 0x258000 LOAD: PYLD 8561277 bytes from 8c000 to 258000 SKIP: 58547587 bytes from 0xa8227d to 0x4258000 Total region size: 0x11d8000 bytes Physical memory: 0x819740000 .. 0x9d93e4000 Guest region start: 0x81a7f8000 Mapping guest physical memory... Loading kernel image (0xa82281 bytes)... ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Copying SEPFW (0x750000 bytes)... Copying TrustCache (0x5c000 bytes)... Adjusting addresses in ADT... Setting up bootargs at 0x81b9cc000... Setting secondary CPU RVBARs... cpu1: [0x210150000] = 0x81a7fc000 cpu2: [0x210250000] = 0x81a7fc000 cpu3: [0x210350000] = 0x81a7fc000 cpu4: [0x211050000] = 0x81a7fc000 cpu5: [0x211150000] = 0x81a7fc000 cpu6: [0x211250000] = 0x81a7fc000 cpu7: [0x211350000] = 0x81a7fc000 Entering hypervisor shell. Type start to start the guest.

start Disabling other iodevs...

• IODEV.UART
• IODEV.FB
• IODEV.USB1
• IODEV.USB0_SEC
• IODEV.USB1_SEC Doing essential MMIO remaps... Updating page tables... PT[200000000:235200000] -> HW PT[235200000:235204000] -> RESERVED VUART PT[235204000:23b700420] -> HW PT[23b700420:23b700424] -> RESERVED PMGR HACK PT[23b700424:23b754000] -> HW PT[23b754000:23b754010] -> RESERVED CPU_START PT[23b754010:23d280088] -> HW PT[23d280088:23d28008c] -> RESERVED PMGR HACK PT[23d28008c:23d280098] -> HW PT[23d280098:23d28009c] -> RESERVED PMGR HACK PT[23d28009c:700000000] -> HW Uploading ADT (0x51110 bytes)... Improving logo... Shutting down framebuffer... Enabling SPRR... Enabling GXF... Jumping to entrypoint at 0x81a7fc800 TTY> Exception taken from EL2h TTY> Running in EL2 TTY> MPIDR: 0x80000000 TTY> Registers: (@0x8046c7b30) TTY> x0-x3: 0000000000000000 000000080465b108 0000000000000001 0000000000000002 TTY> x4-x7: 000000081a7fc800 0000000000000000 0000000000000005 000000000000c40c TTY> x8-x11: 00000000000000be 0000000000000010 00000000000000be 0000000000000014 TTY> x12-x15: 0000000804660038 0000000000010000 00000000000003a0 0000000000000000 TTY> x16-x19: 00000008045e6428 0000000000000000 0000000000000003 0000000804670000 TTY> x20-x23: 0000000000000001 00040008045dfe22 00000008046c7d20 00000008045dfe20 TTY> x24-x27: 000000081a7fc908 00000008045dfe20 00000000b9002020 0000000000000000 TTY> x28-x30: 00000000addedbad 00000008046c7c30 00000008045dfe74 TTY> PC: 0x8045e084c (rel: 0x1084c) TTY> SP: 0x8046c7c30 TTY> SPSR_EL1: 0x600003c9 TTY> FAR_EL1: 0x0 TTY> ESR_EL1: 0x96000006 (data abort (current)) TTY> L2C_ERR_STS: 0x11000ffc00000000 TTY> L2C_ERR_ADR: 0x0 TTY> L2C_ERR_INF: 0x0 TTY> E_LSU_ERR_STS: 0x0 TTY> E_FED_ERR_STS: 0x0 TTY> E_MMU_ERR_STS: 0x0 Traceback (most recent call last): File "/usr/local/bin/lib/python3.9/code.py", line 90, in runcode exec(code, self.locals) File "", line 1, in File "/home/zec/GitTools/m1n1/proxyclient/m1n1/shell.py", line 129, in display val() File "/home/zec/GitTools/m1n1/proxyclient/m1n1/hv.py", line 1297, in start self.p.hv_start(self.entry, self.guest_base + self.bootargs_off) File "/home/zec/GitTools/m1n1/proxyclient/m1n1/proxy.py", line 950, in hv_start return self.request(self.P_HV_START, entry, *args) File "/home/zec/GitTools/m1n1/proxyclient/m1n1/proxy.py", line 624, in request return self._request(opcode, *args2, **kwargs) File "/home/zec/GitTools/m1n1/proxyclient/m1n1/proxy.py", line 590, in _request reply = self.iface.proxyreq(req, reboot=reboot, no_reply=no_reply, pre_reply=None) File "/home/zec/GitTools/m1n1/proxyclient/m1n1/proxy.py", line 357, in proxyreq return self.reply(self.REQ_PROXY) File "/home/zec/GitTools/m1n1/proxyclient/m1n1/proxy.py", line 230, in reply reply += self.readfull(1) File "/home/zec/GitTools/m1n1/proxyclient/m1n1/proxy.py", line 167, in readfull block = self.dev.read(size - len(d)) File "/usr/local/bin/lib/python3.9/site-packages/serial/serialposix.py", line 595, in read raise SerialException( serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

Some problem about serial things?

This isn't particularly useful by itself, but I have another patch which modifies m1n1 so it tries to run the proxy first, then falls back to loading the payload only after it times out. That, I think, is useful, since it means you can have a bootable M1 that you can still start with a different kernel.

• [x] cleanup code some more
• [x] add font license and sign-off the commit
• [x] integrate makefont.sh into Makefile
• [x] figure out how to deal with the logo

I've tried following the QuickStart instructions to the dot on a MacBook Pro with M1 Pro. The mac came with macOS monteray pre-installed.

However, I hit a snag when trying to use kmutil:

-bash-3.2# kmutil configure-boot -c m1n1.macho -C -v /Volumes/Linux
By setting a custom boot object, you will be putting your system into Permissive Security.

Are you sure you want to do this? (enter y or n) y
found valid Mach-O binary, deriving Mach-O boot properties...
Error Domain=KMErrorDomain Code=71 "Fileset Error: Invalid Mach-O boot properties" UserInfo={NSLocalizedDescription=Fileset Error: Invalid Mach-O boot properties}

Any ideas?

This is a bit of a weird one, but I'm using vanilla m1n1 and can reproduce this at will. Here's a log that demonstrates what's going on:

TTY> CPU init... CPU: M1 Icestorm
TTY> 
TTY> boot_args at 0x807f38000
TTY>   revision:     2
TTY>   version:      2
TTY>   virt_base:    0xfffffe0007f44000
TTY>   phys_base:    0x801f44000
TTY>   mem_size:     0x3d7554000
TTY>   top_of_kdata: 0x807f3c000
TTY>   video:
TTY>     base:       0xbdf438000
TTY>     display:    0x0
TTY>     stride:     0x2800
TTY>     width:      2560
TTY>     height:     1600
TTY>     depth:      65566bpp
TTY>   machine_type: 0
TTY>   devtree:      0xfffffe00096a0000
TTY>   devtree_size: 0x5c000
TTY>   cmdline:      -v
TTY>   boot_flags:   0x0
TTY>   mem_size_act: 0x400000000
TTY> 
TTY> 
TTY> 
TTY> m1n1 v7c54aa3-dirty
TTY> Copyright (C) 2021 The Asahi Linux Contributors
TTY> Licensed under the MIT license
TTY> 
TTY> Running in EL2
TTY> 
TTY> Heap base: 0x807f3c000
TTY> MMU: Initializing...
TTY> MMU: SCTLR_EL1: 30100180 -> 30901085
TTY> MMU: running with MMU and caches enabled!
TTY> fb init: 2560x1600 (30) [s=2560] @0xbdf438000
TTY> fb console: max rows 46, max cols 64
TTY> fb: display logo
TTY> Device info:
TTY>   Model: MacBookPro17,1
TTY>   Target: J293
TTY>   Board-ID: 0x24
TTY>   Chip-ID: 0x8103
TTY> 
TTY> WDT registers @ 0x23d2b0000
TTY> WDT disabled
TTY> pmgr: initialized, 272 devices found.
TTY> Initialization complete.
TTY> Checking for payloads...
TTY> No more payloads at 0x803854000
TTY> No valid payload found
TTY> i2c: want to read 1 bytes from addr 63 but can only read 0
TTY> usb: tps6598x_powerup failed for /arm-io/i2c0/hpmBusManager/hpm1.
TTY> usb: failed to init hpm1
TTY> USB0: initialized at 0x807f3c030
TTY> USB1: initialized at 0x807f3c440
TTY> Running proxy...
m1n1 base: 0x80375c000
Have fun!
>>> get_bootargs()
0x807f38000
>>> read64(get_bootargs()+16)
0x801f44000
>>> read64(get_bootargs()+24)
0x3d7554000
>>> top_of_mem = 0x801f44000 + 0x3d7554000
>>> readmem(top_of_mem-8, 8)
Traceback (most recent call last):
  File "/usr/lib/python3.9/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<console>", line 1, in <module>
  File "/home/pip/metalinux/m1n1/proxyclient/m1n1/proxy.py", line 400, in readmem
    raise UartChecksumError(f"Reply data sentinel error: Expected "
m1n1.proxy.UartChecksumError: Reply data sentinel error: Expected 0xb0cacc10, got 0x45cacc10

What happens on the console is an SError with ESR 0xbe000000 and L2C_ERR_ADR 0x300000bd9498000, which is the top_of_mem address.

(The "65566 bpp" thing and the i2c error are real, too, I'm afraid)

I guess this all makes sense since we did create a "normal" mapping for the poisoned page that starts unavailable "memory", but I'm not sure how best to fix it. I think what happens is the CPU speculatively prefetches from the mapped-but-inaccessible page. The ringbuffer read code, which is where the SError hits, does look like it would trigger such a prefetch by reading bytes sequentially (which is why it might not be possible to reproduce this using a UART, come to think of it).

Image Signal Processor is a co-processor in charge of Facetime camera in Apple laptops. This tracer is based on ISP found in M1 processor.

The ISP in M1 SoC exposes seven inter-process communication channels. Using those channels Application Processor (AP) can perform a variety of task in the ISP and viceversa. All channels seems to be bidirectional, with unique exception of TERMINAL channel.

DART translation is used often to translate control structures passed over IPC channels to IO virtual addresses.

How those channels are used is still unclear but tracer should allow us to dive deep on high-level protocol to communicate with Facetime camera. ISP exposes also some I2C peripheral and SPMI registers but I haven't dive into their working or purpose.

This is early implementation, that mostly dumps ISP internal log buffer and memory allocation calls.

Restore the interrupt masks on transition to the next state. The masks are not restored on the USB-C port used by the hypervisor. This prevents an interrupt storm in the guest when the other USB-C port is exposed to the guest. The tps6598x interrupt line is shared amoung both ports.

There are at least two types in the ADT related to USB, but there's a decent chance that there are even more required for other devices:

• A simple tunable that applies to a whole device node and all it's MMIO ranges specified in the "reg" property. This one seems to just be mask32.

• A slightly more complex tunable that applies to a single MMIO range specified in the "reg" property. So far I've only seen 32 bit masks but the format looks like it should also support 8,16 and 64 bit masks.

Open question: How do we expose proxyclient ops for these? Unlike everything else they take a string as an argument and right now there's no way to easily serialize those.

Signed-off-by: Sven Peter [email protected]

Add an explicit wait for monitor reset in display.c, triggered by the display=wait (or display=wait,<mode…>) boot parameter.

TODO:

• ~~Check that it works on monitors other than mine~~
• ~~Check that it works with payloads~~
• Maybe improve parameter parsing so that the order of parameters doesn’t matter
• ~~Currently the wait for the monitor to disconnect/reconnect is done between dcp_ib_swap_begin and dcp_ib_swap_set_layer.~~ The wait is now done before the swap, relying on stage1 m1n1 to wake up the monitor. However, this will fail to initialize the monitor if too much time passes between stage1 m1n1 and stage2 m1n1, allowing the monitor to fully go back to sleep between the two. That is probably not a huge concern.

For those of us who can't write assembly but still need to run code on the other side.

  >>> from m1n1.loadobjs import *
  >>> lp = LinkedProgram(u)
  >>> lp.debug_printf("hello from the other side (%d)\n", 42)
  TTY> hello from the other side (42)
  0x1f
  >>> lp.load_obj("build/test.o")
  >>> lp.test()
  TTY> hello from loaded object
  0x19
  >>> lp.clear_objs()

--

I don't know if anybody else will find this useful but as it doesn't hook into anything maybe it can live in mainline.

• use /vram for the framebuffer, if m1n1 reallocates the framebuffer due to higher configured resolution only /vram is updated
• add memory-region-names properties so the DCP driver can get the framebuffer. There are two use cases for this:
  1. with drm_aperture_remove_conflicting_framebuffers() replacing simpledrm's kms device seems to work reliable
  2. might be required for seamless switching from simpledrm to dcp (I have not checked how that is supposed to work)
• use unique compatible strings for each reserved memory region

This adds an initial config so we'll get per-commit and per-PR package builds published at https://copr.fedorainfracloud.org/coprs/g/asahi/packit-builds/ which should make testing easier and provide useful signal in case of regressions.

To support modules during development with run_guest_kernel.sh a cpio archive with matching modules can be created with make dir-pkg and usr/gen_initramfs.sh in the kernel dir. A specially prepared initramfs can use the modules and make them available to the OS in a similar way as the vendor firmware.

Small improvement to the out-of-the-box experience when using maOS as tethered boot host

• use /dev/cu.usbmodemP_03 and /dev/cu.usbmodemP_01 for picocom-sec.sh and proxyclient respectively

When using

unset CFLAGS CXXFLAGS CPPFLAGS LDFLAGS
make RELEASE=1 ARCH=${CROSS_BUILD:+${XBPS_CROSS_TRIPLET}-} build/m1n1.bin

I noticed

CC    build/kboot.o
src/kboot.c: In function 'dt_append_atc_tunable':
src/kboot.c:812:51: warning: comparison of integer expressions of different signedness: 'int' and 'size_t' {aka 'const long unsigned int'} [-Wsign-compare]
  812 |         if (tunable->offset + (tunable->size / 8) > tunable_info->reg_size) {
      |                                                   ^
/bin/sh: line 1: git: command not found

as well as

CC    build/tinf/tinflate.o
src/tinf/tinflate.c: In function 'tinf_uncompress':
src/tinf/tinflate.c:559:5: warning: stack usage is 1712 bytes [-Wstack-usage=]
  559 | int tinf_uncompress(void *dest, unsigned int *destLen,
      |     ^~~~~~~~~~~~~~~

The full log https://paste.c-net.org/LaurenceNuthouse The template used to build https://github.com/void-linux/void-packages/pull/39796/commits/079293f3f8f01d308cd77a889d083b0cfca56a7a#diff-7317415753e90f594723d1d7e5dd70eb72d51f299d932ebef79986d96c12b5c2