Skip to content

Harden runtime and tests for Apple Silicon hosts#30

Merged
jserv merged 1 commit into
mainfrom
fix-apple-silicon
May 13, 2026
Merged

Harden runtime and tests for Apple Silicon hosts#30
jserv merged 1 commit into
mainfrom
fix-apple-silicon

Conversation

@jserv
Copy link
Copy Markdown
Contributor

@jserv jserv commented May 12, 2026

runtime_set_process_title rewrote argv[0] through the end of envp using memcpy and memset. Apple libc on M-series can lower those into cache-line-aligned stp ladders and DC ZVA stores that step past the explicit byte count when the destination tail sits at the host stack ceiling under a small RLIMIT_STACK. The overshoot crossed the guard page and delivered SIGSEGV before main() could even reach the vCPU run loop, so the binary appeared to crash with no syscall trace.

Walk only the contiguous argv block (stop at the first non-contiguous argv[i] or first NULL) and overwrite it through a volatile char pointer one byte at a time, so the compiler cannot fold the loop into memset. envp is no longer touched, which also removes the need to duplicate the environment block. argc bounds and the title-vs-avail arithmetic remain unchanged.

main.c now reads DCZID_EL0 at startup and aborts if BS != 4. The shim emulates each trapped DC ZVA by zeroing exactly 64 bytes, while guest libc reads DCZID_EL0 directly (no MRS trap) and uses its value as the stride for memset(0). Every Apple M1 through M4 reports BS=4 (64 bytes), but a future host that bumps the granule would cause silent partial-zero corruption of guest memory. Fail closed at startup with a pointer to src/core/shim.S instead of letting the mismatch surface as data corruption later. DZP=1 hosts are also rejected.

mk/shim.mk now inspects the output of OBJCOPY -O binary for residual Mach-O magics (MH_MAGIC / MH_MAGIC_64 / FAT, both byte orders). Apple ships an objcopy that leaves the Mach-O container in place; only GNU binutils objcopy reliably extracts the raw aarch64 section. The check deletes the bogus shim.bin and prints a recovery hint instead of silently embedding a Mach-O header into shim_blob.h.

tests/lib/test-runner.sh now resolves timeout(1) at source time. macOS does not ship timeout, so the runner falls back to gtimeout on PATH or the Homebrew opt symlinks under both arm64 and x86_64 prefixes, and exposes the resolved binary through a shell function so callers keep using the bare name. TIMEOUT_BIN overrides the search.

tests/test-busybox.sh probes busybox --list under elfuse once at start and exposes the applet set through BB_APPLETS. test_skip_missing_tool is overridden to consult that list, so reduced busybox builds (Debian busybox-static drops a handful of applets) yield SKIP instead of FAIL. The probe hard-fails if --list itself fails so a broken elfuse cannot silently degrade the whole suite to SKIP.

tests/test-proctitle-low-stack.sh exercises the proctitle rewrite path under a capped stack and wires into make check via mk/tests.mk. The test runs through elfuse busybox echo and verifies both that the run completes and that the expected output is preserved.


Summary by cubic

Hardened Apple Silicon runtime to prevent low‑stack crashes during argv title rewrite and to fail fast on DC ZVA mismatches. Made the shim build and tests reliable on macOS.

  • Bug Fixes

    • Rewrite process title only within the contiguous argv block and byte‑by‑byte via a volatile pointer; stop touching envp to avoid guard‑page SIGSEGV under small stacks.
    • Assert DCZID_EL0 at startup. Reject DZP=1 or BS != 4 and exit with a clear message pointing to src/core/shim.S.
    • In mk/shim.mk, detect Mach‑O magics after OBJCOPY -O binary; delete bad shim.bin and print guidance to use GNU objcopy.
    • Test runner resolves timeout at source time. Falls back to gtimeout (PATH or Homebrew), or honors TIMEOUT_BIN.
  • Refactors

    • Busybox tests: probe --list once, store in BB_APPLETS, and skip applets not compiled in. Hard‑fail if the probe fails.
    • Added tests/test-proctitle-low-stack.sh and wired it into make check to validate the low‑stack argv rewrite path.

Written for commit 51adfb7. Summary will update on new commits.

@jserv
Copy link
Copy Markdown
Contributor Author

jserv commented May 12, 2026

@devarajabc : Help validate on Apple M2 and M5 based machines
@Max042004 : Help validate on Apple M4 based machines

Steps:

make distclean
make elfuse
make test-busybox

cubic-dev-ai[bot]

This comment was marked as resolved.

@Max042004
Copy link
Copy Markdown
Collaborator

make test-busybox both runs correctly on M4 Sequoia 15.4.1 and Tahoe 26.4.1

runtime_set_process_title rewrote argv[0] through the end of envp using
memcpy and memset. Apple libc on M-series can lower those into cache-line-aligned
stp ladders and DC ZVA stores that step past the explicit byte count when
the destination tail sits at the host stack ceiling under a small RLIMIT_STACK.
The overshoot crossed the guard page and delivered SIGSEGV before main()
could even reach the vCPU run loop, so the binary appeared to crash with
no syscall trace.

Walk only the contiguous argv block (stop at the first non-contiguous
argv[i] or first NULL) and overwrite it through a volatile char pointer
one byte at a time, so the compiler cannot fold the loop into memset.
envp is no longer touched, which also removes the need to duplicate the
environment block. argc bounds and the title-vs-avail arithmetic remain
unchanged.

main.c now reads DCZID_EL0 at startup and aborts if BS != 4. The shim
emulates each trapped DC ZVA by zeroing exactly 64 bytes, while guest
libc reads DCZID_EL0 directly (no MRS trap) and uses its value as the
stride for memset(0). Every Apple M1 through M4 reports BS=4 (64 bytes),
but a future host that bumps the granule would cause silent partial-zero
corruption of guest memory. Fail closed at startup with a pointer to
src/core/shim.S instead of letting the mismatch surface as data
corruption later. DZP=1 hosts are also rejected.

mk/shim.mk now inspects the output of OBJCOPY -O binary for residual
Mach-O magics (MH_MAGIC / MH_MAGIC_64 / FAT, both byte orders). Apple
ships an objcopy that leaves the Mach-O container in place; only GNU
binutils objcopy reliably extracts the raw aarch64 section. The check
deletes the bogus shim.bin and prints a recovery hint instead of
silently embedding a Mach-O header into shim_blob.h.

tests/lib/test-runner.sh now resolves timeout(1) at source time. macOS
does not ship timeout, so the runner falls back to gtimeout on PATH or
the Homebrew opt symlinks under both arm64 and x86_64 prefixes, and
exposes the resolved binary through a shell function so callers keep
using the bare name. TIMEOUT_BIN overrides the search.

tests/test-busybox.sh probes busybox --list under elfuse once at start
and exposes the applet set through BB_APPLETS. test_skip_missing_tool is
overridden to consult that list, so reduced busybox builds (Debian
busybox-static drops a handful of applets) yield SKIP instead of FAIL.
The probe hard-fails if --list itself fails so a broken elfuse cannot
silently degrade the whole suite to SKIP.

tests/test-proctitle-low-stack.sh exercises the proctitle rewrite path
under a capped stack and wires into make check via mk/tests.mk. The
test runs through elfuse busybox echo and verifies both that the run
completes and that the expected output is preserved.

Close #12
@jserv jserv force-pushed the fix-apple-silicon branch from 23f02a9 to 51adfb7 Compare May 13, 2026 10:10
@jserv jserv merged commit 5c7eb2f into main May 13, 2026
4 checks passed
@jserv jserv deleted the fix-apple-silicon branch May 13, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants