Skip to content

Debugging and logs ​

A bring-up produces four kinds of evidence: serial output, the on-screen diagnostic page, the panic screen and build reports. Collect them together.

Log sources ​

SourceTransportUse
UEFI serialUARTFirmware, UniXNU and DarwinBoot
Kernel serialUART selected by serial=XNU console and panic output
Diagnostic pageFramebufferLive state after the console stops
Pulse stripFramebuffer bottom rowHeartbeat while the diagnostic page owns the panel
Panic screenFramebufferPanic text and registers
QEMU logFileUniXNU and loader tests
Build reportsJSONInput hashes and stage results

Serial console ​

Add the arguments to the payload:

toml
[payload]
kernel_args = ["-v", "serial=3", "debug=0x14e"]

The UniXNU boot manager writes to the first EFI_SERIAL_IO_PROTOCOL instance. Its lines start with UNIXNU:. DarwinBoot lines start with DARWINBOOT:.

Capture the port:

sh
tio -b 115200 -d 8 -p none -s 1 /dev/tty.usbserial-0001 -L out/serial.log

Other tools:

sh
picocom -b 115200 /dev/tty.usbserial-0001
screen -L /dev/tty.usbserial-0001 115200
minicom -D /dev/tty.usbserial-0001 -b 115200

The board page records the UART pins and the baud rate. A board with USB device mode can expose a CDC ACM port instead.

Useful early output:

UNIXNU: UniXNU 0.1.0 starting as firmware boot option
UNIXNU: config: loaded \EFI\UniXNU\unixnu.toml
UNIXNU: driver ...: started
UNIXNU: menu: shown 2 entries, default 0
DARWINBOOT: loader v2 installed EL1
DARWINBOOT: ARM64 va=... pa=... args=... entry=... owned=...
DARWINBOOT: inputs validated; exiting boot services

On-screen diagnostic page ​

The diagnostic page is a full-screen text overlay drawn into the XNU framebuffer mapping. Enable it with:

boot-progress=1

Level 1 arms the monitor. The page freezes when no stage or storage activity appears for 30 seconds, or at the 120 second observation limit, and keeps the screen for a photograph.

boot-progress=2

Level 2 runs continuously. Deadlines are reported and the OS keeps running.

The page uses 48 rows of 120 columns. On a panel at least 2160 by 1400 the scale is 3.

Row groupContents
TitleBuild tag, live or stop state, reason
1Time, idle, IRQ count, monitor count, root state, process count, free pages
2Storage requests, verified blocks, failures, integrity counters
3 to 30Stage history, process states, storage detail, last PC and registers
31 to 47Cached storage output, UFS timing, memory, process sampling, UI trace, touch rows
47Independent pulse strip

Label the image so a photograph identifies the build:

sheng-build-tag=ks-001

The title bar then starts with [ks-001].

When userland takes over the display and stops scanning out, take the panel back:

boot-diag-resume=10

After 10 seconds without a scanout, the page redraws. A new validated scanout yields the panel again.

The page reports:

FieldMeaning
timeSeconds since the monitor started
idleSeconds since the last stage or I/O progress
IRQInterrupt entries counted by the monitor
monitorMonitor thread iterations
lostDropped recorder updates
rootOwner root acceptance
procsProcess count
freeFree 4 KiB pages
requestsStorage requests
verified4KVerified 4 KiB blocks
failuresStorage failures
integrityStorage integrity failures
last PCLast interrupt entry PC
TCR, TTBR, SCTLRTranslation control and table base registers

Pulse strip ​

The pulse strip is a 32-pixel strip at the bottom of the panel, drawn at 2x scale in green. It runs on a separate high-priority monitor thread at one second intervals.

P<turn> I<irq> M<monitor>/<seconds> B<bind>:<result> d<depth> @<base> lock<busy>/<render> PC<pc>
TokenMeaning
PPulse turn
IInterrupt entries
MMonitor count and seconds since it advanced
BFramebuffer bind count, result and depth
@Framebuffer base
lockRecorder and renderer busy flags
PCLast interrupt entry PC

The strip survives while the diagnostic page owns the panel. It closes when the OS takes over the display. boot-diag-resume reopens the page and the strip.

Panic screen ​

The payload can patch a panic screen into the kernel copy:

toml
[payload]
panic_screen = true

The payload step runs:

sh
python3 scripts/patch-xnu-entry-probe.py <kernel> --panic-screen --no-reset

The panic screen writes the formatted panic message and registers to the framebuffer and keeps the image visible. --no-reset stops the automatic reset, so the screen stays for a photograph.

The first panic message is also copied to boot_progress_panic_text. A panic on a secondary core parks that core; the live banner keeps the text after the screen is repainted.

The device tree panic region is available with:

CONFIG_XNU_DT_PANIC_REGION=y

The kernel reads /chosen embedded-panic-log-size and the region reg property. The region is validated before it is mapped.

User fault evidence ​

The fault probe records EL0 breakpoint and abort state before exception delivery:

FieldMeaning
pid, nameProcess
PC, LR, SP, FPRegisters at the fault
ESR, FARException syndrome and fault address
slideShared region slide
x0 to x28General registers
framesUp to 12 return addresses

The capture runs in the trap path without allocation. A matching process exit prints the record with the signal. An uncaught exception reason is searched in the writable malloc regions of the exiting process and printed with the crash row.

UI stack trace ​

The UI trace samples the selected processes and prints PC, LR, FP and stack slices:

boot-ui-focus=launchd
boot-ui-focus2=SpringBoard
boot-ui-focus3=backboardd
boot-ui-focus4=Setup
boot-ui-chrono=1

The address tokens use a radix-64 alphabet to fit in the row width. Feed the transcribed rows to the decoder:

sh
python3 scripts/decode-boot-ui-trace.py trace.txt

The decoder prints one JSON object per thread group with pc, lr, frames and error codes.

The trace rotates one page every 8 seconds. Take three or four photographs several seconds apart to capture every page.

Taking photographs ​

The diagnostic page is built to be photographed. Use this sequence:

  1. Set sheng-build-tag to a short image label.
  2. Enable boot-progress=1 for a frozen page, or boot-progress=2 for a live page.
  3. Set boot-diag-resume when userland blanks the panel.
  4. Place the device on a stand. Keep the camera parallel to the panel.
  5. Turn off the flash. Light the panel from the side to avoid a reflection.
  6. Lock focus and exposure on the title bar.
  7. Fill the frame with the panel and include all four corners.
  8. Photograph the title bar, then the full page.
  9. For UI trace pages, take three or four photos 8 seconds apart.
  10. Photograph the bottom pulse strip.
  11. Record the boot image SHA-256 and the capture time.
CheckWhy
Title bar readableBuild tag and state identify the image
No glare over textRows 3 to 30 carry the stage history
All four corners visibleConfirms the full panel and scale
Pulse strip visibleShows the heartbeat at capture time
Two or more framesA changing counter shows a live screen

A camera photo is evidence of what the panel showed. Keep the original file. Transcribe only the rows you need, and keep the photo with the transcription.

QEMU capture ​

sh
python3 package/Scripts/qemu-run.py out/esp \
  --log out/qemu.log --screenshot out/qemu.png \
  --expect 'UNIXNU: menu: shown'

The runner writes the ANSI-stripped log to out/qemu.clean.log. --screenshot captures the panel when the menu appears. --keys sends QEMU key codes after the menu.

The stop helper records registers and memory at a symbol:

sh
python3 scripts/qemu-entry/gdb-stop.py --help

Debug arguments ​

ArgumentUse
-vVerbose kernel output
serial=3Serial console
debug=0x14eDebug output mask
kextlog=0xfffKext logging
boot-progress=1Diagnostic page with a 30 second stall freeze
boot-progress=2Diagnostic page with continuous operation
sheng-build-tag=<tag>Image label in the title bar
boot-diag-resume=<seconds>Take the panel back after the OS stops scanning out
sheng-platform-stage=<n>Stop the platform expert at a stage
sheng-touch=<n>Touch probe stage
offline-legacy-phase=<n>Display probe stage
-no_slto_panicSuppress the SLTO panic
cs_debug=<mask>Code signing output

Evidence bundle ​

Collect:

ItemSource
Serial logUART capture file
Diagnostic photosTitle bar, full page, trace pages, pulse strip
Panic photoPanic screen
Boot reportout/<board>/boot/boot.json
Payload reportout/<board>/payload/payload.json
Build logsout/<board>/logs/

Record the boot image hash with every photo:

sh
shasum -a 256 out/<board>/boot/boot.img
jq . out/<board>/boot/boot.json

Next steps ​

Released under the BSD-2-Clause-Patent License.