Skip to content

[BUG] Socket serial backend freezes guests under debug-channel load (3 bugs + fixes) #248

Open

Description

@BuilderGuy1

Bug report: socket serial backend freezes guests under debug-channel load

Project: DingusPPC (https://github.com/mist64/dingusppc — the upstream of both the native build and
the wasm-port browser build; this report's fixes are on the native lineage)
Reporter environment: Copland (D11E4) NuKernel guest; ESCC modem port driven as the LLNub debug
channel (kernel debug driver LLNubDriver::SendBytes, ~1,200 packets/s of 50-byte ACK frames over the
unix-socket backend, peer = a Python echo process). Fork branch t22-fix, Oct 2026.

Two defects in devices/serial/chario.cpp (CharIoSocket), one in main.cpp's debug sampler, each of
which alone hangs or kills the whole emulated machine. Minimal fixes for all three are in fork commits
54f5904d, 0163e618, fc4a9727; diffs included below.

Build exposure at a glance

bug native (SDL/CLI) build wasm-port browser build
1 — RX duty-cycle throttle (CharIoSocket) affected not exercised — the browser build has no unix-socket backend
2 — blocking recv in rcv_char (CharIoSocket) affected not exercised — same reason
2b — always-true select() / dead reset affected not exercised — same reason
3 — unconditional [PC] sampler (main.cpp) affected likely affected — same main.cpp; worth verifying the sampler isn't compiled into dingusppc.wasm

Bug 1 — rcv_char_available() lies during dense RX ("duty-cycle throttle")

(Native-build path: CharIoSocket is the unix-socket backend; the wasm-port browser build uses a
different transport and does not run this code.)

Before:

bool CharIoSocket::rcv_char_available()
{
    if (consecutivechars >= 15) {
        consecutivechars++;
        if (consecutivechars >= 800)
            consecutivechars = 0;
        return 0;                       // "no character available"
    }
    return this->rcv_char_available_now();
}

Once 15 bytes have been seen in a row, the backend reports no character available for the next ~785
polls
(~98% of the time at memory speed). The kernel socket buffer already prevents overrun — unread
bytes stay pending and select() reports them when the guest is ready — so this throttle protects
nothing and starves any guest that polls.

Observed failure (Copland): the guest's LLNubDriver_WaitForByte polls RR0_RX_CHARACTER_AVAILABLE
with MSR[EE]=0 (interrupts masked). With the throttle active the bit stays dark through the driver's
wait, the machine parks in the poll loop, the ADB interrupt goes unserviced, and the guest wedges
visibly: a viacuda.cpp "Cuda transaction probably in progress" warning flood, then a dead cursor
~2 seconds after link saturation. Reproduced on every boot under load (18-run matrix).

Fix (commit 54f5904d): availability is the select() truth:

bool CharIoSocket::rcv_char_available()
{
    // The socket buffer already prevents overrun; be honest.
    return this->rcv_char_available_now();
}

Validation: the same workload that wedged every prior boot now runs to completion with the machine
fully alive.


Bug 2 — rcv_char() blocks the emulation thread indefinitely

Before:

int CharIoSocket::rcv_char(uint8_t *c)
{
    if (this->acceptfd == -1)
        this->rcv_char_available_now();
    if (this->acceptfd != -1) {
        int received = (int)recv(this->acceptfd, c, 1, 0);   // blocking, no availability check
        ...
    }
    return 0;
}

recv() with no MSG_DONTWAIT and no availability pre-check: any guest read of the ESCC data
register while the peer-side has sent nothing blocks the emulator's CPU thread forever
. The entire
emulated machine freezes (CPU pegged at 100% host-side, no further serial, video or ADB progress).
Reproduced deterministically: socket backend + a connected-but-silent peer + any guest code touching
the SCC data register ⇒ freeze at the boot splash.

Fix (commit 0163e618):

int received = (int)recv(this->acceptfd, c, 1, MSG_DONTWAIT);
if (received == -1) {
    if (errno != EAGAIN && errno != EWOULDBLOCK)
        LOG_F(INFO, "socket accept read err: %s", strerror(errno));
}

EAGAIN means "no byte pending"; rcv_char_available() remains the sole truth the guest polls.

Validation: silent-peer boots that previously froze at the splash now proceed normally.


Bug 2b — dead reset branch and always-true select() in rcv_char_available_now()

writefds = readfds places the listening socket (plus, on macOS, the connect-ready unix socket) in
the write set, so select() returns > 0 on every call once sockfd != -1. Consequences in the current
code:

  1. The else consecutivechars = 0; reset is unreachable — the counter only ever resets via the
    800-wrap, which makes Bug 1's throttle even more aggressive than its author likely intended.
  2. recv() is called on the listening socket whenever it shows readable (a pending connection),
    logging a spurious error per call; the accept then happens unconditionally anyway.
  3. The write/error FD_ISSET checks on both fds are meaningless with this construction.

Suggested cleanup upstream: select on readfds only (acceptfd + sockfd), drop the write/error sets;
accept on the listen socket's readability; never recv() the listen socket; consider a small real RX
FIFO so guests with one-byte ESCC windows aren't at the mercy of host scheduling.


Bug 3 — the built-in [PC] sampler kills kernel boot (observer effect)

main.cpp's periodic sampler (500 ms cyclic, ~20 mmu_read_vmem chain-walk reads per tick, installed
unconditionally) DSI-kills the Copland kernel boot during MMU bring-up: two independent boots died
with "Nub reentered with 300 (data access)" — DSI: PC=C01F15C0, LR=C0294000 … LLNubExceptionHandler → LLNubGetTime → AbsoluteToNanoseconds with a garbage AbsoluteTime argument — while the identical disk
image boots clean on the same hardware state without the sampler. The fork's own comment already
admitted the reads "measurably stall the boot".

Fix (commit fc4a9727): install the sampler only when PPC_PC_SAMPLER is set in the environment.
Anyone debugging the sampler itself can still enable it; every other boot stops paying the tax.

Lesson for the general case: debug instrumentation that walks guest memory through the MMU is not
observationally neutral on this emulator — gate every such instrument behind an env var by default.


Reproduction recipe

  1. Guest: Copland D11E4 disk image (the kernel's LLNubDriver speaks the debug protocol over the modem
    port; the tertiary loader maps the SCC as a debug channel before the kernel runs).
  2. SERIAL_BACKEND=socket + a unix-socket peer that replies to each 50-byte frame with an ACK.
  3. Under load (a guest launch that opens the debug session) the frame rate reaches ~1,200 pkt/s:
    • unpatched: the machine wedges ~2 s in (Bug 1) or freezes outright on any RX read against a quiet
      peer (Bug 2);
    • patched: the session runs to completion, machine interactive throughout.
  4. Instruments that help: CUDA_TRACE=1 (a viacuda stall watchdog that samples pc/msr/ee when the
    guest stops collecting ADB packets — the wedge's smoking gun), and a peer-side frame log for the
    wire census.

Validation summary

configuration boot guest alive at session end
stock (throttle + blocking recv) ✔ ✘ — wedge ~2 s after session open
stock + silent peer ✘ — — freeze at splash (Bug 2)
patched (both fixes) ✔ ✔ — full session, interactive throughout

The Copland/LLNub guest makes a good stress case for this backend because its debug driver polls with
interrupts masked and its protocol runs at memory speed on an unpaced backend — but both bugs are
generic to any guest that polls RR0 and reads RR8.

Activity

  1. BuilderGuy1 commented on Oct 29, 2026

    @BuilderGuy1
    Author

    Correction / scope reduction — while preparing the PR for this issue, we found that Bug 3 is
    withdrawn
    : the [PC] sampler in main.cpp exists only in our local fork (a debug instrument added
    during Copland work). Upstream master has no sampler — there is nothing to gate. Apologies for the
    noise; it was a fork-only artifact that we mistook for shared code.

    Bugs 1, 2, and 2b stand — both chario.cpp fixes cherry-pick cleanly onto current master and build
    clean, and the blocking-recv and throttle behaviors are verifiable there. PR with the two fixes is
    incoming.

  2. joevt commented on Oct 30, 2026

    @joevt
    Contributor

    My response will be in #249.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions