Skip to content

wifi: brcmfmac: BCM4378 association and flowring timeouts across sleep and roam - #218

Draft
ludagoo wants to merge 1 commit into
omacom:integration/aurorafrom
ludagoo:fix/bcm4378-sleep-roam
Draft

ludagoo wants to merge 1 commit into
omacom:integration/aurorafrom
ludagoo:fix/bcm4378-sleep-roam

Conversation

@ludagoo

@ludagoo ludagoo commented Oct 25, 2026 •

Copy link
Copy Markdown

On BCM4378 only, retry assoc_info after -EBADE (-52) and reclaim a flowring's tx pktids instead of waiting for a txstatus that does not arrive.

BCM4378/3 answers assoc_info with a firmware error while a connect or roam is still committing. fwil reports that as -EBADE. The same windows drop in-flight flowring frames without a txstatus, and brcmf_msgbuf_delete_flowring() then waits about 75 ms on the event worker, logs the timeout, and zeros outstanding_tx while those pktids stay allocated. This chip retries assoc_info up to eight times (175 ms of backoff) only after -EBADE, and reclaims that ring's tx pktids instead of waiting. Other chips keep the historical wait and do not retry.

Machine: MacBook Air M1 2020 (J313), PCI 14e4:4425 rev 03, firmware 18.20.383 loaded as brcmfmac4378b1-pcie.

Session build, Monday 26 Oct 2026 05:02 PM -03: linux-aurora 7.2.11-6-2-ARCH (package 7.2.11.aurora6-2), branch local/j313-test at 7638df0. makepkg exited 0. rustc 1.85.0, LLVM/clang 18.1.8. This machine is running that kernel. The stacked commit compiled in was f07926d. This pull request's head is 3a93601, which is not the commit that was built.

Host policy check of sleep_roam.h printed ok. The KUnit suite brcmfmac-sleep-roam was not run. Wi-Fi sleep/roam stress has not been run on this kernel.

@chriskearney chriskearney left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on 5f0966df72cabe2d69c87771fb8cedad189a256f: the flowring reclaim can recycle transmit buffers before firmware ownership has ended. Details inline.

The bounded BCM4378 assoc_info retry is separable from this reclaim change.

Validation: source review of the allocation, reclaim and completion paths. I have not built or boot-tested this head, or reproduced a hardware failure.

* completes them late, or not at all once the bus is down.
* Reclaim the pktids so a late txstatus finds them free.
*/
brcmf_msgbuf_reclaim_flowring_tx(msgbuf, flowid);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please defer reclaim and pktid reuse until firmware can no longer access these buffers. This call runs before the bus-state check and before FLOW_RING_DELETE is submitted, so closing the host ring does not establish that firmware has drained it.

take_pktid() immediately unmaps the DMA buffer and marks the slot FREE. The allocator can reuse that slot, while process_txstatus() looks it up using only request_id - 1 (match_flow=false, no generation check). A delayed completion for the reclaimed frame can then claim and unmap a newer frame occupying the same slot. The atomic claim prevents concurrent host claims, but cannot distinguish those two allocations.

Please preserve buffer ownership until a protocol-defined completion/quiescence point and prevent stale completions from consuming reused slots. The added KUnit policy tests do not cover this lifetime ordering.

@malik-na

Copy link
Copy Markdown
Member

Tested on a J313 (M1 Air). This fixes what it targets, but the device suspend phase now fails on 2 of 2 boots (the baseline passed 1 of 1, and a second baseline run is queued to rule out flakiness).

Improved vs the baseline (19f4965180f4):

baseline this PR
nmcli con up re-association (×2) rc 4, failed both times rc 0 both times (~0.7 s)
delete_flowring: timed out waiting for txstatus 2 0
brcmf errors during re-association 8 2
gateway ping after the s2idle cycle 0/5; Wi-Fi was gone ~3 min and only came back after iwd restarted 5/5, no help needed

Regression: brcmfmac fails the device suspend phase. On both boots, the pm_test=devices suspend ran in the PR kernel's first minute:

brcmf_msgbuf_query_dcmd_len: Timeout on response for query command
brcmf_cfg80211_reg_notifier: Country code iovar returned err = -5
brcmf_fil_cmd_data: bus is down. we have nothing to do.
brcmfmac 0000:01:00.0: PM: pci_pm_suspend(): brcmf_pcie_pm_enter_D3 [brcmfmac] returns -5
PM: Some devices failed to suspend, or early wake event detected

systemd-suspend.service is then left failed. The same boot also logged one new DART fault:

apple-dart 681008000.iommu: translation fault: status:0x81000004 stream:1 code:0x4 (NO PTE FOR IOVA) at 0xbfc339d8

I haven't tied the fault to a specific path. It fits a reclaimed tx pktid whose buffer the firmware still referenced: brcmf_msgbuf_reclaim_flowring_tx() unmaps the buffer and frees it while the firmware may still complete it. A reused index also means a late txstatus for the old frame would finalize the newer one early. Worth checking whether the D3 failure follows from the firmware still holding frames the host has already reclaimed.

Not run: the brcmfmac-sleep-roam KUnit suite (CONFIG_BRCMFMAC_KUNIT_TEST is off in this config).

How this was tested
  • Hardware: MacBook Air (M1, 2020), J313, BCM4378 (brcmfmac4378b1-pcie), DCP 13.5, booted through U-Boot → Limine.
  • Kernels: this PR's head and its parent 19f4965180f4 (the baseline), built with the Air's own /proc/config.gz and booted once each as a one-shot test entry. The Mac returns to its installed 7.2.11-6-1-ARCH after every run.
  • Each boot ran:
    • a device-phase suspend test (pm_test=devices through systemctl suspend)
    • omarchy-m-test, compared with a run on the installed kernel
    • a check script that reads dmesg counters, writes the keyboard and panel backlight (420, 419, 189, then back), lists ALS IIO channels, re-associates Wi-Fi twice with nmcli con up, and does one pm_test=platform s2idle cycle (noirq + resume_early, auto-wakes after 5 s)
  • Not covered: a real s2idle with a keypress wake, Touch ID enrollment/verify with a finger, and how brightness looks.

…p and roam

BCM4378/3 (PCI 14e4:4425 rev 03, firmware 18.20.383 loaded as
brcmfmac4378b1-pcie) returns a firmware error from assoc_info while a
connect or roam is still committing, which fwil surfaces as -EBADE
(-52). The same windows drop in-flight flowring frames without a
txstatus. brcmf_msgbuf_delete_flowring() waits about 75 ms per ring on
the event worker, logs the timeout, and zeros outstanding_tx while the
pktids stay allocated.

On this chip, reclaim those pktids instead of waiting, and retry
assoc_info up to eight times (175 ms of backoff) only after -EBADE.
Other chips keep the historical wait and do not retry.

Signed-off-by: Lucas Goossen <lucas.goossen@hey.com>
@ludagoo
ludagoo force-pushed the fix/bcm4378-sleep-roam branch from 5f0966d to 3a93601 Compare October 26, 2026 18:15
@malik-na

Copy link
Copy Markdown
Member

Follow-up after more runs: 4 automated boots of this PR, 4 of the baseline, and one hands-on real sleep. My earlier comment overstated the fix. The baseline's Wi-Fi failure is intermittent, and with more runs this PR doesn't measurably improve re-association. The suspend regression showed up on every boot, and also in real use.

baseline (4 boots) this PR (4 boots)
nmcli con up re-association failed 3 of 4 2 of 3 with data (iwd connect-failed, NM Activation: failed)
delete_flowring: timed out waiting for txstatus 3 of 4 0 (expected: BCM4378 no longer waits)
brcmfmac fails to suspend (brcmf_pcie_pm_enter_D3: Timeout → pci_pm_suspend returns -5) 0 of 4 4 of 4
Wi-Fi dead after resume until reboot (automated pm_test cycles) 0 2 of 4
apple-dart … translation fault … (NO PTE FOR IOVA) 0 3 of 4

Real lid-close sleep:

  1. 01:05:52: lid closed. The first suspend fails on the same D3 timeout (systemd-sleep: Failed to put system to sleep. System resumed again: Input/output error).
  2. The Mac then stays awake with the lid shut while Wi-Fi is dead (brcmf_msgbuf_query_dcmd_len: Timeout, bus is down).
  3. 01:06:28: Omarchy's omarchy-wifi-resume-fix reloads brcmfmac after 12 s (the device comes back as phy1).
  4. 01:06:37: logind's second suspend attempt succeeds.

In normal use this shows up as a delayed sleep, about 45 s here, with a userspace driver reload covering for it. The automated pm_test cycles bypass that hook, which is why they stayed wedged:

brcmfmac 0000:01:00.0: brcmf_pcie_pm_enter_D3: Timeout on response for entering D3 substate
brcmfmac 0000:01:00.0: PM: failed to suspend async: error -5
PM: Some devices failed to suspend, or early wake event detected
brcmf_msgbuf_tx_ioctl: Failed to reserve space in commonring
_brcmf_set_multicast_list: Setting BRCMF_C_SET_PROMISC failed, err=-12
brcmf_run_escan: error (-12)

A hypothesis I haven't verified: the D3 handshake and the commonring time out because the firmware still holds tx frames whose pktids the host reclaimed in brcmf_msgbuf_reclaim_flowring_tx(). The repeated DART faults fit DMA to buffers that are already unmapped. A run with the reclaim disabled (keeping the assoc_info retry) would separate the two halves of this PR.

How this was tested
  • MacBook Air (M1, 2020), J313, DCP 13.5, BCM4378, booted through U-Boot → Limine.
  • This PR's head and its parent 19f4965180f4 (the baseline), built with the Air's own /proc/config.gz and booted as one-shot test entries. The Mac returns to its installed 7.2.11-6-1-ARCH after every run.
  • Automated boots: device-phase suspend (pm_test=devices), omarchy-m-test, and a check script (dmesg counters, keyboard and panel backlight writes, ALS channels, two nmcli con up re-associations, one pm_test=platform s2idle cycle).
  • Hands-on boots: a person at the Mac doing a real lid-close sleep and keypress wake, using the brightness keys, and covering or lighting the ALS, while the kernel's state was logged every 2 s.
  • Tested commit: 5f0966df72ca on 19f4965180f4. The PR has since been rebased to 3a9360174756 on 937b727ef9a3 with an identical diff; the new base wasn't re-tested.

@ludagoo
ludagoo marked this pull request as ready for review October 27, 2026 00:24
@ludagoo
ludagoo marked this pull request as draft October 27, 2026 00:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants