Skip to content

drm/asahi: observe and retire a G17 render pair in one device-mutex hold - #502

Merged
DjDeveloperr merged 1 commit into
integration/aurorafrom
perf/g17-observe-hold-20261026
Oct 27, 2026
Merged

DjDeveloperr merged 1 commit into
integration/aurorafrom
perf/g17-observe-hold-20261026

Conversation

@DjDeveloperr

Copy link
Copy Markdown
Contributor

Stacked on #494.

observe_renders used to take the device mutex twice for each render pair it checked. The first time, it loaned the pair out of its entry to observe its front command. The second time, it returned the pair and retired whatever the observation found. Between those two sections the event worker held no lock, so a run_job thread could take the mutex, and the worker then queued behind it to return the pair.

This change selects the pair, reads its front and retires it in one hold, so each observed pair costs one acquisition of the device mutex instead of two. The observation is a short read of the pair's completion memory and doesn't block. The pair now stays in its entry, so observations no longer use the loan: the preparing marker, the borrowed dependency and switch snapshots, and the return path with its invariant-failure handling are gone from this path. Preparers still loan pairs, and the observer still skips an entry that a preparer holds (entry.preparing). The epoch, recovery and crash checks now run once before selection, under the same hold.

The read takes the context's work mutex for the shared list head, so that mutex now nests inside the device mutex. It is a leaf lock: all four of its call sites touch only the context's work storage. The new state → work order can't invert.

Measured effect

Neo (A18 Pro) on the core with AIC2 effective affinity (#447), same Mesa throughout, 3 rounds per boot, medians in µs per submit. The boots ran in this order: with the change, without it, with it again.

Row With Without With (2nd boot)
render.light tri, queue d8 31.29 33.73 31.04 (−7.6% vs without)
render.light tri, queue d1 105.49 105.75 105.53
render.light tri, queue d32 / d64 30.06 / 30.06 30.01 / 30.00 29.68 / 29.68
render.light tri, 2 processes / 2 queues 52.18 / 51.98 52.24 / 52.12 52.29 / 52.12
compute.light, queue d1 103.61 103.73 104.34
sync.signal queues d2, timeline / binary 22.74 / 22.53 22.67 / 22.27 22.69 / 22.53

The d8 gain reproduces the earlier same-boot measurement with a diagnostic switch that toggled only this behaviour: 33.45 → 30.77 (−8.0%, rounds don't overlap), completion latency 259 → 243 µs, and every other row within ±1%. In that same-boot run, sync.signal binary moved −1.0%. The +1.2% it shows across boots above is inside the boot-to-boot spread, which reached 3.5% on that row in earlier runs. For reference, macOS on the same machine does about 29.9 µs at depth 8.

Waiters that sleep on the device mutex, per submit (0.3 s tracepoint capture): run_job 0.55 → 0.49–0.50, event worker 0.18 → 0.12–0.13.

Context: IRQ thread placement and #447

The same investigation measured where the GPU mailbox IRQ thread runs. On a core without #447, the thread's effective affinity is empty, and it moves between CPU0, which takes every hard IRQ, and CPU5:

IRQ thread on render.light tri queue d1
CPU0 104–107 µs
CPU5 138–139 µs

That held for all 15 config-rounds measured. With #447, genirq binds the thread to CPU0, and d1 stays at 104.5–109 µs every round. This matches the d1 improvement in #447's own description.

Moving the event work off the IRQ CPU was also tried, as a diagnostic switch. It removes the device-mutex waiter sleeps entirely but costs +17% at d1 from the cross-CPU wake. It is not part of this change.

Testing

  • Stop-on-error smoke: render.light d1/d32, procs d2, queues d2, compute.light d1 and timeline sync. All clean.
  • Regression gate: PERFORMANCE-REGRESSION-PASS.
  • Targeted CTS (5368 cases: synchronization2 basic + timeline_semaphore, api.command_buffers, renderpass suballocation/subpass_dependencies, dynamic_rendering basic/random): 4687 Pass, 681 NotSupported, 0 Fail.
  • No GPU recoveries or faults on any of the three boots. The desktop stayed alive.

The event worker read a render pair's completion memory outside the
device mutex. It took the pair out of its entry on loan under one hold,
read the front's stamps and graph cursors unlocked, then took the mutex
again to return the pair and retire what it had seen. The read is a
short memory snapshot, so the loan bought no useful concurrency, and
every observed pair cost two acquisitions of the mutex that submission
also takes, with publishers able to slip in between.

Select the pair, read its front and retire it in one hold. The pair
stays in its entry, so observations no longer need the loan bookkeeping
(the preparing marker and the borrowed dependency and switch snapshots)
or the return path and its invariant-failure handling. Preparers keep
using the loan. The read takes the context's work mutex for the shared
list head, which now nests inside the device mutex; that mutex is a
leaf, held only around accesses to the context's work storage.

Measured on Neo across boots, with this change, without it, and with it
again (3 rounds each, medians, us per submit):

  render.light tri queue d8         31.29 / 33.73 / 31.04  (-7.6%)
  render.light tri queue d1        105.49 / 105.75 / 105.53
  render.light tri queue d32/d64    30.06 / 30.01 / 29.68
  render.light tri procs d2         52.18 / 52.24 / 52.29
  render.light tri queues d2        51.98 / 52.12 / 52.12
  compute.light queue d1           103.61 / 103.73 / 104.34
  sync.signal queues d2 timeline    22.74 / 22.67 / 22.69
  sync.signal queues d2 binary      22.53 / 22.27 / 22.53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant