Skip to content

media: apple: avd: allow more than 32 capture buffers per context - #281

Closed
sithglan wants to merge 1 commit into
aurora-silicon:aurora-wipfrom
sithglan:avd-max-capture-buffers
Closed

sithglan wants to merge 1 commit into
aurora-silicon:aurora-wipfrom
sithglan:avd-max-capture-buffers

Conversation

@sithglan

@sithglan sithglan commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

The AVD capture queue never sets max_num_buffers, so vb2 caps every decoding context at VB2_MAX_FRAME (32) buffers. The driver doesn't need that cap: it looks up reference frames by timestamp (vb2_find_buffer()) and gives the hardware DMA addresses, never buffer indices. This adds a max_capture_buffers module parameter (default 128). Buffers are still allocated on demand by userspace, so it only raises the limit.

Why it matters: the t8103 AVD can't decode every 4K60 H.264 stream in real time. On a 4K59.94 High@5.2 camera clip (65 Mbit/s average), FFmpeg's V4L2 request hwaccel decodes:

Part of the clip Bitrate Decode speed
Most of it ≤ 100 Mbit/s ~80 fps
First 5 s ~110 Mbit/s 59.4 fps
Burst at 7–9 s 125–156 Mbit/s ~41 fps

GStreamer v4l2slh264dec gives the same numbers. CPU, firmware build (-O0 vs -O3) and the CPU governor make no difference, so this is the hardware's throughput. During the burst, mpv drops 60–70 frames. Decoding ahead (mpv --vd-queue) can cover the burst, but it needs a lead of 60–90 frames. With 32 buffers, a 16-frame DPB, 4 frame threads and the frames held for display, queues above ~20 frames fail: VIDIOC_CREATE_BUFS returns ENOBUFS and mpv silently falls back to software decoding.

Testing: Mac mini M1 (j274), kernel 7.1.12 built from aurora-wip plus #275–#277; drivers/media/platform/apple/avd/ is identical to current aurora-wip. The patched module was built out of tree against that kernel. Userspace: Debian FFmpeg 9.0.2 with the V4L2 request hwaccels (FFmpeg PR 20847), mpv 0.41 with mpv PR 14690.

stock (32) this patch (128)
mpv --hwdec-extra-frames=104 --vd-queue-max-samples=96 ENOBUFS, falls back to software stays on v4l2request, ~110 capture buffers
Burst (6–11 s), queue filled before playback starts 60–70 dropped frames 0 dropped frames
Errors during playback — no allocation errors, no DART faults; ≥ 9.8 GB of 16 GB still free
H.264 / VP9 output vs software decode bit-exact bit-exact

Notes:

  • Each 4K NV12 capture buffer is ~12.4 MB, so a 96-frame queue costs ~1.2 GB. That's the application's choice; the default of 6 extra frames allocates the same as before.
  • Getting a lead with mpv also needs playback to start only after the queue has filled. Otherwise the queued frames count as latency rather than lead. A small Lua script that pauses for 1.5 s at start and after seeks handles that; that part is outside the kernel.

checkpatch --strict: no errors, warnings or checks.

The capture queue never sets max_num_buffers, so vb2 limits every
context to VB2_MAX_FRAME (32) buffers. Nothing in the driver needs that
limit: reference frames are looked up by timestamp (vb2_find_buffer())
and handed to the hardware by DMA address, never by buffer index.

32 buffers leave too little room for userspace to decode ahead. With a
16-frame DPB, four frame threads and the frames held for display, mpv's
decoder queue (--vd-queue) gets only a few frames before
VIDIOC_CREATE_BUFS fails with ENOBUFS, and the FFmpeg V4L2 request
hwaccel then falls back to software decoding. Decoding ahead is needed
because the t8103 AVD decodes 4K59.94 H.264 at only ~41 fps during a
125-156 Mbit/s burst, so playback drops frames unless the decoder has
built up a lead of 60-90 frames beforehand.

Add a max_capture_buffers module parameter, default 128. Buffers are
still allocated on demand by userspace, so this only raises the limit.

Signed-off-by: Thomas Glanzmann <thomas@glanzmann.de>
@RyanTheTide

Copy link
Copy Markdown
Contributor

Closing as superseded: this work was ported by #328 (merged 2026-10-16) and carried into the maintained 7.3 integration/aurora line. Thanks for the original change; nothing here is lost.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants