Skip to content

media: apple: isp: add T6040 (M4 Pro) camera support - #384

Merged
RyanTheTide merged 3 commits into
aurora-silicon:integration/aurorafrom
jgoyvaerts:m4pro/isp-t6040
Oct 23, 2026
Merged

RyanTheTide merged 3 commits into
aurora-silicon:integration/aurorafrom
jgoyvaerts:m4pro/isp-t6040

Conversation

@jgoyvaerts

Copy link
Copy Markdown
Contributor

T6040 (M4 Pro) support for the Apple ISP driver, so the FaceTime HD camera works on the MacBook Pro M4 Pro. The T6040 ISP is T8140's ISP17a block with the same H17 firmware interface, so this is a hardware descriptor on top of the existing T8140 path plus a few small deltas.

  • Binding: apple,t6041-isp (fallback apple,isp), in its own schema with a select, so it doesn't apply to the T8140 node through the shared fallback. Firmware TEXT and DATA are reserved memory mapped at the addresses the firmware runs them at, next to its main memory window. It has eight power domains (sys, cpu, fe, dprx, secure, be, clr, sensor), a doorbell window of its own (mbox2) and apple,cam-connections-scheme. There is no capture watchdog.
  • Lens shading fix (H17): zero the lens shading strides. The H17 firmware reads that halfword of the xx24 setfiles' lens shading records as the high half of the table offset, so FlowLSC200 faulted on every CH_SET_FILE_LOAD. Apple's xx25 setfiles are their xx24 twins with exactly those halfwords zeroed. Setfiles that already have them zeroed are left alone. It comes before the T6040 commit, so every commit streams on hardware. T8140's sensors are still turned away as unsupported, so nothing changes there today; once they are enabled, their setfiles go through it too (the xx24 siblings of 7523 and 7622 in appleh16camerad carry such strides).
  • T6040 support:
    • The firmware is started as iBoot left it, with no reset sequence. The driver checks that the coprocessor has not run yet and that its reset vector is the firmware text, and refuses to start anything else.
    • No clock enable word; legacy boot mode 1.
    • The boot descriptor carries the camera connection layout and the 0xde600000-byte shared window, as macOS passes them.
    • The broadcast-clear command takes four windows in eight slots. The H17 command now takes several windows; T8140 still sends its single one.
    • The firmware's PMGR, PMP and DSID registers are mapped 1:1 through its DART, from the J616s ADT's address filter ranges, as for T8140.
    • iBoot's TEXT+DATA end 80 KiB short of the firmware's 32 MiB main memory. The driver backs the rest with zeroed pages, as macOS does. Without them, the firmware's first writes fault and it never answers its first command.
    • Output goes through scaler 1. There is no capture metadata pool or watchdog.
    • Front camera: IMX958 (sensor 0x958, lens 0x29), with setfile 2224. Section 0x50f of each Apple setfile names its sensor; of the 32 setfiles in appleh16camerad, only 2224 names 0x958 (2924 and 2921 name the IMX614). With 2224, the firmware logs the same module match as macOS and reports no lens shading calibration mismatch.

Firmware / setfile: iBoot loads the ISP firmware itself; the driver doesn't load it. The driver requests one file, apple/isp_2224_01XX.dat: the IMX958 setfile 0x22240106, cut to 0x6ff4 bytes. asahi-fwextract doesn't produce it because it only reads appleh13camerad. Our packaging extracts it from macOS's /usr/sbin/appleh16camerad with tools/firmware-from-macos.sh (https://github.com/jgoyvaerts/omarchy-m4/blob/main/tools/firmware-from-macos.sh).

Hardware tested on: Apple MacBook Pro (16-inch, M4 Pro, 2024), J616s / T6040, macOS 26.6.x firmware, booted by our own loader with our private T6040 device tree.

How tested: On our daily kernel 7.3.0-m4.3: probe, then 1920x1080 NV12 at 30 fps with v4l2-ctl. Auto-exposure settles by about frame 15 in a lit room. Close and reopen work. Capture works through PipeWire in a browser, and keeps working after suspend/resume (checked on 7.1.12).

The lens shading fixup was also run in userspace, as the same C function, over all 32 setfiles in appleh16camerad and over truncated copies of each, under AddressSanitizer: no access outside the file, every xx25 file unchanged, and 4 to 24 records changed in 21 of the other 24.

Not tested / limits:

  • Only the 1080p/720p presets of config 5 have been used.
  • Firmware denoise tuning (macOS's RNF/MBNR/TNR commands) isn't sent, so the image is a bit grainy in low light.
  • The resident firmware can't be restarted: if the driver fails or is unbound, the camera stays unusable until the next boot.

Not in this PR / dependencies:

Overlap: draft #231 (J616S ISP preparation) touches the same driver files and keeps the T6040 match gated. This PR is the working T6040 match on today's integration. I'm happy to rebase onto #231 if it lands first.

Base: integration/aurora (51cda52). Built and checked there:

  • m4_defconfig with VIDEO_APPLE_ISP=m.
  • W=1 on drivers/media/platform/apple/isp/, with no warnings.
  • checkpatch --strict, clean apart from the MAINTAINERS reminder for the new schema: the driver has no MAINTAINERS entry, and the schema names its maintainer.
  • dt_binding_check with dtschema 2026.9, for apple,t6041-isp.yaml and apple,t8140-isp.yaml.

Hardware testing was done on our daily tree, which is based on integration/aurora (ed55042, Linux 7.3) and carries the same code plus two timeout diagnostics left out here.

🤖 Generated with Claude Code

The T6040 ISP runs firmware that iBoot loads. Unlike T8140, its text and
data are described as reserved memory mapped at the addresses the
firmware runs them at, next to the window of its main memory, it has no
capture watchdog, its doorbell has a window of its own, and the firmware
takes the board's camera connection layout at boot.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
The firmware's lens shading thread (FlowLSC200) faulted on every
CH_SET_FILE_LOAD on T6040, printing "SetLSParamFullImage baseHi
R=120" where macOS prints R=0, and reading 0xf00000 past the setfile.

A setfile is a chain of { size, 0, tag, payload } sections, and the
lens shading tables are described by 18-byte records whose fifth
halfword is the row stride (2 * width) in the xx24 setfiles. The H17
firmware reads that halfword as the high half of the table offset.
Apple's xx25 setfiles are their xx24 twins with exactly these
halfwords zeroed (7524 -> 7525 and 8824 -> 8825 match byte for byte
apart from the header version), and only xx24 files exist for the
IMX958 module, so macOS must zero them as well.

Do the same for H17 firmware after copying the setfile. Setfiles
that already have the field zeroed are left alone.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
The T6040 (M4 Pro/Max) ISP is the T8140 coprocessor with the same H17
firmware interface: its register map in the macOS kext has the T8140
reset, control, fabric and interrupt words, and its firmware asks for
the 0x290-byte H16 boot descriptor. What differs is taken from how
macOS 26.6.2 boots it on the J616s (the firmware's own log of the boot
and of every command it receives) and from aurora's m1n1 proxy captures
on that machine:

 - The firmware is started as iBoot left it, with no reset sequence.
   The coprocessor must not have run yet and its reset vector must be
   the firmware text at the base of the heap region; the driver checks
   both instead of starting something else.
 - No clock enable word, legacy boot mode 1 in GPIO 6.
 - The boot descriptor carries the board's camera connection layout and
   a 0xde600000-byte shared window, as macOS passes it.
 - The broadcast-clear command takes four windows in eight slots, the
   unused ones all ones. Generalise the H17 command to several windows;
   T8140 still sends its single one.
 - Like T8140, the firmware reaches PMGR, PMP and DSID registers through
   its DART at their physical addresses, among them the clock and
   bandwidth scratch words named in PMP_CTRL_SET and the broadcast-clear
   bases. Map the J616s ADT's DART address filter ranges for the ISP
   1:1, rounded out to 16 KiB pages.
 - The firmware's main memory is 32 MiB at IOVA 0. iBoot loads __TEXT
   and __DATA into it, and they end 80 KiB short of the top (0x1fec000
   of 0x2000000). The rest is the host's to back, as macOS does; left
   unbacked, the firmware's first writes there take DART translation
   faults and it never answers its first command. Read the end of the
   "data" memory-region and map zeroed pages from there to the heap top
   for as long as the firmware surfaces live.
 - Output goes through scaler 1, so the rendered pool is the scaler 1
   pool, as the buffers submitted to it already were.
 - There is no capture metadata pool or capture watchdog. Skip that
   pool and its size check on H17 firmware that has none.

The front camera is an IMX958 (sensor 0x958, lens 0x29). Of the 32
setfiles in macOS's appleh16camerad, only module 2224 revision 01 names
it in section 0x50f, where each setfile names the sensor it tunes; 2924
and 2921 name the IMX614. With 2924 the firmware streams, but reports
a lens shading calibration for a 3744-pixel-wide sensor on every frame
("fullRes (3072 x 3024) lsCalibRes (3744 x 3024) not match!") and never
logs the module match macOS logs once the setfile has loaded. 2224
carries a 3072 x 3024 calibration. The file is 0x6ff4 bytes up to the
end of its section chain.

T6040 sets none of T8140's fw_iova_mask, mbox2_offset and
mbox_irq_route. The firmware's own addresses are at 0x100_0000_0000,
above the 36 bits T8140 keeps, so no mask is applied. Its doorbells have
a register window of their own in Apple's device tree, described as
"mbox2", instead of sitting inside the mailbox window. Only the base
interrupt enable is written; T8140's extra enable and routing words are
not needed on T6040.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
RyanTheTide pushed a commit that referenced this pull request Oct 23, 2026
From the J616s ADT (isp0, dart-isp0 and the PMGR device table): the
coprocessor and its mailbox windows, the first of its four interrupts,
three DART instances on one interrupt with streams 1, 6, 7, 10 and 11
aliased to stream 0, and the ISP's power states. isp_cpu and isp_fe are
at PMGR +0x4000; the ADT lists isp_secure, isp_be and isp_clr without
an address, so the slots follow T8140's order (CPU, FE, DPRX, SECURE,
BE, CLR) without the blocks T6040 lacks. With only CPU and FE on, the
firmware takes an SError while it loads its tunables. isp_sys hangs off
the always-on afi and fab6_soc fabric states.

The J616s front camera is an IMX958. iBoot loads the ISP firmware below
the kernel's memory; its text and data segments are reserved where the
firmware runs them, with the firmware's 32 MiB main memory as the heap
window. Two presets of firmware configuration 5 (1920x1080, 1280x720).

Binding: apple,t6041-isp from #384 ("media: apple: isp: add T6040 (M4
Pro) camera support").

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
@RyanTheTide
RyanTheTide merged commit 25f21f9 into aurora-silicon:integration/aurora Oct 23, 2026
@jgoyvaerts
jgoyvaerts deleted the m4pro/isp-t6040 branch October 24, 2026 16:14
erikwb pushed a commit to erikwb/linux that referenced this pull request Oct 28, 2026
Port the processed config-5 preview command sequence from our m1n1 J616s work, with contiguous V4L2 P010 strides and pool geometry. Query all advertised H17 sensor configs and retain the board defaults while appending consistent output modes. Keep NV12 the default and scope the new path to T6040/IMX958.

Retire output and metadata leases after the report ACK, restart an empty P010 queue on requeue, bound ring servicing and latch malformed or timed-out command replies. Add command-byte, geometry, ownership and ring-wrap coverage. These source corrections still require sustained capture validation on 26.6.2.

Builds on Jonathan Goyvaerts’ T6040 camera support in PR aurora-silicon#384. Protocol reference: our m1n1 PR aurora-silicon#20 at 9be380c8a5 and Linux PR aurora-silicon#231 at a4e7ee8. No firmware payload is included.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
erikwb pushed a commit to erikwb/linux that referenced this pull request Oct 28, 2026
Port the processed config-5 preview command sequence from our m1n1 J616s work, with contiguous V4L2 P010 strides and pool geometry. Query all advertised H17 sensor configs and retain the board defaults while appending consistent output modes. Keep NV12 the default and scope the new path to T6040/IMX958.

Retire output and metadata leases after the report ACK, restart an empty P010 queue on requeue, bound ring servicing and latch malformed or timed-out command replies. Add command-byte, geometry, ownership and ring-wrap coverage. These source corrections still require sustained capture validation on 26.6.2.

Builds on Jonathan Goyvaerts’ T6040 camera support in PR aurora-silicon#384. Protocol reference: our m1n1 PR aurora-silicon#20 at 9be380c8a5 and Linux PR aurora-silicon#231 at a4e7ee8. No firmware payload is included.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants