Repository navigation
Conversation
|
Tested on a second J713: it works here too, including with the calibration file. Setup. MacBook Air 13" M4 (J713), macOS 26.7.1 (25G241) alongside. This machine runs an Asahi-based 7.1.9 kernel rather than integration/aurora, so I built
For the installer follow-up. On this machine the 2224 setfile is in
Tested with help from Claude (Anthropic). |
Apple DARTs have address filter (DAPF) registers next to their translation registers. They restrict the physical addresses the DART lets its masters reach, and lose their contents whenever the DART's power domain is off. Allow them as an optional second register range named "dapf", so that a device whose coprocessor reaches fixed MMIO windows through the DART can program them after every power-up, finding them on the DART it already references. Signed-off-by: Erik Bourget <erik@ebourget.net>
The T8132 (M4) ISP has the T6041's register layout and firmware memory model: the bootloader loads the firmware's text and data segments, described as reserved memory at the addresses the firmware runs them at. It uses the same fourteen ISP power domains as the T8140, plus two camera sensor power states. Describe it in its own schema, as the T6041 and T8140 are, since its power-domain list differs from both. Signed-off-by: Erik Bourget <erik@ebourget.net>
The ISP's own power states are children of ISP_SYS at PMGR + 0x4000, in the ADT's order: CPU, CPU cores 0 and 1, FE, DPRX, secure, STS0, STS1, Pearl, BE, CLR, CVD and CVM, with the parents the ADT gives them. The camera sensor power states ISPSENS0 and ISPSENS1 are at 0x188 and 0x190; the ADT lists both among the ISP's power gates. Add them disabled, like ISP_SYS, and let the J713 policy enable them. They have no consumer yet, so they stay off. Signed-off-by: Erik Bourget <erik@ebourget.net>
Describe the ISP from the ADT: the coprocessor, mailbox, handshake and doorbell registers at the T6041's offsets, the mailbox interrupt, and three DART instances on one interrupt, which remap streams 1, 2, 3, 6, 7, 10 and 11 to stream 0. The first instance also has the address filter registers. The DART's 0xe0000000 window above the firmware's 20 MiB main memory is the host's allocation aperture. Everything stays disabled until a board describes its firmware memory. Signed-off-by: Erik Bourget <erik@ebourget.net>
On T8132 the ADT gives the first ISP DART address filter windows for the fixed MMIO windows the ISP's coprocessor reaches, among them the ISP power states and the DSID broadcast-clear windows. The filter loses its windows whenever ISP_SYS is off and powers up with arbitrary contents. The camera streams with the filter cleared and with it left as it powers up, but program it as the ADT describes rather than leave it undefined. Take an optional per-SoC window list, find the filter registers on the first iommus DART, and program and verify the windows after every power-up, before the coprocessor starts. The windows lie inside the fw_mmio windows that the driver already maps into that DART for the firmware, so the list belongs next to fw_mmio; the DART driver itself has no use for them. Two KUnit cases check that every SoC's windows lie inside its fw_mmio windows, and the register image the writer produces. Signed-off-by: Erik Bourget <erik@ebourget.net>
Resident firmware is started at probe and runs while the driver is bound, so the ISP stays powered from boot even when the camera is never used. On the J713 (T8132) that costs about 100 mW, at idle and in suspend-to-idle. The video device can be registered without the firmware: its formats come from the device tree presets. Let a SoC defer the firmware start and the camera detection to the first stream start; from then on the firmware stays resident as before. Opening the device or listing its formats does not start it, as desktop services open every video node at login. On the J713 the deferred start, from power-up to a detected camera, adds about 0.2 s to the first VIDIOC_STREAMON. Deferring moves what detection checks to that first stream: an unsupported sensor, unexpected metadata sizes or a missing channel now fail it instead of probe, and leave the camera unusable until the next boot, like any failed resident start. Presets the firmware lacks can no longer be dropped from the formats userspace has already seen, so a stream on one is refused with -EINVAL instead. For the same reason a deferred start keeps the device tree presets: the IMX958 configuration discovery and P010 preview, which add formats at detection, stay with starts at probe. A KUnit case checks that only resident firmware defers its start. Signed-off-by: Erik Bourget <erik@ebourget.net>
The T8132 (M4) ISP is T6040's: the same coprocessor layout, firmware interface and boot path, with the bootloader's firmware described as reserved memory. It differs in: - two AMCCs (ADT mcc amcc_aperture_count), so two DSID broadcast-clear windows, passed one per AMCC without padding slots; - no PMP_CTRL_SET: T8132's PMP scratch offsets are not established, and the camera streams without the command. The firmware's MMIO windows still cover those PMP registers; - the firmware's MMIO windows are also opened in the DART's address filter. The firmware is resident as on T6040 and T8140. A second start after its domains were cycled fails on T8132 too: the ISP CPU cores then do not reach their active power state. So it runs while the driver is bound, and T8132 defers its start to the first stream, keeping the ISP off until the camera is used. T6040 and T8140 have not been tested with the deferred start and keep starting at probe. Signed-off-by: Erik Bourget <erik@ebourget.net>
Reserve the ISP firmware segments iBoot loads, from the ADT's isp segment-ranges, and the firmware's main memory above them. Enable the ISP with the J713's platform ID, the ADT isp node's sensor-type, and the front IMX958's 1080p and 720p presets, configuration 5 as on the J616s. The ISP stays off until the first stream. From then on it stays powered until the driver is unbound, as on the other resident ISPs; on the J713 that costs about 100 mW at idle: 4.13 W against 4.03 W with the ISP off, mean battery power over 180 s with the display on. Signed-off-by: Erik Bourget <erik@ebourget.net>
…tart Firmware that the boot loader loads changes its data segment while it runs, and does not start again from that state. Add hw->fw_restore: probe keeps a copy of the data segment, taken before the firmware first runs, and every start puts it back and clears the firmware main memory above it. Stopping the firmware puts it back too, also after a start that failed, and a shutdown stops a stream and such firmware, so that a rebound driver or a kexec'd kernel finds the firmware as it was loaded. No SoC uses it yet. Signed-off-by: Erik Bourget <erik@ebourget.net>
Firmware that is started again after it stopped (hw->fw_restore) has to be stopped for good. SUSPEND leaves the camera module powered, with its light on, and it stays so even across a reboot. Send STOP before POWER_DOWN instead, which also powers the module down; POWER_DOWN follows even if STOP failed. This replaces the unused STOP sequence that was kept under #if 0. Some coprocessors also need a quiesce before their domains go down. Its CPU core domains otherwise stay busy after the power-down and never run again, so the next start of the firmware fails. Add hw->coproc_quiesce, the register whose bit 25 requests the quiesce and reads back once it has happened, and hw->coproc_status. With a quiesce register, the power-down of a coprocessor that was started waits for it to reach WFI, stops it, quiesces it, and only then powers its domains down. A start that fails after the coprocessor was started thus also leaves it ready for the next one. No SoC uses them yet. Signed-off-by: Erik Bourget <erik@ebourget.net>
T8132 firmware starts again after it stopped, from its data segment as loaded, once the coprocessor was quiesced before the power-down. Stop keeping it resident: restore its data before each start, quiesce it before each power-down, and power the ISP down after every stream, as the older SoCs do. The camera then costs nothing when it is not in use, not only until its first use, and can be used again after the firmware stopped (for example across hibernation). The deferred start no longer needs resident firmware, so it becomes deferred_start: the camera is detected at the first stream, which with firmware that starts again boots it a second time to stream. A deferred detection that fails is reported to the stream instead of deferring a probe. Metadata buffers are reset for each stream of firmware that starts again, as for resident firmware: none survives the restart. Signed-off-by: Erik Bourget <erik@ebourget.net>
This was referenced Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The J713 (MacBook Air 13" M4) camera works with this series:
/dev/video0streams NV12 at 1920x1080 and 1280x720, and the ISP is powered only while it streams: it is off from boot until the first stream, and again after every stream.The T8132 ISP is T6040's ISP with a different power topology and two AMCCs, so the series reuses the T6040 path and adds what T8132 needs.
Patches
dt-bindings: iommu: apple,dart: Add the address filter registers: an optional secondregrange,dapf, for the DART's address filter (DAPF) registers. If a node has two ranges,reg-namesis required.dt-bindings: media: Add the Apple T8132 ISP: its own schema, as for T6041 and T8140. It has T6041's register layout and firmware memory model, and T8140's fourteen ISP power domains plus two camera sensor power states.arm64: dts: apple: t8132: Add the ISP power states: the thirteen ISP power states below ISP_SYS (PMGR + 0x4000) and ISPSENS0/1. They are disabled in the SoC file likeps_isp_sys, and only the J713 policy enables them, so the other T8132 boards are unchanged.arm64: dts: apple: t8132: Add the ISP and its DARTs: the ISP and its three DARTs, disabled. The first DART carries thedapfrange.media: apple: isp: Program the DART address filter: an optional per-SoC window list, programmed after every power-up and read back. Two KUnit cases cover it.media: apple: isp: Let resident firmware start at the first stream: a SoC can defer its resident firmware's start, and the camera detection, from probe to the first stream start. One KUnit case covers it.media: apple: isp: Add T8132 support: the hardware entry: T6040's interface, two DSID broadcast-clear windows (one per AMCC), noPMP_CTRL_SET, and resident firmware with the deferred start (patch 11 makes it restartable).arm64: dts: apple: t8132-j713: Enable the camera: the firmware segments as reserved memory (from the ADT's ispsegment-ranges), the platform ID, the IMX958 presets, and the DARTs.media: apple: isp: Optionally restore the firmware data before each start:hw->fw_restore. Probe keeps a copy of the firmware's data segment, as the boot loader loaded it, and every start and stop puts it back. A shutdown stops a stream and the firmware, so a rebound driver or a kexec'd kernel finds the firmware as it was loaded.media: apple: isp: Power restartable firmware down completely: such firmware is stopped with STOP and POWER_DOWN rather than SUSPEND, which powers the camera module and its light down too. A coprocessor with a quiesce register (hw->coproc_quiesce) is quiesced after it reaches WFI and before its domains go down; without that its CPU cores stay busy and never start again.media: apple: isp: Power the T8132 ISP down between streams: T8132 uses both, and its firmware is no longer resident. The ISP powers down after every stream, as on the older SoCs. The deferred start no longer needs resident firmware, soresident_fw_deferredbecomesdeferred_start. A KUnit case checks the flag combinations.Power
VIDIOC_STREAMONdetects the camera, which boots the firmware once for detection and again to stream.-EINVAL. For the same reason it keeps the device-tree presets: the IMX958 configuration discovery and P010 preview from media: apple: isp: add IMX958 P010 previews and firmware config discovery #462 add formats at detection, so they stay with starts at probe. T6040 and T8140 are unchanged; they haven't been tested with the deferred start.sync_state).The address filter
The ADT gives the first ISP DART address filter windows for the fixed MMIO windows the ISP's coprocessor reaches: the ISP power states, the DSID broadcast-clear windows and PMP registers. The filter loses them whenever ISP_SYS is off and powers up with arbitrary contents.
The camera also streams with the filter cleared, and with it left as it powers up. So this isn't required for the camera to work. The driver programs the windows as the ADT describes rather than leaving the filter undefined:
fw_mmiowindows it maps into the same DART for the firmware, so the per-SoC list sits next tofw_mmio. The DART driver itself has no use for them.reg. The ISP driver finds them through itsiommusphandle and maps them non-posted, like the rest of the SoC's MMIO.If you'd rather leave the filter to m1n1, or not touch it at all, patches 1 and 5 and the
dapfrange drop out cleanly.Testing (J713)
Load: with the driver bound,
/dev/video0is present, and all sixteen ISP domains are off, also after listing formats. The power-domain devices are runtime-suspended.Streams: NV12 1920x1080, then 1280x720, 30 frames each, with no errors. 720p scales the full 1080p field; it is not a crop. While streaming all sixteen domains are on, and
/proc/iomemshows48c000000.isp dapf.Power-down and restart, one boot:
Every stream delivered all of its frames. After each one all sixteen ISP domains were off with the CPU core domains in their cold-boot state, and the camera light was off. It came on only while streaming.
Reboot while streaming (
systemctl reboot -ff, so the stream is still running when the kernel shuts the device down): the light goes off, and the next boot finds the ISP CPU core domains in their cold-boot state.Base: rebased onto
integration/aurora41c4055 with the ISP driver's IOVA allocation withoutdrm_mm(f729605) and the P010 timeout recovery (504cb5d). The commits are unchanged. On that base: four 30-frame streams (1080p, 720p, 1080p, and 1080p after a module reload) delivered every frame. All sixteen ISP domains were off after each one, with the CPU core domains in their cold-boot state. Across 222 s of s2idle, a 600-frame stream carried on and completed, and a stream after resume worked.Filter: separately, the camera streams with the filter windows cleared and with them left as they power up.
Static checks:
--strict: clean on all eleven commits, except the MAINTAINERS note for the newisp-drv-test.c;dt_binding_check: clean for both bindings;CHECK_DTBS+ W=1: no new warnings on any of the six T8132 DTBs;isp-ipc.c.KUnit on arm64 QEMU: 33/33 on the new base, including the new
apple-isp-dapfandapple-isp-hwcases.Not covered
PMP_CTRL_SETis not sent. The camera streams without it.apple/isp_2224_01XX.dat, the IMX958 calibration file it already uses for T6040. The installer has no J713 entry yet, so that's a follow-up.This change was developed and tested by Claude (Anthropic), working with me on the J713.