Skip to content

engine: on macOS a held chat's folder costs one watch handle, not one per file (#2471) - #2551

Merged
AbirAbbas merged 4 commits into
app/v3from
fix/2471-darwin-folder-watch
Nov 9, 2026
Merged

AbirAbbas merged 4 commits into
app/v3from
fix/2471-darwin-folder-watch

Conversation

@AbirAbbas

@AbirAbbas AbirAbbas commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes most of #2471. On macOS a held chat's session folder now costs one kernel watch handle instead of one per file in it.

Cause

internal/session/presence_watch.go gives each held chat's session folder to fsnotify. On macOS, fsnotify's kqueue backend opens a descriptor for every file in a watched folder so that it can report writes to each one: transcript, meta, lock, presence, card and told files. A long-lived app keeps that many descriptors for every chat it has shown. On Linux (inotify) and Windows it is one handle per folder.

Change

The watch only needs request files to appear: takeover.json, .conversation-delete-request and .task-delete-request-*. Every writer publishes them by creating a file or renaming one into that folder (AskTakeover, writePresenceRequest).

  • darwin (presence_watch_darwin.go): one kqueue, one O_EVTONLY handle per folder, EVFILT_VNODE with NOTE_WRITE|NOTE_DELETE|NOTE_RENAME|NOTE_REVOKE. When the folder's entries change, it lists the folder and reports the request names present. A removed or renamed folder is dropped, and the existing safety pass adds it again. The reader blocks with no timeout; Close and Remove wake it with an EVFILT_USER event, so an idle engine gains no wakeups.
  • answers.jsonl on darwin: since Factory: any-source intake, router and AI filter (draft, e2e first) #2478 a quiet session skips its doorstep pass for a minute, because the watch wakes it when an answer is written (presenceDoorstepEvent). deliverAnswer appends to answers.jsonl, and once that file exists an append doesn't change the folder's entries. So the darwin watcher also watches that one file while it is there:
    • a write reports the answer;
    • a created but still empty file reports nothing;
    • a drain's rename gives the handle back.
  • other platforms (presence_watch_fsnotify.go): fsnotify exactly as before, behind a small doorWatcher seam. The loop, safety pass and clocks are shared.
  • Not changed: holds, viewOpen, the idle sweep, heartbeat and refresh cadence. (Let idle desktop chat views beyond the three most recent go #2472 changed holds and broke live turns started by another process, so it was reverted in Revert: let idle desktop chat views beyond the three most recent go #2473.)
  • Extra wakes: a folder change that is not a request can report a request that is already there. That is harmless, because each request check re-reads and claims the actual file (conversation_delete.go, task_delete.go, takeover.go).

Contract → tests

Behaviour Test
C1 darwin: one handle per watched folder, however many files it holds; more files add none TestDarwinPresenceDoorC1OneHandlePerFolder
C2/C3 a published request wakes that folder's clients by event, and no other folder's TestDarwinPresenceDoorC2PublishedRequestsWakeByEvent, TestPresenceDoorC2C3RoutesRequestEvents
C4 a removed or replaced folder is dropped, then watched again by the safety pass TestDarwinPresenceDoorC4ReplacedFolderIsWatchedAgain, TestPresenceDoorC4SafetyPassReaddsDroppedFolders
C5 the last client closes the handles; cancel is idempotent; restart works TestDarwinPresenceDoorC5ClosesHandlesAndRestarts, …CloseWithoutDrainingEvents, TestPresenceDoorC5CancelClosesOnceAndCanRestart
C6 a watcher error runs the bounded safety pass and knocks every session once (#2478) TestPresenceDoorC6ErrorsRunBoundedSafetyPassAndKnock
C7 non-darwin keeps fsnotify delivery TestPresenceDoorC7KeepsFsnotifyDelivery
C8 ordinary file events wake nobody TestPresenceDoorC8IgnoresOrdinaryFileEvents
C9/C10 darwin: an answer appended to a new or waiting answers.jsonl wakes by event, and an empty file doesn't; the file costs one handle while it exists, which a drain gives back TestDarwinPresenceDoorC9C10AnswersWakeAndReleaseTheirHandle

The TestDarwin… tests only run on a Mac, and CI here doesn't run tests on macOS.

Checks

  • Compile gates, with GOFLAGS=-buildvcs=false: gofmt -l internal/session (empty), go vet ./internal/session/ (linux and darwin/arm64), go test -c ./internal/session (linux and darwin/arm64), GOOS=windows go build ./internal/session. All clean.

  • Linux, run locally: the 6 TestPresenceDoor* tests plus Factory: any-source intake, router and AI filter (draft, e2e first) #2478's quiet and idle presence tests (presence_quiet_test.go, presence_idle_test.go), 15 tests, all pass. codeaf-changes check is clean.

  • Mac proof (macOS 15.6, Apple Silicon; codeaf app driven over its wire in a throwaway HOME; first run at 52ad985 vs its base 32747af). The change since then: the rebase onto Factory: any-source intake, router and AI filter (draft, e2e first) #2478, the answers.jsonl watch, and a test-helper fix. Re-run at the current head 1cfcceb: see below.

    engine open files (lsof -p | wc -l) hello 10 held chats per held chat all closed
    base 47 110 +6 47
    head 47 68 +2 47
    • Per chat on base: the journal's write handle, the folder, meta.json, meta.lock, a second read-only transcript.jsonl and presence.json. The last four are fsnotify's per-file kqueue handles. On head: the journal's write handle and the folder. kqueue stays at 3 on both, so there is one watcher in total, not one per chat.
    • Takeover request, published as AskTakeover does into a held chat (3 runs each): base takes it off disk in 3–4 ms and reacts on the wire in 1–2 ms; head takes 3–4 ms and 1–3 ms.
    • Idle 2 min on head: the count stays flat. No Keychain or password dialog appeared, judged by SecurityAgent process checks at every step.
    • Darwin unit tests: 11 of 12 passed. C1 failed inside its own fd-count helper (os.ReadDir("/dev/fd") gets EBADF on macOS). The helper now counts names, and with that all 12 passed (3/3).
  • Mac re-run at 1cfcceb (after the rebase onto Factory: any-source intake, router and AI filter (draft, e2e first) #2478 and the answers.jsonl watch):

    • 20 tests pass 3 of 3: the 7 darwin and portable PresenceDoor tests, C9/C10, and Factory: any-source intake, router and AI filter (draft, e2e first) #2478's quiet, idle and watcher-delivery presence tests.
    • Engine open files: 48 at hello, 69 with 10 held chats, +2 per chat at every step, back to 48 when all close. Reopening 10 gives +2 each again.
    • Takeover is taken off disk in 3–4 ms and reacted to on the wire in 1–3 ms.
    • answers.jsonl, sampled every 1 ms: an empty file gets its own handle within 1.7 ms and wakes nothing for 2 s. Two appended lines are drained within 0.5 ms, the handle is released within 2 ms, and nothing is left behind. With no file first, the drain takes 2.3 ms and the handle goes at 3.7 ms.
    • No Keychain or password dialog.

What stays per chat

Files a held chat keeps open for its own use stay open while it is held. Releasing them needs a "view closed" call from the desk, a separate two-repo change. agents.close can't serve, because it is the person's Close. The Mac lsof listing will name which files remain.

🤖 Generated with Claude Code

AbirAbbas and others added 3 commits November 9, 2026 17:43
… per file (#2471)

The presence watch handed each held chat's session folder to fsnotify, whose
kqueue backend on macOS opens a descriptor for every file in a watched folder
(transcript, meta, lock, presence, card, told files). A long-lived app kept
about that many descriptors per chat it had shown. Linux and Windows already use
one handle per folder.

The watch only needs request files to appear (takeover.json,
.conversation-delete-request, .task-delete-request-*), and every writer
publishes them by create or rename in that folder. On darwin a kqueue watch on
the folder itself now sees those entry changes and lists the folder for
requests; a removed or renamed folder is dropped and re-added by the safety
pass as before. Other platforms keep fsnotify behind the same small seam.
Holds, viewOpen, the idle sweep and the heartbeat clocks are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…jsonl

#2478 lets a quiet session skip its doorstep pass for a minute because the
watch wakes it when an answer is WRITTEN. deliverAnswer appends to
answers.jsonl, and once that file exists an append does not change the folder's
entries, so a folder-only kqueue watch missed it. The darwin watcher now also
watches answers.jsonl while it is there: a write reports the answer, an empty
just-created file reports nothing (the line comes after the create), and a
drain's rename gives the handle back. A held chat costs its folder handle plus
one for an answers file only while an answer waits.

TestPresenceDoorC6 follows #2478's contract: a watcher error now knocks every
session once as well as running the safety pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AbirAbbas
AbirAbbas force-pushed the fix/2471-darwin-folder-watch branch from 52ad985 to 47c07cb Compare November 9, 2026 22:48
os.ReadDir("/dev/fd") stats every entry and fails with EBADF inside a test
binary on macOS 15.6, so TestDarwinPresenceDoorC1 failed before any watch was
made. Readdirnames only counts names (found by the Mac proof run).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AbirAbbas
AbirAbbas marked this pull request as ready for review November 9, 2026 22:57
@AbirAbbas
AbirAbbas merged commit 0dc2bd1 into app/v3 Nov 9, 2026
6 checks passed
@AbirAbbas
AbirAbbas deleted the fix/2471-darwin-folder-watch branch November 9, 2026 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant