Repository navigation
Read quiet writer directories less often, and slow down in the background - #101
Merged
Merged
Conversation
…ound (#52) A scan now reads a writer directory on every pass only while it is active. One where nothing has changed for 5 minutes is read every 30 s, and a new one is read at once. At 1,000 directories a quiet tick drops from 76 ms to 0.5 ms on a local disk. The ledger takes a Cadence and gains pace() and refresh(); scans queue and never overlap. The app paces every ledger by window focus: 1 s while focused, 5 s (quiet every 2 min) in the background, and a full read when a window gains focus. The numbers are under "Noticing change" in multi-writer.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
createLedgers takes its two paces, defaulting to the app's, so a test can use one that never scans by itself. The tests write from a second session and check that gaining focus reads at once, that the background pace holds until then, and that a ledger opened in the background starts at it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A directory first seen empty, as sync clients create folders before their files, now counts as just changed instead of quiet, so a new session's first segment is read on the next scan. - The cadence is timed on performance.now(), so a wall clock set back cannot stop quiet directories being read. - Gaining focus refreshes only the project on screen; refreshes asked for before one starts share it. - openOrCreate and pace refuse a cadence that is not one, before anything is written. - The spec says why size is enough to tell a change, and how old the status of a quiet directory can be. app/CLAUDE.md says a tested shell module stays loose. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # app/CLAUDE.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #52.
What changed
Writer directories pile up, one per session that writes, and almost all of them have ended. The 1 s poll read every one of them every second, so its cost grew with how long a project had been in use. Now:
The ledger tiers directories. A directory is read on every scan while it is active. One where nothing has changed for
quietAfterMsis read everyquietEveryMs, and never skipped for good. A new directory is read at once. "Changed" means the segment list, or the size of the segment being read, differs from the last read; on first sight the newest segment's modification time stands in, and a directory with no segment yet (sync clients create the folder before its files) counts as just changed. Neither orders or gates anything, so a wrong guess costs reads, never correctness. The cadence is timed on a monotonic clock, so a wall clock set back cannot stop the reads.Cadence,pace()andrefresh().openOrCreate(path, cadence)replacespollMs.pacechanges the cadence from the next scan, andrefreshscans now, reading every directory. BothopenOrCreateandpacethrow aTypeErroron a cadence that is not one, the oldpollMsnumber included, andopenOrCreatedoes so before writing anything. Scans queue and never overlap; refreshes asked for before one starts share it.The app paces by focus.
setFocusedinapp/src/main/ledgers.ts, wired tobrowser-window-focusandbrowser-window-blurinindex.ts.createLedgerstakes its two paces, defaulting to these:Gaining focus also refreshes the open project's ledger, so what comes on screen is never behind by the background pace. The other projects kept open this session are only paced, not read in full on every focus.
Spec. "Noticing change" in
packages/ledger/multi-writer.mdnow has the cadence, what counts as a change and why size is enough, how old a quiet directory's status can be, the app's pace, and why there is still nofs.watch.app/CLAUDE.md. A shell module that gets a unit test stays loose, with the test beside it (ledgers.ts,ledgers.test.ts), so it no longer reads as breaking the folder-per-tested-module rule.Measurement
One scan on a local disk (APFS), median of 200, every directory but one aged an hour:
Not measured inside a sync client's folder.
Testing
refresh, then active again; a quiet one is still read everyquietEveryMs; a directory written just before open is active from the start; a directory first seen empty is active, so its first segment is read at once; a wall clock set back 10 minutes does not stop quiet reads; a bad cadence is refused before anything is written; refreshes asked for before one starts share it;pacechanges the cadence; nothing scans afterclose.app/src/main/ledgers.test.ts, each writing from a second session: gaining focus reads the project on screen at once; it reads only that one, not other open projects; the background pace holds until focus returns; a ledger opened in the background starts at the background pace.Date.now(), the matching test fails.npm test(all workspaces),tsc --noEmitandeslintpass.Review
A review of this PR found eight issues. Six are fixed in the last commit: the empty first sight, the wall clock, refreshing every project on focus, unvalidated cadences, the
app/CLAUDE.mdrule conflict and the missing test. Two needed no code change and are now explained in the spec. Change is told by size only, but the key decides how often a directory is read, never what a read folds, so a same-size fill-in (such as an online-only placeholder) is folded on the next read either way. A quiet directory's status can be up toquietEveryMsold, because keeping it current would mean listing every directory every scan, which gives back half the saving.In the running app
These runs predate the review fixes. None of the fixes changes the cases timed below, but they were not rerun afterwards.
The dev app ran with a scratch profile against fresh scratch projects. A driver wrote blocks from another session (
openWriter, and once the realpbnotes append) and timed each from the write being on disk to the block arriving in the window. "Quiet" directories were seeded with segments dated an hour back before the app first saw them. Two back-to-back runs, with nobody at the machine:pbnotes append, focusedThe quiet cases sit on their limits because each write came right after a full read, which is the worst case. In both runs every focus and blur the app heard was one the driver made, and a 250 ms timer in main was never more than 100 ms late. A log of every directory read matched the rules: active directories every 1 s focused and 5 s in the background, quiet ones on focus, 30 s later, then every 120 s in the background.
One unexplained miss. In an earlier run, a focused write to a quiet directory had not arrived after 45 s. It was folded later, and the next quiet write, in the background, arrived after 9.6 s rather than up to 2 min. It has not come back since: three full runs and two focused-only checks gave 27.6 to 30.0 s, and the same sequence in plain Node gave 30.0 s. That run logged neither directory reads nor focus timestamps, and a later run was spoiled by a stray blur from desktop activity, so a focus change the driver did not make is the likely cause, but it is not confirmed.
🤖 Generated with Claude Code