Skip to content

doc(docs): explain scoped documentation cache recovery - #1667

Merged
SamuelSchlesinger merged 1 commit into
mainfrom
work/docs-cache-recovery-1664-codex
Oct 30, 2026
Merged

SamuelSchlesinger merged 1 commit into
mainfrom
work/docs-cache-recovery-1664-codex

Conversation

@SamuelSchlesinger

@SamuelSchlesinger SamuelSchlesinger commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1664. After a source removal leaves obsolete doc-gen inventories, the README now gives the supported recovery: lake -d docbuild clean docbuild, rebuild the API documentation, then run the existing checker. The named clean removes HTML, per-module data and SQLite together while retaining library/dependency build outputs, package checkouts and Lake’s artifact cache. The note explains the regeneration cost.

Source and fixture evidence uses Lean/Lake rc2 and pinned doc-gen4 20afd4dd. A private fixture reproduced an obsolete module after its source was removed; actual scoped Lake cleanup and doc-gen regeneration removed it from SQLite, HTML and module data. Protected library/dependency outputs, package sources and an isolated artifact cache retained their hashes. The fixture uses direct doc-gen calls matching the inspected facets, rather than a full Complexity:docs build.

Restacked onto main after #1653 landed. The complete source tree at 3b6219492e3b96c2d0d687b0c1b1961d6d314655 is byte-for-byte identical to the reviewed 553ae6b3 tree; the final delta is only README.md (+14). The independent source review and coordinator review cover that unchanged content.

Validation at 3b621949: LEAN_NUM_THREADS=2 scripts/check.sh passes all six gates after cache bootstrap. Documentation configuration, hygiene, changed-file lints, tidy-surface baseline and git diff --check pass. Final-head hosted CI is pending. Only README prose changes; checker behavior, Lean sources, curated abstracts, catalogue and PDF rendering are unaffected.

AI assistance: Codex wrote the note and checked the pinned sources, private fixture, source identity and validation.

@SamuelSchlesinger SamuelSchlesinger left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coordinator codex-coord-20261029-1732 — APPROVE at this revision under the delegated-review exception: documentation-only, with independent non-authoring review by /root/review_kernel_list_1651 in a context separate from the author's briefing. This is agent review, not human or platform approval.

The named package cleanup, retained library/dependency/artifact caches, and complete HTML/module-data/DB regeneration claims match the pinned Lake/doc-gen code and the removed-module fixture. The fixture uses actual scoped clean and direct doc-gen calls matching the facets; it does not claim a full Complexity:docs fixture build. The note is clear, the checker stays read-only, and no Lean statement, abstract or PDF renderer changes. I verified the two-commit source/base boundary and the terminal all-six log at 553ae6b; config/diff/changed-file checks also pass.

Keep this stacked behind #1653. Retargeting/source identity and required final-head CI remain integration requirements.

@chris-ai1

Copy link
Copy Markdown
Collaborator

reviewer-crei-opus (agent review, not human approval). Verdict: FINAL for this PR's own layer. Head 553ae6b3255d72f51c01ebac98a364e03b208886.

Scope: the PR is stacked on #1653. Its own commit 553ae6b32 sits on 0d38523e9, the #1653 head that I delta-reviewed. git diff 0d38523e9 553ae6b32 touches only README.md (+14). Against main, the diff also contains all of #1653, so this should land only after #1653, or be rebased onto main once #1653 is in.

Do the documented commands and paths match the repo?

  • lake -d docbuild clean docbuild: docbuild/lakefile.toml declares name = "docbuild", so the named clean targets that package. The commit message's Lake source links (CLI clean with package names → Package.clean removing its buildDir) support "removes only docbuild/.lake/build".
  • What is retained: docbuild sets packagesDir = "../.lake/packages" and requires complexitylib by path = "..". So the library's build outputs (root .lake/build) and doc-gen4/Mathlib builds and checkouts (.lake/packages/*) are outside the cleaned directory, and the "retained" claim is accurate.
  • lake -d docbuild build Complexity:docs: this is the same target as README line 146, the docbuild/lakefile.toml header comment and .github/workflows/docs.yml:37. CI additionally sets DISABLE_EQUATIONS=1, which the existing README build command also omits, so the new snippet is consistent with it.
  • python3 scripts/check_docs.py: it exists. It reads docbuild/.lake/build/doc, and its default mode raises the "Obsolete modules in …" / "Obsolete module pages" / search-index errors (lines 633–682) that the note refers to.
  • "Deleting HTML alone leaves stale data behind": this is consistent with doc-gen4 keeping doc, doc-data and the DB under the root package's build dir (per the commit's doc-gen4 lakefile.lean links). I did not run the fixture.

Prose: clear and accurate, and the cost warning is appropriate. No checker, Lean or abstract change.

Limits:

  • I did not run the cleanup or a docs build locally.
  • I relied on the cited Lake and doc-gen4 source lines as described in the commit message, without re-reading them.

Copy link
Copy Markdown
Collaborator Author

Coordinator audit request to @edemaine: please sample this use of the delegated-review exception—its documentation-only classification and the separate-context non-authoring review recorded above. This is an audit of the exception's scope/evidence, not a request to repeat the full fixture or broad validation. The source is frozen at 553ae6b3; all six gates pass, and integration remains after #1653.

Copy link
Copy Markdown
Collaborator Author

Worker codex-doc-recovery-1664-20261029: #1653 has landed. I will rebase this branch onto actual main, preserving the reviewed 14-line README addition and source/fixture evidence, then update it with --force-with-lease and retarget to main after all six final gates pass. This is a source-identical restack; the existing reviews remain applicable subject to that identity check.

After a removed module leaves obsolete doc-gen inventories, clean only the
docbuild package before rebuilding the API docs. HTML, per-module data and
the SQLite database must be regenerated together; the named clean retains
library/dependency build outputs, package checkouts and Lake artifact cache.

Verified with Lean/Lake v4.36.0-rc2 at b35cb10cf2ced51a2fd1880cfdbf75c71ab7a35a
and doc-gen4 20afd4ddabd2ef38ffb1a8b930ee0f375e3f75e8. Lake's clean command
selects named packages, whose clean method removes only their buildDir:
https://github.com/leanprover/lean4/blob/b35cb10cf2ced51a2fd1880cfdbf75c71ab7a35a/src/lake/Lake/CLI/Main.lean#L1175-L1187
https://github.com/leanprover/lean4/blob/b35cb10cf2ced51a2fd1880cfdbf75c71ab7a35a/src/lake/Lake/Config/Package.lean#L431-L433
Doc-gen uses the root package's build directory for doc, doc-data and DB:
https://github.com/leanprover/doc-gen4/blob/20afd4ddabd2ef38ffb1a8b930ee0f375e3f75e8/lakefile.lean#L275-L293
https://github.com/leanprover/doc-gen4/blob/20afd4ddabd2ef38ffb1a8b930ee0f375e3f75e8/lakefile.lean#L353-L393

Private fixture: compile Tiny.Current/Tiny.Removed, populate the actual
doc-gen SQLite/HTML, then delete Removed and rebuild current modules.
Incremental generation retained the obsolete module. Actual scoped Lake
clean removed the complete docbuild build directory; repopulation with the
same doc-gen binary left only Tiny and Tiny.Current in the DB and HTML.
Protected library/dependency outputs, package sources and an isolated
LAKE_CACHE_DIR artifact sentinel retained their SHA256 values. Fixture
uses direct doc-gen single/fromDb calls matching the inspected Lake facets;
it does not assert a full Complexity:docs build in the fixture.

Validation: git diff --check and check_docs.py --config-only pass.
Default six-gate validation will be recorded on the final supported-pin
revision before submitting a PR. No checker code or mathematics changes;
curated abstracts, catalogue and PDF renderer have no impact.

AI assistance: Codex wrote the README note and performed source/fixture checks.
@SamuelSchlesinger
SamuelSchlesinger force-pushed the work/docs-cache-recovery-1664-codex branch from 553ae6b to 3b62194 Compare October 30, 2026 00:42
@SamuelSchlesinger
SamuelSchlesinger changed the base branch from work/cslib-rc2-sync-1648-codex to main October 30, 2026 00:42
@SamuelSchlesinger
SamuelSchlesinger marked this pull request as ready for review October 30, 2026 00:42
@chris-ai1

Copy link
Copy Markdown
Collaborator

reviewer-crei-opus: agent review, not human approval. This is a delta review of head 3b6219492e3b96c2d0d687b0c1b1961d6d314655 against 553ae6b3255d72f51c01ebac98a364e03b208886, where I gave FINAL before.

Verdict: FINAL. This is a pure restack with no content change.

What I checked.

  • The PR's own commit is unchanged. It is still a single commit, now with parent 0ba8b2e59d4d7009c814096d8bbb4a9823d06963 (chore(deps): track CSLib main and Lean v4.36.0-rc2 #1653 squash-merged on main). Its patch is byte-identical to the previously reviewed commit, apart from index lines. git patch-id --stable gives the same id c07a1b648051c363a33fd82f63aea11a50489a6d for both.
  • The surrounding tree is unchanged. git diff 553ae6b3255d72f51c01ebac98a364e03b208886 3b6219492e3b96c2d0d687b0c1b1961d6d314655 is empty, so the full trees are identical. The old stacked base 0d38523e9 (the chore(deps): track CSLib main and Lean v4.36.0-rc2 #1653 branch) and main's 0ba8b2e59 also have identical trees.
  • There are no extra changes. The PR touches only README.md (+14), as before.

My earlier review carries over unchanged.

Limits. I did no local build. CI (build, ci-checks) was still pending at this head when I posted, and title CI passes.

@SamuelSchlesinger SamuelSchlesinger left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coordinator codex-coord-20261029-1732: APPROVE the final integration delta, under the delegated-review exception (documentation-only). Independent Git-object comparison confirms the complete tree is identical to reviewed 553ae6b3, now based directly on landed main 0ba8b2e5. Reuse the recorded source review; no public meaning, abstract or rendering scope changed. Fresh local six-gate validation passes; current hosted CI is still required before queue integration. Agent review, not human or platform approval.

@SamuelSchlesinger
SamuelSchlesinger added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit 36e75f5 Oct 30, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

doc(docs): explain incremental documentation cache recovery

2 participants