fix: docker check never modifies the project; atomic catalog cache save (#951) - #956
Merged
Merged
Conversation
…race (#951) SaveToFile now writes to a temp file in the cache's directory (VACUUM INTO, manual-copy fallback into the same temp file) and renames it over the cache. buildCatalog and refresh catalog communities no longer remove the cache first. Opening a cache at the current schema version no longer writes to it: a write on a file renamed underneath an open connection fails with SQLITE_READONLY_DBMOVED. File-backed connections get a busy_timeout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docker check ran mx update-widgets on the user's project with only the MPRv2 storage snapshotted, so an MPRv1 .mpr was rewritten permanently, and mx check itself rewrote theme-cache/ and created deployment/sass/ on both formats, with or without --no-update-widgets. Both mx steps now run on a temporary copy (build output, caches and VCS folders skipped), mx output is rewritten to the project's own paths, and the output says that widgets were normalised on a copy and what that hides (#568, #646). docker build keeps its snapshot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nto itself (#951) Check copies the project's directory, so a missing .mpr (or a path in /tmp) would copy an unrelated directory: fail early instead, and give the existing fake-mx tests a project file of their own. With TMPDIR inside the project the walk met its own copy and recursed until the path was too long; skip it. Trim the custom-widgets skill back under the 700-line limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #951.
1.
docker checknever modifies the projectMeasured before the fix (11.13 project, whole-tree sha256 + mtime diff before/after):
App.mprrewritten permanently,theme-cache/web/theme.compiled.css(.map)rewritten,deployment/sass/main.scsscreated.mprcontents/*.mxunitrewritten (restored by the snapshot, new mtimes), plus theme-cache/deployment as above.--no-update-widgets, on v1 and v2:mx checkitself rewritestheme-cache/and createsdeployment/sass/. So the copy is needed for both modes, not only for update-widgets.Fix (
cmd/mxcli/docker/check.go, newcheck_copy.go):Checkcopies the project to$TMPDIR/mxcli-check-*and runsmx update-widgets(unless--no-update-widgets) andmx checkthere. It skipsdeployment/,releases/,theme-cache/,.mendix-cache/,.mxcli/,.docker/at the root, and.git/.svn/.hg/node_modulesat any depth. A symlinked file is copied as a file, so nothing can write through it. mx output is rewritten line by line from the copy's path to the project's path. The copy is removed afterwards. A missing project file fails early. With$TMPDIRinside the project, the walk skips its own copy. The output says a temporary copy is checked. With update-widgets it also prints a note: widgets were normalised on the copy, so a CE0463 the stored project still has is not reported;--no-update-widgetschecks the project as stored;mxcli fix widgetsapplies the normalisation. This is the honesty #568/#646 ask for; those issues stay open for their other items.--help,docs-site/src/guides/marketplace.mdand the custom-widgets / migrate-design-prototype skills (they saiddocker checkclears CE0463) are updated.Performance: on this ~37 MB project, the check took 12 s (v1) and 15 s (v2) with update-widgets, and 4–6 s without. The copy is the model plus the widgets/theme/source folders.
Other docker subcommands:
lint/reportdon't run mx.status/logs/up/down/shelldon't touch the project.docker buildstill usesrunUpdateWidgets(v2 snapshot) and is left alone, as agreed. See follow-ups.2. Catalog cache save race
Repro (
/home/vscode/t/jts/racecopy, 16 parallelmxcli lint -pon a fresh copy): before the fix, 4×failed to create table catalog_meta: table catalog_meta already existsand 3×database is locked. After: 0 errors in 3×16 + 5×8 runs.integrity_checkis ok and no temp files are left.Fix:
Catalog.SaveToFilenow doesVACUUM INTO(or the manual-copy fallback) intoos.CreateTempin the cache's directory, thenos.Renameover the cache.buildCatalogandrefresh catalog communitiesno longeros.Removethe cache first.attempt to write a readonly database, 1032), andNewFromFilealways wrote (createTablesplus the schema-version row). So opening a cache already at the current schema version is now read-only.busy_timeout(10000)in the DSN, which covers the remaining writes when an old-version cache is upgraded.Test plan
TestSaveToFile_ConcurrentWritersAndReaders(mdl/catalog): 8 goroutine writers over an existing cache, plus 4 reader loops. It asserts no write or read error, a final cache in the new mode, and no leftover files. Revert check: on the unfixed code it fails withcatalog_meta already exists×8 and readers'database is locked. With only the rename, it fails withattempt to write a readonly database(which is why the read-only open is needed).TestCheck_DoesNotModifyProject(v1, v2, each with and without--no-update-widgets): stub tools do to their target what the real ones do (rewrite the .mpr, drop mprcontents, write theme-cache and deployment). The test asserts the tree is identical, mx never points into the project, the copy's path does not leak into output, the "normalised on a temporary copy" note appears, and the temp dir is cleaned up. Revert check: with mx pointed back at the project it fails withchanged App.mpr,changed theme-cache/...,added deployment/...andremoved mprcontents/....TestCopyProjectForCheck_SkipsOutputs,_TempDirInsideProject(revert check: without the skip, the copy recursed until mkdir failed on path length),_MissingProject.TestCheck_LeavesProjectUntouched(-tags integration, real mx 11.x,mx create-project, v2 plus v1 converted via update-widgets, both flag settings): PASS. Revert check: v1 fails with "docker check modified the project".TestCheck_PreservesMPRv2StorageFormatstill passes.--no-update-widgets. Control: a microflow withdeclare $n Integer = 1 + 'a'is still reported ([CE0117], exit 1) on both formats, with the tree unchanged.go test ./mdl/catalog/ ./mdl/executor/ ./cmd/mxcli/docker/ ./cmd/mxcli/,make build,make lint,make check-conformance,make check-findings,make check-skill-mdl,make sync-skills.mdl-other.jsonl,cmd-mxcli.jsonl). CHANGELOG entries are under Unreleased → Fixed.Follow-ups (not in this PR)
docker buildstill rewrites an MPRv1.mprvia update-widgets, andmx checkthere writes theme-cache.buildwritesdeployment/by design, but the model rewrite on v1 is the same defect. It could use the same copy for its pre-check, though MxBuild then needs the normalised model.cmd/mxcli/tui/checker.go) andevalrunner(cmd/mxcli/evalrunner/checks.go) run plainmx checkon the project, which writestheme-cache/anddeployment/sass/. They are not docker subcommands, so they were left alone.refresh catalog communitieson a loaded cache viaAddGraphAnalysis) can still hitreadonly databaseif another process renames a fresh cache over it mid-run. This is rare, and it surfaces as an error rather than corruption.🤖 Generated with Claude Code