fix(schema): make current V2 the sole fresh-install DDL owner

docs/deployment.md

@@ -75,13 +75,21 @@ docker compose logs -f cloudfile
 
 核心栈包括 `cloudfile`、`db`、`cache` 和 `proxy`。首次启动会初始化数据库与 Seafile
 配置；`scripts/scripts_14.0/start.py` 在每次启动时重写带 `CF_BEGIN`/`CF_END` 标记的
-CloudFile 配置块，并幂等应用 `cloudfile.sql`。
-
-新部署在启动 Server/Hub 前使用随镜像交付的版本化 schema runner 创建并校验当前
-DDL（包括资源表、统一编辑锁表及迁移记录），避免只创建旧集成表而遗漏写入屏障依赖。
-初始化意图保存在共享卷 `.cloudfile-schema-init-pending`；失败时停止启动并保留标记，
-重启后按正式 runner 的断点继续。既有数据目录且无该标记时不自动升级。
-文件锁与 Checkout 仍默认关闭，建表不代表启用能力。
+CloudFile 配置块。**当前 V2 不再自动执行历史 `cloudfile.sql`、001–042 版本化建表器和旧审计表建表器。**
+
+**仅全新安装**从 Server 随发行包交付的 `cloudfile-v2.sql` 建立三张核心表
+(`cf_library_storage`、`cf_dir_acl`、`cf_sso_group_map`)；
+明确启用了 `CF_ENABLE_CHECKOUT` 或 `CF_ENABLE_FILE_LOCK` 时，额外使用
+`cloudfile-v2-edit.sql` 建立当前单表 `cf_edit_guard`。
+初始化意图保存在共享卷 `.cloudfile-schema-init-pending`；失败停止启动并保留标记，
+不自动重放旧迁移，也不自动改写已有数据目录。
+
+**已有部署先运行只读结构盘点**，不要靠重新启动覆盖旧 Guard。检查命令：
+`python3 /scripts/cloudfile-v2-schema.py plan`，也可使用 `check` 获得非零退出状态。
+如果旧 `cf_edit_guard` 仍使用 `lifecycle_ref` / `lease_until` 等字段，
+Go 文件服务会明确以 `CF_EDIT_SCHEMA_MISMATCH` 拒绝启动，而不是让普通上传因
+SQL 1176 反复重试。完整停机、备份、数据审查和人工迁移边界见
+[当前 V2 数据库迁移说明](v2-schema-recovery.md)。
 
 ## 可选 profile
 

docs/directory-schema-upgrade.md

@@ -1,4 +1,11 @@
-# Directory authority schema
+# Directory authority schema (historical 001–042 runner)
+
+> **Not the current V2 schema entry.** This document is retained for offline
+> inspection of historical deployments. Current `dev` uses three V2 core
+> tables (optionally one current Guard); do **not** run `directory-apply` or
+> `SchemaRunner.apply` on a V2 database. Start with the read-only
+> `cloudfile-v2-schema.py plan` in
+> [current V2 schema recovery](v2-schema-recovery.md).
 
 `cf_dir_acl` and `cf_dir_admin` belong to the native Seafile
 database (`SEAFILE_MYSQL_DB_SEAFILE_DB_NAME`, normally `seafile_db`). Hub policy

docs/v2-schema-recovery.md

@@ -0,0 +1,110 @@
+# CloudFile V2 database contract and DEV upload recovery
+
+Current branch: **dev = V2**, no extra V2-named feature branches or repeated full builds.
+This document describes the **current V2 contract**, not the historical
+v0.x/v0.9 migrations in `cloudfile_extensions/schema/migrations/001..042`.
+
+## What failed
+
+- DEV Go fileserver observed SQL **1176** for
+  `cf_edit_guard FORCE INDEX(cf_edit_guard_active)`, followed by repeated
+  final Branch commits. This is a deterministic schema error, not ordinary
+  concurrent uploads.
+- The old `029_editing_core` creates `cf_edit_guard(resource_uid,
+  lifecycle_ref, lease_until, ...)`. Current V2 requires
+  `cf_edit_guard(repo_id, path_hash BINARY(32), path, lease_expires_at, ...)`
+  and `cf_edit_guard_active(repo_id,guard_id)`. Merely adding an index does
+  not repair the data model.
+- The old `008/042` ACL and `006/007` group-map migrations also conflict
+  with the current three-table V2 model.
+- Old Docker bootstrap and fresh-install path invoked legacy DDL. They now
+  use only `cloudfile-v2.sql` and optional `cloudfile-v2-edit.sql` on **new**
+  volumes; existing volumes never upgrade automatically.
+- Go Branch retries only true HEAD conflicts. Guard/ACL/SQL/layout errors
+  propagate immediately and do not write new Branch pointers.
+
+## Canonical current V2 tables
+
+| Area | Required schema | Ownership |
+| --- | --- | --- |
+| Library storage | `cf_library_storage` | V2 native storage |
+| Directory ACL, including `share_mode` | `cf_dir_acl`, binary SHA-256 path hash | Hub V2 authority |
+| UID, organization and role group projection | `cf_sso_group_map` | Hub V2 identity |
+| Optional Checkout/local editing | `cf_edit_guard`, one Guard row/path, `cf_edit_guard_active` | current V2 Guard |
+
+Core: **three** extension tables. With Checkout: **four**. Do not create
+`cf_resource`, `cf_commit_intent`, `cf_edit_session` or
+`cf_schema_migration` as a prerequisite for current V2.
+
+The final Go/C Guard checks preserve the lock and fail closed; a corrupt
+Guard is **not** treated as absent, and is not disabled to restore uploads.
+
+## DEV preflight (read-only, no build)
+
+Run within the matching release container with existing
+`SEAFILE_MYSQL_DB_*` environment. No credentials are printed:
+
+```sh
+python3 /scripts/cloudfile-v2-schema.py plan
+python3 /scripts/cloudfile-v2-schema.py check
+```
+
+The first command returns table states and **all extra `cf_*` table names**.
+The second exits non-zero when current V2 cannot be established. Both use
+`information_schema` SELECT only and never issue CREATE/ALTER/DROP,
+change schema ledgers or inspect user content.
+
+After confirming the exact database and that the table exists, optionally
+count active checkout reservations with a **read-only query**:
+
+```sql
+SELECT COUNT(*) AS active_guard_rows
+FROM cf_edit_guard
+WHERE guard_id IS NOT NULL;
+```
+
+Run that query only after verifying the `guard_id` column exists; active
+records, nonzero generations and foreign resource references must be
+preserved. Never infer that expired rows or an empty current session
+automatically make a legacy Guard safe to destroy.
+
+## Recovery of an existing DEV or production volume
+
+1. Freeze writes and capture the **deployed image digest/source receipt**, the
+   native database name, `SHOW CREATE TABLE` for every existing `cf_*`
+   table, the current migration ledger and Guard active/row counts.
+2. Back up MariaDB (including Seafile's Branch/Repo metadata), ccnet/seahub,
+   `/shared/seafile` and any object storage. Confirm recovery before DDL.
+3. Categorize **each** table as current, legacy, missing, or incompatible.
+   A mixed legacy `cf_dir_acl` / group-map / Guard layout is not a normal
+   additive upgrade. Do **not** turn a 1176 into silent authorization.
+4. Prepare an explicit, reviewed **offline** data migration for the actual
+   snapshot. Preserve original tables in separately named backups and map
+   old resource UID/lifecycle rows to canonical path and binary hash only
+   with confirmed identity, collision and generation checks. Stop on
+   ambiguous, active or orphaned records. Never issue an unconditional
+   `DROP TABLE`, truncate Guard, reset generations, mark old migrations
+   applied, or use `ALTER ADD INDEX` as the entire fix.
+5. After migration, restart the matching Server/Hub/fileserver images
+   together; the Go fileserver must report a **current Guard layout** on
+   startup. Retry only real concurrent Branch conflicts.
+6. Validate one small file with a normal Etech user and then a directory-only
+   `rw` shared user: upload link, bytes, final Branch commit, download,
+   rename, effective read/write, revoke, and root/sibling denial. Confirm
+   that neither ACL/Guard errors nor EXDEV storage staging errors recur.
+
+**No automatic legacy-to-V2 data migration is intentionally provided.**
+The correct transformation depends on the live data, active reservations,
+native library/group mappings and rollback evidence, which cannot be
+deduced from the three GitHub source repositories.
+
+## Limited cost verification
+
+- Go: targeted `TestCFV2GuardSchemaContract` and
+  `TestCFUploadRetriesOnlyHeadConflict`; C file-lock source now targets the
+  current one-table Guard.
+- Hub: `test_schema_inventory.py`, pure metadata classification without a
+  database.
+- Docker: `test_fresh_schema.py`, `test_v2_schema_inventory_command.py`.
+- PRs use quick static source checks; do not automatically run expensive
+  full C/Go builds, migration/recovery scripts or full E2E on every edit.

scripts/scripts_14.0/bootstrap.py

@@ -1384,12 +1384,16 @@ def write_cloudfile_config():
     write_cloudfile_seafile_conf()
     write_seafevents_search_config()
     write_seafevents_audit_config()
-    apply_cloudfile_schema()
-    apply_sso_hierarchy_schema_compatibility()
+    # CloudFile dev is V2. The only core database owner is cloudfile_ext.v2
+    # FreshSchema (three tables, plus one optional current edit Guard). Keep
+    # legacy helpers for historical manual recovery, but never automatically
+    # materialize old cf_sso_group_map/cf_audit_event or other v0.x tables:
+    # those would make a clean V2 install fail its exact-schema verification.
+    # Existing mixed deployments require an explicit offline plan; startup
+    # must not run legacy CREATE IF NOT EXISTS or silent ALTER migrations.
     apply_metadata_schema_compatibility()
     apply_tag_schema_compatibility()
     apply_starred_obj_id_schema_compatibility()
-    apply_audit_schema()
 
 
 def apply_sso_hierarchy_schema_compatibility():

scripts/scripts_14.0/cloudfile-v2-schema.py

@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+"""Read-only inventory of CloudFile V2 schema compatibility on an existing DB.
+
+plan  — report missing, legacy, incompatible and extra cf_* tables using only
+        information_schema SELECT queries.
+check — same read-only probe, non-zero exit if the core V2 contract or installed
+        optional Guard is not current. Never runs CREATE/ALTER/DROP or ledger
+        updates; schema/data migration requires an offline reviewed procedure.
+"""
+import argparse
+import json
+import os
+from pathlib import Path
+import sys
+
+
+def database_options(env):
+    required = ('SEAFILE_MYSQL_DB_HOST', 'SEAFILE_MYSQL_DB_USER',
+                'SEAFILE_MYSQL_DB_PASSWORD', 'SEAFILE_MYSQL_DB_SEAFILE_DB_NAME')
+    if any(not env.get(name) for name in required):
+        raise ValueError('explicit Seafile native database configuration required')
+    return dict(host=env[required[0]], user=env[required[1]],
+                password=env[required[2]], database=env[required[3]],
+                port=int(env.get('SEAFILE_MYSQL_DB_PORT', '3306')),
+                charset='utf8mb4', autocommit=True, connect_timeout=5,
+                read_timeout=10)
+
+
+def inspect(installdir, env, connect=None):
+    from cloudfile_schema import _v2_schema_dir
+    directory = _v2_schema_dir(installdir)
+    sys.path.insert(0, str(Path(installdir) / 'seahub'))
+    db = None
+    try:
+        from cloudfile_ext.v2.schema import FreshSchema
+        if connect is None:
+            import pymysql
+            connect = pymysql.connect
+        db = connect(**database_options(env))
+        # The optional table is inventoried even when Checkout is disabled.
+        # Read-only plan is metadata-only and never creates a table.
+        ddl = (directory / 'cloudfile-v2.sql').read_text()
+        return FreshSchema(db, ddl).inspect()
+    finally:
+        if db is not None:
+            db.close()
+        sys.path.pop(0)
+
+
+def main(argv=None):
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('command', choices=('plan', 'check'))
+    args = parser.parse_args(argv)
+    try:
+        from utils import get_install_dir
+        report = inspect(get_install_dir(), os.environ)
+        print(json.dumps(report, sort_keys=True))
+        return 0 if args.command == 'plan' or report['status'] == 'current' else 2
+    except Exception:
+        # Never print connection/driver exceptions, which could expose secrets.
+        print('CloudFile V2 schema inventory unavailable; check database and installed DDL',
+              file=sys.stderr)
+        return 1
+
+
+if __name__ == '__main__':
+    raise SystemExit(main())

scripts/scripts_14.0/cloudfile_schema.py

@@ -15,6 +15,11 @@ def prepare_fresh_schema(shared_dir):
     return marker
 
 
+def _v2_schema_dir(installdir):
+    """Packaged CloudFile Server DDL alongside the installed CE SQL."""
+    return Path(installdir) / 'sql' / 'mysql'
+
+
 def initialize_fresh_schema(marker, installdir, environment=None):
     marker = Path(marker)
     if not marker.exists():
@@ -26,7 +31,16 @@ def initialize_fresh_schema(marker, installdir, environment=None):
     connection = None
     try:
         import pymysql
-        from cloudfile_extensions.schema.runner import SchemaRunner
+        from cloudfile_ext.v2.schema import FreshSchema
+        schema_dir = _v2_schema_dir(installdir)
+        # One current V2 DDL owner; never replay the old 001..042 migration
+        # catalog into a fresh database (it creates an incompatible Guard,
+        # directory ACL and group mapping structure).
+        ddl = (schema_dir / 'cloudfile-v2.sql').read_text()
+        checkout = str(environment.get('CF_ENABLE_CHECKOUT', 'false')).lower() in ('true', '1')
+        file_lock = str(environment.get('CF_ENABLE_FILE_LOCK', 'false')).lower() in ('true', '1')
+        edit_ddl = ((schema_dir / 'cloudfile-v2-edit.sql').read_text()
+                    if checkout or file_lock else None)
         connection = pymysql.connect(
             host=environment['SEAFILE_MYSQL_DB_HOST'],
             port=int(environment.get('SEAFILE_MYSQL_DB_PORT', '3306')),
@@ -35,14 +49,17 @@ def initialize_fresh_schema(marker, installdir, environment=None):
             database=environment.get('SEAFILE_MYSQL_DB_SEAFILE_DB_NAME', 'seafile_db'),
             charset='utf8mb4', autocommit=True, connect_timeout=5,
         )
-        # apply() verifies DDL and checksums under the runner's database lock.
-        # Keep the marker on failure so partial MySQL DDL can resume safely.
-        SchemaRunner(connection).apply()
+        # Install only three reviewed V2 tables (plus the explicitly enabled
+        # optional one-table CF-EDIT). verify() rejects legacy/mixed layouts;
+        # never auto-convert old identity, ACL or active Checkout data.
+        # Keep the marker on failure; the operator must inspect and reconcile
+        # any incompatible existing structure instead of replaying old DDL.
+        FreshSchema(connection, ddl, edit_ddl=edit_ddl).install()
         marker.unlink()
         return True
     except Exception:
         # Driver errors can contain credentials; expose only an actionable cause.
-        raise RuntimeError('Fresh CloudFile schema initialization failed; inspect the schema ledger before retry') from None
+        raise RuntimeError('CloudFile V2 fresh schema initialization failed; inspect existing cf_* tables before retry') from None
     finally:
         if connection is not None:
             connection.close()

tests/test_fresh_schema.py

@@ -31,31 +31,44 @@ def test_failed_first_install_retains_intent_after_ce_initializes(self):
             self.assertEqual(SCHEMA.prepare_fresh_schema(directory), marker)
             self.assertTrue(marker.exists())
 
-    def _run(self, fail=False):
+    def _run(self, fail=False, editing=False):
         driver = ModuleType('pymysql')
         driver.connect = Mock()
-        runner_module = ModuleType('cloudfile_extensions.schema.runner')
-        runner_module.SchemaRunner = Mock()
+        v2 = ModuleType('cloudfile_ext.v2.schema')
+        v2.FreshSchema = Mock()
         if fail:
-            runner_module.SchemaRunner.return_value.apply.side_effect = ValueError('private-database-value')
-        modules = {'pymysql': driver, 'cloudfile_extensions.schema.runner': runner_module}
+            v2.FreshSchema.return_value.install.side_effect = ValueError('private-database-value')
+        modules = {'pymysql': driver,
+                   'cloudfile_ext': ModuleType('cloudfile_ext'),
+                   'cloudfile_ext.v2': ModuleType('cloudfile_ext.v2'),
+                   'cloudfile_ext.v2.schema': v2}
         environment = dict(SEAFILE_MYSQL_DB_HOST='db', SEAFILE_MYSQL_DB_USER='native',
                            SEAFILE_MYSQL_DB_PASSWORD='fixture', SEAFILE_MYSQL_DB_SEAFILE_DB_NAME='native_db')
+        if editing:
+            environment['CF_ENABLE_CHECKOUT'] = 'true'
         with tempfile.TemporaryDirectory() as directory, patch.dict(sys.modules, modules):
+            root = Path(directory) / 'installed'
+            schema_dir = root / 'sql/mysql'
+            schema_dir.mkdir(parents=True)
+            (schema_dir / 'cloudfile-v2.sql').write_text('current V2 three-table DDL')
+            (schema_dir / 'cloudfile-v2-edit.sql').write_text('current V2 optional Guard DDL')
             marker = SCHEMA.prepare_fresh_schema(directory)
             previous_path = list(sys.path)
             if fail:
-                with self.assertRaisesRegex(RuntimeError, '^Fresh CloudFile schema initialization failed;') as error:
-                    SCHEMA.initialize_fresh_schema(marker, '/installed', environment)
+                with self.assertRaisesRegex(RuntimeError, '^CloudFile V2 fresh schema initialization failed;') as error:
+                    SCHEMA.initialize_fresh_schema(marker, root, environment)
                 self.assertNotIn('private-database-value', str(error.exception))
                 self.assertTrue(marker.exists())
             else:
-                self.assertTrue(SCHEMA.initialize_fresh_schema(marker, '/installed', environment))
+                self.assertTrue(SCHEMA.initialize_fresh_schema(marker, root, environment))
                 self.assertFalse(marker.exists())
                 # A completed installation must not issue DDL again on restart.
-                self.assertFalse(SCHEMA.initialize_fresh_schema(marker, '/installed', environment))
+                self.assertFalse(SCHEMA.initialize_fresh_schema(marker, root, environment))
             self.assertEqual(sys.path, previous_path)
-            runner_module.SchemaRunner.return_value.apply.assert_called_once_with()
+            v2.FreshSchema.assert_called_once_with(driver.connect.return_value,
+                'current V2 three-table DDL',
+                edit_ddl='current V2 optional Guard DDL' if editing else None)
+            v2.FreshSchema.return_value.install.assert_called_once_with()
             driver.connect.return_value.close.assert_called_once_with()
             self.assertEqual(driver.connect.call_args.kwargs['database'], 'native_db')
             self.assertTrue(driver.connect.call_args.kwargs['autocommit'])
@@ -66,6 +79,20 @@ def test_success_verifies_and_clears_intent(self):
     def test_failure_stops_startup_and_allows_retry(self):
         self._run(fail=True)
 
+    def test_explicit_checkout_adds_only_current_guard(self):
+        self._run(editing=True)
+
+    def test_bootstrap_does_not_create_legacy_extension_tables(self):
+        # V2 bootstrap retains CE metadata/tag compatibility, not the retired
+        # 001..042/029 Guard or cloudfile.sql integration table creators.
+        code = (Path(__file__).resolve().parents[1] /
+                'scripts/scripts_14.0/bootstrap.py').read_text()
+        body = code.split('def write_cloudfile_config():', 1)[1].split(
+            'def apply_sso_hierarchy_schema_compatibility', 1)[0]
+        for call in ('apply_cloudfile_schema()', 'apply_sso_hierarchy_schema_compatibility()',
+                     'apply_audit_schema()'):
+            self.assertNotIn('    '+call, body)
+
 
 if __name__ == '__main__':
     unittest.main()