Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions .ai/contexts/build-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,60 @@ coverage stays on PR Node 22/Linux only. Lint and e2e still run for every PR
and main push; changelog stays PR-only. Test jobs have 25 minutes; lint,
changelog and e2e have 10 each.

Linux runs the full Node suite with coverage. Every non-Linux runner runs
`node scripts/run-tests.js --platform-only` without c8; its matrix and job
name stay unchanged, including `test (22, windows-2022)`. The coverage step
previously had no explicit environment; both test steps retain that inherited
environment. The PR patch gate reads `coverage/lcov.info` only on Node 22/Linux.

`scripts/platform-tests.js` computes rule C from current files, with no stored
allowlist. It strips physical carriage returns before scanning each test for
strong platform markers: platform checks, Windows paths, conpty/node-pty,
Windows shells, links and filesystem errors, Windows environment variables,
escaped CRLF and case-insensitive paths. Real process calls (`spawn`,
`spawnSync`, `execFile`, `execFileSync`, `.kill`) and filesystem identity or
rename markers (`fs.rename`, sync/promises variants, `realpath`, `birthtime`,
`bigint: true`) also select a test. A tmpdir fixture using realpath therefore
stays selected, including on Windows with an 8.3 TEMP path.
The exported `STRONG_MARKERS` retains the study's POSIX/platform variants.
Relative production requires are followed transitively with a per-test visited
set: reaching a production file containing `process.platform` selects the test.
Test helpers are traversed to reach production dependencies, but their own
platform markers do not opt a caller in. Node_modules are excluded. Literal requires
resolve files, `.js`, `.cjs` and `index.js`; unresolved literals in test fixtures
are ignored. The read/dependency caches are scoped to one selection call.

Limits: this is a source heuristic, not a JavaScript parser or proof that all
Windows-sensitive behavior is covered. It does not follow dynamic imports,
non-relative packages, worker filenames or source-extraction reads. It can
overselect dead code and mocked calls. Windows bug regressions with no visible
marker must retain the explicit opt-in: `task-review-regressions.test.js`
retains the #650 reader/worker regression family, and
`derive-project-path.test.js` exercises the separator expression in
`resolveWorktreePath` using real tmpdir paths, without a `process.platform`
branch. No production-module exclusion list is applied. The selection was 368
of 556 files when this was written; selection above roughly 75% requires
reporting contributors before considering justified exclusions.

To force Windows execution, add a relevant strong marker or the explicit test
source comment `// platform: windows`; extend the exported marker set when a
new category is needed. Physical CRLF endings alone do not opt in. The runner
filters both ordinary and isolated files, prints `platform-only: N of M test
files`, and refuses fewer than 150 selected files or any listed-file read
failure. Empty stages never fall back to Node's implicit test discovery.
Normal mode also refuses a missing `ISOLATED_FILES` inventory entry, naming it
before launching any stage; platform mode may intentionally filter that entry.
`test/platform-test-selection.test.js` pins selection, floor and stage behavior;
`test/build-workflow.test.js` pins coverage and the Windows command.
Its routing guard evaluates every matrix OS plus macOS and an unknown future
OS, requiring exactly one of coverage (`runner.os == 'Linux'`) and platform-only
(`runner.os != 'Linux'`). Matrix JSON, key order and job names stay unchanged.

The fast CLI canary's source harness intercepts the runner's exit expression
before compiling it. The runner retains that expression outside `runSuite()`;
the platform-selection tests read the consumer's interception anchor and check
that it remains present, without executing the locally excluded canary.

The required-check list is documented in [releasing.md](../../docs/releasing.md).
Build is not a required check. JB owns the external settings update that removes
`test (20, windows-2022)` from both ruleset 17240725 (`main-protection`) and the
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -135,8 +135,13 @@ jobs:
echo "SWITCHBOARD_REQUIRE_REAL_BWRAP=1" >> "$GITHUB_ENV"

- name: Run tests with coverage
if: runner.os == 'Linux'
run: npm run coverage

- name: Run platform-specific tests
if: runner.os != 'Linux'
run: node scripts/run-tests.js --platform-only

# Gate only NEW/changed lines (patch coverage), not the whole codebase:
# main.js (Electron IPC/native) and the DOM glue cannot be unit-tested, so
# a global threshold is meaningless. Runs once (Node 22 on ubuntu-latest)
Expand Down
59 changes: 59 additions & 0 deletions scripts/platform-tests.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
'use strict';

const fs = require('node:fs');
const path = require('node:path');

const STRONG_MARKERS = {
platform: /process\.platform|os\.platform\(\)|\bplatform\s*:\s*['"](win32|linux|darwin)|['"]win32['"]|path\.win32|path\.posix|isWindows|IS_WINDOWS/,
winpath: /[A-Za-z]:\\|\b[A-Za-z]:\/|RUNNER~1|~1[\/]|\\\\\?\\|\bUNC\b/,
conpty: /conpty|useConptyDll|node-pty/i,
shell: /cmd\.exe|powershell|pwsh|\.bat['"]|\.cmd['"]|COMSPEC|ComSpec/,
links: /junction|symlink|reparse|EPERM|EBUSY|EACCES/,
env: /USERPROFILE|LOCALAPPDATA|APPDATA|\bTEMP\b|process\.env\.TMP\b|HOMEDRIVE/,
crlf: /\\r\\n|CRLF/,
case: /toLowerCase\(\)[^;\n]{0,40}(path|dir|cwd)|case-?insensitive/i,
optIn: /\/\/\s*platform:\s*windows\b/i,
processes: /\b(?:spawn(?:Sync)?|execFile(?:Sync)?)\s*\(|\.kill\s*\(/,
filesystem: /\bfs\.(?:promises\.)?rename(?:Sync)?\s*\(|\brealpath(?:Sync)?\b|\bbirthtime(?:Ms|Ns)?\b|\bbigint\s*:\s*true/,
};
const MIN_PLATFORM_TESTS = 150;

function selectPlatformTests(files, { root = path.resolve(__dirname, '..'), minimum = MIN_PLATFORM_TESTS } = {}) {
const sources = new Map();
const read = file => {
if (!sources.has(file)) sources.set(file, fs.readFileSync(file, 'utf8').replace(/\r/g, ''));
return sources.get(file);
};
const testDir = path.join(root, 'test') + path.sep;
const dependencies = new Map();
const relativeDependencies = file => {
if (!dependencies.has(file)) {
const resolved = [];
for (const match of read(file).matchAll(/require\(\s*['"](\.{1,2}\/[^'"]+)['"]\s*\)/g)) {
const base = path.resolve(path.dirname(file), match[1]);
const dependency = [base, base + '.js', base + '.cjs', path.join(base, 'index.js')]
.find(candidate => fs.existsSync(candidate) && fs.statSync(candidate).isFile());
if (!dependency || dependency.includes(path.sep + 'node_modules' + path.sep)) continue;
resolved.push(dependency);
}
dependencies.set(file, resolved);
}
return dependencies.get(file);
};
const reachesPlatform = (file, visited) => {
if (visited.has(file)) return false;
visited.add(file);
return relativeDependencies(file).some(dependency =>
(!dependency.startsWith(testDir) && /process\.platform/.test(read(dependency))) || reachesPlatform(dependency, visited));
};
const selected = files.filter(file => {
const absolute = path.resolve(root, file);
return Object.values(STRONG_MARKERS).some(marker => marker.test(read(absolute))) || reachesPlatform(absolute, new Set());
});
if (selected.length < minimum) {
throw new Error(`platform-only: ${selected.length} of ${files.length} test files; below the minimum of ${minimum}`);
}
return selected;
}

module.exports = { STRONG_MARKERS, MIN_PLATFORM_TESTS, selectPlatformTests };
41 changes: 32 additions & 9 deletions scripts/run-tests.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ const { spawnSync } = require('child_process');
const fs = require('fs');
const os = require('os');
const path = require('path');
const { selectPlatformTests } = require('./platform-tests');

const TEST_DIR = path.join(__dirname, '..', 'test');
const ISOLATED_FILES = ['trigger-watcher.test.js', 'trigger-delivery.test.js', 'trigger-continuation-delivery.test.js'];
Expand All @@ -36,27 +37,49 @@ function stageOneConcurrency() {
return Math.max(1, Math.min(DEFAULT_CONCURRENCY, available));
}

const mainFiles = fs.readdirSync(TEST_DIR)
.filter((name) => name.endsWith('.js') && !ISOLATED_FILES.includes(name))
.map((name) => path.join('test', name));

function run(args) {
const result = spawnSync(process.execPath, args, { stdio: 'inherit' });
if (result.error) throw result.error;
return result.status === null ? 1 : result.status;
}

if (require.main === module) {
const mainStatus = run(['--test', `--test-concurrency=${stageOneConcurrency()}`, ...mainFiles]);
function runSuite() {
let files = fs.readdirSync(TEST_DIR)
.filter(name => name.endsWith('.js'))
.map(name => path.join('test', name));
const platformOnly = process.argv.includes('--platform-only');
const missing = ISOLATED_FILES.filter(name => !files.includes(path.join('test', name)));
if (!platformOnly && missing.length) {
console.error(`run-tests: missing isolated test files: ${missing.join(', ')}`);
return { mainStatus: 1, isolatedStatus: 0 };
}
if (platformOnly) {
try {
const selected = selectPlatformTests(files);
console.log(`platform-only: ${selected.length} of ${files.length} test files`);
files = selected;
} catch (error) {
console.error(`run-tests: ${error.message}`);
return { mainStatus: 1, isolatedStatus: 0 };
}
}
const mainFiles = files.filter(file => !ISOLATED_FILES.includes(path.basename(file)));
const isolatedFiles = ISOLATED_FILES.map(name => path.join('test', name)).filter(file => files.includes(file));
const mainStatus = mainFiles.length ? run(['--test', `--test-concurrency=${stageOneConcurrency()}`, ...mainFiles]) : 0;

// No --test-timeout: with an explicit file operand node applies it to the
// file-level entry too, and this file legitimately runs for minutes on CI.
const isolatedStatus = run([
const isolatedStatus = isolatedFiles.length ? run([
'--test',
'--test-concurrency=1',
...ISOLATED_FILES.map((name) => path.join('test', name)),
]);
...isolatedFiles,
]) : 0;

return { mainStatus, isolatedStatus };
}

if (require.main === module) {
const { mainStatus, isolatedStatus } = runSuite();
process.exit(mainStatus !== 0 ? mainStatus : isolatedStatus);
}

Expand Down
28 changes: 27 additions & 1 deletion test/build-workflow.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,32 @@ test('Test keeps its triggers, real-bwrap, patch coverage, lint and e2e conditio
assert.match(bwrap.run, /SWITCHBOARD_REQUIRE_REAL_BWRAP=1/);
assert.equal(steps.find(step => step.name?.startsWith('Patch coverage gate')).if,
"matrix.node-version == 22 && matrix.os == 'ubuntu-latest' && github.event_name == 'pull_request'");
assert.equal(steps.find(step => step.run === 'npm run coverage').if, undefined);
assert.equal(steps.find(step => step.run === 'npm run coverage').if, "runner.os == 'Linux'");
assert.equal(testWorkflow.jobs.e2e.steps.find(step => step.uses?.startsWith('actions/upload-artifact@')).if, 'failure()');
});

test('Windows runs platform-only without coverage and Linux retains coverage', () => {
const steps = testWorkflow.jobs.test.steps;
const windows = steps.find(step => step.run === 'node scripts/run-tests.js --platform-only');
assert.ok(windows, 'Windows must have its own platform-only test step');
assert.equal(windows.run, 'node scripts/run-tests.js --platform-only');
const linux = steps.find(step => step.name === 'Run tests with coverage');
assert.equal(linux.if, "runner.os == 'Linux'");
assert.equal(linux.run, 'npm run coverage');
assert.equal(testWorkflow.jobs.test.name, undefined);
});

test('every matrix OS and a future OS run exactly one test step', () => {
const steps = testWorkflow.jobs.test.steps.filter(step =>
step.run === 'npm run coverage' || step.run === 'node scripts/run-tests.js --platform-only');
assert.equal(steps.length, 2);
const runnerOs = os => os.startsWith('ubuntu-') ? 'Linux' : os.startsWith('windows-') ? 'Windows' : 'macOS';
for (const event of ['pull_request', 'push', 'workflow_dispatch']) {
const operatingSystems = evaluate(testWorkflow.jobs.test.strategy.matrix.include, event).map(cell => runnerOs(cell.os));
for (const os of [...operatingSystems, 'macOS', 'FutureOS']) {
const matched = steps.filter(step => vm.runInNewContext(step.if, { runner: { os } }, { timeout: 1000 }));
assert.equal(matched.length, 1, `${event}: ${os} must run exactly one test step`);
assert.equal(matched[0].run, os === 'Linux' ? 'npm run coverage' : 'node scripts/run-tests.js --platform-only');
}
}
});
1 change: 1 addition & 0 deletions test/derive-project-path.test.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// platform: windows
const test = require('node:test');
const assert = require('node:assert/strict');
const fs = require('fs');
Expand Down
Loading
Loading