Skip to content

ci: add a separate per-plugin test workflow and plugin test rules - #84

Merged
MyPrototypeWhat merged 4 commits into
mainfrom
ci/plugin-tests
Oct 27, 2026
Merged

MyPrototypeWhat merged 4 commits into
mainfrom
ci/plugin-tests

Conversation

@MyPrototypeWhat

@MyPrototypeWhat MyPrototypeWhat commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Adds a workflow that runs a plugin's own tests only when a pull request changes that plugin and the plugin declares a test script. It also documents how contributors write those tests.

What changes

.github/workflows/plugin-tests.yml (new)

  • Triggers on pull_request only, limited to changes under plugins/**. Runs with permissions: contents: read, and checkout uses persist-credentials: false. There are no secrets, so it is safe on fork PRs.
  • detect (ubuntu): checks out the PR merge commit with fetch-depth: 2 and diffs it against its first parent. From that diff it lists the changed plugins/<author>/<plugin-id> directories and keeps those whose package.json has a non-empty scripts.test.
  • test (macOS, one job per plugin, fail-fast: false): sets up Node.js 22, installs dependencies (npm ci when the plugin has a lockfile), then runs npm test in the plugin directory.
  • Untouched plugins never run. If no changed plugin has a test script, the test job is skipped.

CONTRIBUTING.md / CONTRIBUTING.zh-CN.md: a "Plugin tests" section.

  • Tests are optional, live in the plugin (e.g. tests/), and run through its npm test.
  • No external network hosts; a server the test starts on 127.0.0.1 is fine.
  • No dependency on the MiniMax Code host app.
  • Dependencies are installed only inside the plugin, with a committed lockfile. Delete that node_modules before npm run check, which rejects any node_modules inside a package.
  • The root test/ stays reserved for the repository's rule checks.
  • The runner provides macOS, Node.js 22, and Git only. A test that needs another tool, such as FFmpeg or a browser, must skip itself when the tool is missing.

AGENTS.md: the same rules as an optional step before npm run check, since agents read AGENTS.md rather than CONTRIBUTING.

The rules CI (npm run check), the root test/, the scripts, and the root package.json are unchanged.

Review changes (on top of the first commit)

  • Diff the merge commit with fetch-depth: 2 instead of cloning the full history (108 MiB) for base...head.
  • The docs now use the tests/ convention, allow loopback servers, add the node_modules note, and gain the AGENTS.md step.
  • Run the test job on macos-latest, and state the test environment (see the trial below).

Verification

  • Detection script (extracted verbatim), run against the current tree with synthetic change lists. The results were correct for:
    • one plugin with tests;
    • a plugin with tests/ but no scripts.test;
    • three plugins, deduplicated and sorted;
    • a deleted plugin, files above plugin level, paths with spaces, and no plugin changes.
  • Merge-commit diff, on a simulated PR merge where main had also moved: only the PR's own files are listed.
  • End-to-end trial, draft PR [trial] Exercise the Plugin tests workflow from #84 (do not merge) #88 (now closed, branch deleted). It touched the four plugins that have a test script, plus cq-viewer, which has none. detect selected exactly the four:
Plugin ubuntu-latest (reference only: MiniMax Code has no Linux build) macos-latest
liuchen19960902/timezone-guard ✅ ✅ 14 s
stevenjj33/workflow-console ✅ ✅ 15 s
zch0227/disk-insight 2 scanner tests differ (directory sizes, hard links); not a defect, since the plugin never runs on Linux ✅ 23 s
klarkxy/cutline ❌ 20 of 188 ❌ 21 of 188

The workflow on this branch differs from the one that ran on #88 only by a comment line. Windows, MiniMax Code's other platform, was not run; the squash commit's note that disk-insight's tests assume Windows behaviour too is unverified.

Known red: cutline

cutline's failures come from the test environment, not from this workflow:

  • 14 tests spawn ffmpeg to make fixture clips (tests/helpers.mjs), and the runner has none. The other failures follow from those missing clips.
  • On macOS, import-url-security.test.mjs also listens on 127.0.0.2. macOS configures only 127.0.0.1 on loopback, so the listen fails with EADDRNOTAVAIL.

The browser tests in i18n-browser.test.mjs already skip when no Chromium is found; the same pattern fixes the FFmpeg tests. Until cutline does that, a PR that changes cutline shows this check red. main has no required checks, so this does not block merging.

MyPrototypeWhat and others added 3 commits October 26, 2026 11:23
Add .github/workflows/plugin-tests.yml, which runs on pull_request for
plugins/** changes, detects changed plugins/<author>/<plugin> dirs that
define scripts.test, and runs npm test in each via a matrix (Node 22,
read-only permissions). Document plugin test conventions in
CONTRIBUTING.md and CONTRIBUTING.zh-CN.md.
… and tell agents about plugin tests

Review follow-up on the per-plugin test workflow:

- detect: check out the PR merge commit with fetch-depth 2 and diff it against its first parent,
  instead of cloning the full history (108 MiB and growing) to diff base...head.
- CONTRIBUTING (en/zh): use the repository's tests/ convention in the example, allow servers the
  test starts on 127.0.0.1 (only external hosts are off-limits), and say to delete the plugin's
  node_modules before npm run check, which rejects any node_modules inside a package.
- AGENTS.md: add the plugin-test step, since agents read AGENTS.md rather than CONTRIBUTING.

Assisted-by: Claude Opus 5.5 (OpenCode)
…nment

A trial run (#88) on ubuntu-latest failed disk-insight's scanner tests, which assume macOS and
Windows filesystem behaviour; on macos-latest they pass. MiniMax Code runs only on macOS and
Windows, so the test job now runs on macOS (detection stays on Linux).

The runner provides Node.js 22 and Git only. CONTRIBUTING (en/zh) and AGENTS.md now say so, and
that a test needing another tool, such as FFmpeg or a browser, must skip itself when the tool is
missing. cutline's FFmpeg-based tests fail on both runners until they do.

Assisted-by: Claude Opus 5.5 (OpenCode)
@MyPrototypeWhat
MyPrototypeWhat merged commit 475a658 into main Oct 27, 2026
1 check passed
@MyPrototypeWhat

Copy link
Copy Markdown
Collaborator Author

@klarkxy, a heads-up about cutline. This workflow now runs npm test for any pull request that changes a plugin with a test script. It runs on macOS with Node.js 22 and Git only. On the trial run (#88), cutline failed 21 of 188 tests:

  • FFmpeg is not installed. tests/helpers.mjs spawns ffmpeg to make fixture clips, which gives 14 spawn ffmpeg ENOENT failures; the remaining failures follow from the missing clips. i18n-browser.test.mjs already skips when no Chromium is found, and the same { skip } check around FFmpeg would turn these into skips.
  • 127.0.0.2 does not exist on macOS. import-url-security.test.mjs starts a listener on 127.0.0.2. macOS configures only 127.0.0.1 on loopback, so the listen fails with EADDRNOTAVAIL.

Until then, a PR that changes cutline shows the Plugin tests check red. It does not block merging. The test environment is described under "Plugin tests" in CONTRIBUTING.md.

@klarkxy

klarkxy commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

@klarkxy, a heads-up about cutline. This workflow now runs npm test for any pull request that changes a plugin with a test script. It runs on macOS with Node.js 22 and Git only. On the trial run (#88), cutline failed 21 of 188 tests:

  • FFmpeg is not installed. tests/helpers.mjs spawns ffmpeg to make fixture clips, which gives 14 spawn ffmpeg ENOENT failures; the remaining failures follow from the missing clips. i18n-browser.test.mjs already skips when no Chromium is found, and the same { skip } check around FFmpeg would turn these into skips.
  • 127.0.0.2 does not exist on macOS. import-url-security.test.mjs starts a listener on 127.0.0.2. macOS configures only 127.0.0.1 on loopback, so the listen fails with EADDRNOTAVAIL.

Until then, a PR that changes cutline shows the Plugin tests check red. It does not block merging. The test environment is described under "Plugin tests" in CONTRIBUTING.md.

I know, will do it soon.

@MyPrototypeWhat

Copy link
Copy Markdown
Collaborator Author

Update: this workflow has been reverted in #89. Plugin authors keep their own tests passing, and the repository CI is back to the rule checks only.

@klarkxy, this means cutline PRs will not show a red Plugin tests check after all. One finding from the trial is still worth a look for your own runs: on macOS, tests/import-url-security.test.mjs listens on 127.0.0.2. macOS configures only 127.0.0.1 on loopback, so that test fails with EADDRNOTAVAIL on any Mac, MiniMax Code's main platform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants