Skip to content

feat(tooling): run the unit tests that mention a changed file in ci:prepush - #1209

Merged
qnbs merged 11 commits into
mainfrom
feat/prepush-related-tests
Nov 10, 2026
Merged

qnbs merged 11 commits into
mainfrom
feat/prepush-related-tests

Conversation

@qnbs

@qnbs qnbs commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

User description

Summary

ci:prepush ran policy scripts only and never executed a unit test. A push that changed .github/workflows/tauri-build.yml therefore passed the local gate while tests/unit/workflowPolicy.test.ts still held the old expectations, and the failure only showed up in the hosted Quality Gate on both Node versions (PR #1208).

This adds a "Related unit tests" step to the local admission gate.

What changes

  • scripts/ci-prepush-related-tests.mjs selects the unit test files that were changed plus every unit test that mentions a changed source, workflow, script or config file. It searches with git grep for the path, the extension-less import path (TypeScript) and the basename (workflows, scripts, config), so a test that reads a workflow from disk is found too. A pure import graph such as vitest related would not find those.
  • The selected files run as an explicit list (vitest run <files>), never as a suite. End-to-end specs are never selected.
  • At most 8 files run locally. A larger set is reported as DEFERRED_TO_REQUIRED_CI with the count, and missing change evidence is reported as deferred as well. Nothing is skipped silently.
  • The step is routed by the existing admission registry (relatedTests) and runs before the TypeScript check.
  • Ambiguous change evidence (for example a new branch) is reported as DEFERRED_TO_REQUIRED_CI instead of presenting a meaningless selection.
  • The pre-push hook started the whole local gate with the default 120 s child bound, so any run that took longer was sent SIGTERM mid-step and failed with a bare FAIL. That is why a full-range typecheck of a new branch failed after about two minutes. The gate now has its own finite bound (25 minutes), the typecheck and the related-test run have theirs (10 and 8 minutes), and a test requires the gate bound to exceed the sum of the two heavy steps.
  • A failing typecheck or related-test run now prints how it ended (exit status, signal, timed out or interrupted), because the SIGTERM was undiagnosable before.
  • The related tests run without any GIT_* variable. Run from the pre-push hook, signing.test.ts failed because git hands its variables to its children, while the same tests pass from a normal shell; this was reproduced with GIT_DIR set.
  • Test selection mirrors the Vitest include list exactly (.test and .spec, .ts and .tsx, minus tests/e2e/), accepts JSON sources such as src-tauri/tauri.conf.json, and drops bare generic names such as package.json.

Tests

  • tests/unit/tooling/ciPrepushRelatedTests.test.ts covers needle derivation, selection, ordering, deleted tests, excluded end-to-end specs, the limit boundary (8 runs, 9 defers) and registry routing.
  • One regression test pins the incident: changing tauri-build.yml selects workflowPolicy.test.ts and releaseSecurityGate.test.ts.
  • pnpm run ci:prepush ran the new step on this change: 3 related test files, 60 tests passed. The pre-push hook then caught a real README test-count drift introduced by the added test, which is fixed in the same PR.
  • Tests pin that all three bounds are finite and above the 120 s default, that the gate bound exceeds the sum of its heavy steps, and that every GIT_* variable is removed from the test environment.

Scope

Local tooling only. The required hosted CI, its checks and the merge rules are unchanged. No release, tag or deployment is involved.

Summary by Sourcery

Run relevant unit tests in the pre-push gate and make deferred checks and local failures explicit.

New Features:

  • Run up to eight unit tests related to changed source, workflow, script, and configuration files during the pre-push admission gate.
  • Report oversized or unavailable test selections as deferred to required CI instead of silently skipping them.

Bug Fixes:

  • Prevent pre-push checks from inheriting GIT_* variables that can break tests using temporary Git repositories.
  • Avoid premature termination of longer local checks and provide actionable failure details when checks exit, time out, or are interrupted.

Enhancements:

  • Route related-test admission through the existing check registry and align selection with the Vitest unit-test scope.
  • Update user-facing CI guidance, changelog entries, and test-count metrics for the expanded local gate.

Documentation:

  • Document related unit-test selection, deferral behavior, exclusions, environment isolation, and local check timeouts.

Tests:

  • Add coverage for test classification, related-file matching, workflow and configuration regressions, selection limits, deferrals, environment cleanup, timeout bounds, and registry routing.

Summary by cubic

Runs the unit tests that mention a changed file in ci:prepush, so a changed workflow or script no longer reaches hosted CI with stale test expectations in the tests that read it.

What changes

  • Selects changed test files plus every unit test that references a changed source, workflow, script, or config file, via git grep on the path, extension-less import, and basename. Selection mirrors the Vitest include list, so chosen files are always runnable.
  • Up to 8 files run as an explicit vitest run list; larger sets, missing or ambiguous change evidence, and excluded files are reported as deferred to required CI, never skipped silently.
  • Changes that select nothing by design are documented: root package.json and tsconfig*.json, bare index/types/constants/utils names, and JSON under locales/, public/, community-templates/, docs/, reports/, and tests/.
  • The related-test run strips GIT_* variables the pre-push hook passes on, which broke tests that run git in disposable repositories.
  • The typecheck, related-test run, and whole gate get bounded timeouts above the 120 s default child bound; a dead child now reports its exit status and signal, so failures are diagnosable.

Written for commit f63ba3b. Summary will update on new commits.

View guided diff Turn on auto-fix


CodeAnt-AI Description

Run relevant unit tests before a push and clearly report checks deferred to CI

What Changed

  • The pre-push gate runs changed unit tests and tests that mention changed source, workflow, script, or configuration files, up to 8 files.
  • When too many tests match or change evidence is unavailable, the gate reports that required CI must run them instead of silently skipping them.
  • Longer-running local checks have bounded timeouts and report whether they failed, timed out, or were interrupted.

Impact

✅ Stale workflow and source expectations caught before push
✅ Deferred unit tests clearly identified for required CI
✅ Clearer reports when local checks time out or fail

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

Summary by CodeRabbit

  • New Features
    • Pre-push checks now run up to eight unit tests related to changed files. If more than eight tests match—or the changes can’t be determined reliably—the tests are deferred to CI.
    • Local checks now have defined time limits, with clearer reporting when a check fails or times out.
  • Documentation
    • Updated CI guidance to explain related-test selection and deferral behavior.
    • Updated reported test totals to 8,735+ tests across 657 files.

qnbs added 2 commits November 10, 2026 01:24
…repush

ci:prepush ran policy scripts only, so a changed workflow reached CI with stale expectations in the unit tests that read it from disk. Select the changed test files and every unit test that mentions a changed source, workflow, script or config file, and run that explicit list. More than eight files, or missing change evidence, is reported as deferred to required CI instead of being skipped silently.
…efault child limit

A full-range typecheck of a new branch took 123 s and was killed by the 120 s default child bound, so the pre-push gate failed without a diagnostic. Give the single-checker typecheck and the related-test run their own finite timeouts, and report ambiguous change evidence as deferred instead of presenting a meaningless test selection.
@vercel

vercel Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
worldscript-studio Ignored Ignored Preview Nov 10, 2026 5:27am UTC

@sourcery-ai

sourcery-ai Bot commented Nov 10, 2026

Copy link
Copy Markdown

Sorry @qnbs, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 9 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@codeant-ai

codeant-ai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

CodeAnt  CodeAnt Review Status

✅  Review completed

Status Commit Started (UTC) Finished (UTC)
✅  #2f104 Incremental review completed 6ce89c3 Nov 10, 2026 · 05:01 05:01
✅  #32e8d Incremental review completed d04769a Nov 10, 2026 · 02:51 02:51
✅  #b4c8b Reviewed your PR dd32b35 Nov 10, 2026 · 00:35 00:41

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@codeant-ai

codeant-ai Bot commented Nov 10, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@coderabbitai

coderabbitai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Explain Change →

Warning

Review limit reached

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Next included review available in 34 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 74 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 0c9e97a and f63ba3b.

Files selected for processing: 9

  • CHANGELOG.md
  • README.md
  • docs/CI.md
  • scripts/ci-prepush-check-registry.mjs
  • scripts/ci-prepush-lowend.mjs
  • scripts/ci-prepush-related-tests.d.mts
  • scripts/ci-prepush-related-tests.mjs
  • scripts/hooks/pre-push.mjs
  • tests/unit/tooling/ciPrepushRelatedTests.test.ts
Walkthrough

The pre-push checks now select unit tests related to changed files and run selections of up to eight tests. Larger or unresolved selections are deferred to required CI. The change also adds local-check timeouts and updates related documentation and README test totals.

Changes: Related pre-push tests

Layer / File(s) Summary
Related-test selection
scripts/ci-prepush-related-tests.mjs, scripts/ci-prepush-related-tests.d.mts, tests/unit/tooling/ciPrepushRelatedTests.test.ts
Adds helpers to identify unit tests, derive search needles, and find matching tests. The selector returns NONE, RUN, or DEFERRED based on the selection size. Tests cover path classification, selection results, and the limit.
Pre-push admission and execution
scripts/ci-prepush-check-registry.mjs, scripts/ci-prepush-lowend.mjs, scripts/hooks/pre-push.mjs, tests/unit/tooling/ciPrepushRelatedTests.test.ts
Adds related-test admission to the registry and low-end runner. Eligible selections run under Vitest with a Git-free environment and a timeout. The hook applies the gate timeout. Tests cover routing, environment filtering, and timeout settings.
Behavior and test-count updates
docs/CI.md, CHANGELOG.md, README.md
Documents related-test selection, deferral, and local-check timeouts. Updates the README totals to 8,735+ tests across 657 files.
Merge Risk: Low · previous review 6ce89

The local gate’s documentation overstates which tests run and when they are deferred. Correct those claims so contributors can choose appropriate manual checks; the established issues are bounded and do not block merging.

@sourcery-ai

sourcery-ai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

Data structures

Define the related-test selector’s exported types and options so its callers share a clear contract.

  • scripts/ci-prepush-related-tests.d.mts L1-24

Uses of the API

Give the pre-push hook’s whole local gate a finite timeout using the related-check timeout constants.

  • scripts/hooks/pre-push.mjs L5, L25-30

Implementation

Implement test-file classification, search needles, selection limits, Git-environment cleanup, and finite local-check timeouts.

  • scripts/ci-prepush-related-tests.mjs L1-121

Integrate related-test selection into the admission flow, report deferred cases, and make bounded child failures diagnosable.

  • scripts/ci-prepush-lowend.mjs L27-46, L58-88, L202-211 (+1 small edits)

Configuration

Route applicable source, workflow, script, and config changes—and changes to the selector itself—to the related-tests admission check.

  • scripts/ci-prepush-check-registry.mjs L82-93

Documentation

Document related-test selection, exclusions, deferral, timeout, and diagnostic behavior in the CI guide.

  • docs/CI.md L390-415

Update the README’s test totals to reflect the added unit-test file and tests.

  • README.md L23-1032 (4 small edits)

Record the local-admission change and its qualification that some selections are deferred to required CI.

  • CHANGELOG.md L15

Tests

Cover test classification, needle generation, selection and limits, environment sanitization, timeout bounds, and admission routing.

  • tests/unit/tooling/ciPrepushRelatedTests.test.ts L1-167

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@codeant-ai

codeant-ai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

View results CodeAnt  CodeAnt Quality Gate Results

Commit f63ba3b3 · Scanned Nov 10, 2026 at 05:27 UTC

✅  Overall Status: PASSED

Quality Gate Status Details
New code coverage ⚪  Not applicable Not applicable — no applicable code changes
Secrets ✅  Passed 0 secrets found
Duplicate code ✅  Passed 0.0% duplicated
SAST ✅  Passed No security issues
Bugs ✅  Passed Rating S · No bugs
IaC ✅  Passed No IAC issues

@codeant-ai

codeant-ai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

View results CodeAnt  CodeAnt Coverage Report

Commit f63ba3b3 · Scanned Nov 10, 2026 at 05:27 UTC

⚪  Overall Status: Not applicable

Coverage Status Details
New Code Coverage ⚪  Not applicable Not applicable — no applicable code changes

New-code coverage threshold not configured; coverage is reported but not enforced.

@codeant-ai codeant-ai Bot added the size:L This PR changes 100-499 lines, ignoring generated files label Nov 10, 2026
codescene-access[bot]

This comment was marked as outdated.

Comment thread scripts/ci-prepush-lowend.mjs Resolved
Comment thread scripts/ci-prepush-related-tests.mjs Resolved
Comment thread scripts/ci-prepush-related-tests.mjs Resolved
@codeant-ai

codeant-ai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

CodeAnt  CodeAnt PR Risk

🟢  Low Risk

  • The PR appears safe to merge; the related-test selector handles resolved full runs, Vitest .spec files, and governed JSON paths.
  • Regression tests cover these selection cases, including workflow and Tauri configuration files.

Assessed commit: 6ce89c3ddec9

@codecov

codecov Bot commented Nov 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

qnbs added 3 commits November 10, 2026 03:48
… selecting tests

Mirror the Vitest include list so a selected file is always runnable, including spec files and excluding mjs tests that Vitest does not run. Accept JSON sources outside a few roots, such as the Tauri configuration read by the CSP tests, while dropping bare generic names that would match almost every test. A full run with resolved change evidence now selects and runs the related tests instead of deferring them.
sourcery-ai[bot]
sourcery-ai Bot previously approved these changes Nov 10, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sourcery assessment

Approved.

codescene-access[bot]

This comment was marked as outdated.

qnbs added 4 commits November 10, 2026 05:47
…ook provides

Run from the pre-push hook, signing tests that start git in disposable repositories failed because the hook passes GIT_* variables to its children, while the same tests pass from a normal shell. Drop every GIT_* variable from the environment of the related-test run.
A failing child of the local admission gate printed only FAIL when it died without output, which left a pre-push failure undiagnosable. Print the exit status, the signal, and whether the child timed out or was interrupted.
…he 120 s default

The pre-push hook started the complete local gate with the default 120 s child bound, so any run that took longer was sent SIGTERM mid-step and failed without a diagnostic. This also explains the earlier typecheck failures. Give the gate one finite bound that exceeds the sum of its two heavy steps.
@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

[check-pr-size] PR size is over the hard tier (normal profile): 9 files, 435 meaningful lines, 11 commits — limit ≤20 files / ≤1200 lines / ≤10 commits. Consider splitting into smaller, independently reviewable PRs.

codescene-access[bot]

This comment was marked as outdated.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

Minor · Correct the fail-closed claim. · CI.md:414-415

docs/CI.md:414-415
Minor · Functional Correctness · Quick win

Correct the fail-closed claim.

When change evidence is unresolved, the new related-test step reports DEFERRED_TO_REQUIRED_CI. It does not run every check. Update this paragraph to describe that exception. (raw.githubusercontent.com)

Additional comments: 1

tests/unit/tooling/ciPrepushRelatedTests.test.ts: 1

104-112: Trivial · Maintainability & Code Quality · Low value

Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

These two tests depend on live repository content.

selectRelatedTests(...) without options runs real git grep against the working tree. The tests need a git checkout with tracked test files. They fail in an exported source archive or a shallow environment without git. They also fail if the referenced test files stop mentioning those config files. This is acceptable for a regression guard. Note the coupling if these tests run outside a normal git checkout.
[optional]

Also applies to: 150-155

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 0c9e97a and 6ce89c3.

Files selected for processing: 9

  • CHANGELOG.md
  • README.md
  • docs/CI.md
  • scripts/ci-prepush-check-registry.mjs
  • scripts/ci-prepush-lowend.mjs
  • scripts/ci-prepush-related-tests.d.mts
  • scripts/ci-prepush-related-tests.mjs
  • scripts/hooks/pre-push.mjs
  • tests/unit/tooling/ciPrepushRelatedTests.test.ts

Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Comment thread CHANGELOG.md Resolved
Comment thread docs/CI.md Resolved
qnbs added 2 commits November 10, 2026 06:26
…xclusions

The changelog promised that stale expectations no longer reach CI, but large sets, ambiguous evidence and excluded files are deferred. State what runs locally and which changes select nothing by design.
…ut sentence

The previous edit cut the sentence about the bounded timeouts in half. Rewrite the whole bullet: selection rules, exclusions, the GIT variables, the three finite bounds and the failure diagnostics.

@codescene-access codescene-access Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gates Passed
3 Quality Gates Passed

See analysis details in CodeScene

Quality Gate Profile: The Bare Minimum
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.

@qnbs
qnbs merged commit ba32e5e into main Nov 10, 2026
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant