Repository navigation
Allow benchmark model ids past the secret scan - #154
Merged
Merged
Conversation
The secret scan reads every branch, and the generic API key rule flags any "key" field. A benchmark models file on an open branch names its embedding setups under "key", so CI failed on main and every other branch. The allowlist covers only JSON under benchmarks/ where the key is lowercase letters, digits, dots and dashes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
CI fails on
mainand on every other branch because of a false positive in the secret scan. The scan checks the history of all branches (fetch-depth: 0), and gitleaks' defaultgeneric-api-keyrule flags any"key"field. A benchmarks file on an open branch (#153,benchmarks/kudos-pilot/models.json) names its embedding setups under"key", for exampleqwen3-8b-1024. Those are names, not credentials.This adds one allowlist entry to
.gitleaks.toml. It applies only when both conditions hold: the file is JSON underbenchmarks/, and the line is a"key"whose value is only lowercase letters, digits, dots and dashes. Secrets in other paths or fields, or with other characters, are still caught.How it was tested
Ran
ghcr.io/gitleaks/gitleaks:v8.28.0 git /repo --redact --exit-code 1on a fresh clone with all branches, the same command CI runs. Before the change it found 1 leak, the benchmark model id. After the change it found none (107 commits scanned).I checked the flagged value's shape without printing it: 13 characters, only lowercase letters, digits and dashes, like the other model ids in the file.
npm run checkpasses (not run: only.gitleaks.tomlchanges, and the check does not read it)Checklist
packages/providers-*docs/architecture/modulesare updated if a contract, boundary or module changed