Skip to content

ci(tooling): publish agent skills to a skills branch by an approved deploy - #102

Open
Theauxm wants to merge 7 commits into
mainfrom
feat/ms16-agent-skills
Open

Theauxm wants to merge 7 commits into
mainfrom
feat/ms16-agent-skills

Conversation

@Theauxm

@Theauxm Theauxm commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Trax's agent skills are published from a release tag to a skills branch, only when a maintainer approves

Background

An agent skill is a folder with a SKILL.md: a name, a one-line description of when to use
it, and instructions a coding agent loads when its task matches the description. Claude Code reads
them natively, as a plugin installed from a marketplace (a repository with a
.claude-plugin/marketplace.json). Most other agents (Cursor, Codex and so on) get them through the
skills CLI, npx skills add <source>, which copies the folders into the agent's own directory.

People building on Trax use both. The skills themselves come in the next PR; this one decides where
they live and how they are published. ADR 0056 records it.

Problem

⬛ The obvious install would serve unreleased skills
The skills CLI installs from a repository's default branch. Skills written on main would reach an
agent ahead of the packages on nuget.org, so it would be taught APIs the app cannot call yet.

⬛ A folder outside the CLI's usual places is not enough
The CLI looks in skills/, .claude/skills/ and the root. When it finds nothing visible there, it
falls back to a recursive search. Measured with skills 1.7.4: with only the contributor skill in
.claude/skills/ marked internal, it found Trax.Docs/skills/ and offered every skill in it.

Caution

🟥 Whoever can change the published skills can instruct every consumer's agent.
A skill is instructions an agent follows in someone else's repository, so what is published has to
change only when a maintainer says so.

⬛ Nothing checks a skill
A skill that teaches an API Trax no longer has, or links a page that moved, reads exactly like a
correct one.

Example

Release 1.70.0 is out. A maintainer then dispatches Skills with v1.70.0, as they dispatch
Website:

Job Holds What happens
stage read-only token checks out v1.70.0; Trax.Skills.Publish stage copies Trax.Docs/skills/*, strips internal: true, writes trax-version: "1.70.0" into each SKILL.md, writes plugin.json at 1.70.0, a README and the licence; uploads the tree with its digests
prepare contents: write checks the digests, resets skills-next to skills, commits the tree on it through createCommitOnBranch (GitHub signs it), and lists every file added, changed or removed, with a compare link, in the run summary
publish the deploy key, after approval waits on the skills-deploy environment; once approved, fast-forwards skills to exactly the commit prepare made and tags it skills-v1.70.0

The release workflow is unchanged and never touches skills.

A source skill and the same skill on the branch:

# main: Trax.Docs/skills/trax/SKILL.md
name: trax
description: Use before writing or changing code in a .NET app that references Trax packages ...
metadata:
  internal: true
# skills branch: skills/trax/SKILL.md
name: trax
description: Use before writing or changing code in a .NET app that references Trax packages ...
metadata:
  trax-version: "1.70.0"

Then, in an app:

/plugin marketplace add TraxSharp/Trax                                   # Claude Code
/plugin install trax@traxsharp
npx skills add https://github.com/TraxSharp/Trax/tree/skills             # any other agent
npx skills add https://github.com/TraxSharp/Trax/tree/skills-v1.68.0     # stay on a release

npx skills add TraxSharp/Trax on its own finds nothing, --full-depth included: every source
skill is marked internal, and the CLI does not follow a marketplace to another ref (both measured).
An install from a tree URL records the ref in skills-lock.json, and npx skills check keeps it
there.

How this solves it

⬛ Skills are written in Trax.Docs/skills/, held to the docs' rules, and kept off the site
sync-docs.sh and both RepoRoot.IsPublished skip skills/. A skill's C# fences carry the docs'
compile / from= / fragment markers; CompiledSnippetsTests already scanned every .md, and
the marker and excerpt checks now read skills as well as pages. SdkReferenceBlockTests exempts
them, since a skill links its pages inline.

⬛ Every source skill is marked internal
metadata.internal: true hides a skill from the CLI unless INSTALL_INTERNAL_SKILLS=1. Staging
removes it. recording-decisions is marked too, so consumers are never offered a contributor skill.

⬛ One set of rules, run by the tests and by the stage
SkillRules checks the name (the folder's, lowercase, at most 64), the description (present, at most
1024, no < >), the marker, no stamped version, only known keys, markdown files only (a published
skill never ships a script), and that relative links stay inside the skill.
SkillsAreWellFormedTests runs it over the real folder, resolves every
https://traxsharp.net/docs/... link and anchor against the pages this checkout publishes, and holds
.claude-plugin/marketplace.json on main to Stager.MarketplaceManifest(). Stager.Stage runs the
same rules and writes nothing when one fails.

Caution

🟥 Only an approved run moves the skills branch.
Its ruleset refuses updates, deletion and force pushes to everything but deploy keys, and the tag
ruleset does the same for skills-v*, creation included. The key is a secret of the skills-deploy
environment, so it is released only when a maintainer approves the run. A deploy key cannot push a
commit GitHub would verify, and the Commits ruleset requires one, so the work is split: the
workflow token writes the signed commit to skills-next, which nobody installs from, and the key
only moves skills to it, as a fast-forward that is refused if skills moved since. A tag is
never moved: one that already names another commit fails the job.

flowchart LR
  Tag["v1.70.0 tag"] --> Stage["stage: Trax.Skills.Publish"]
  Stage -->|"tree + digests"| Prep["prepare: commit to skills-next"]
  Prep --> Next["skills-next: signed preview"]
  Next --> Approve{"🟥 skills-deploy approval"}
  Approve -->|"deploy key, fast-forward"| Branch["skills + skills-v1.70.0"]
  Market["main: .claude-plugin/marketplace.json"] -->|"plugin source ref: skills"| Branch
  Branch --> CC["/plugin install trax@traxsharp"]
  Branch --> CLI["npx skills add .../tree/skills"]
  class Tag,Stage,Prep,Next,Approve,Branch,Market,CC,CLI tooling
  classDef tooling fill:#334155,stroke:#0f172a,color:#fff
Loading

⚠️ Repository settings. The skills branch exists already, created by hand with a signed
commit. Before the first run: an environment skills-deploy with you as required reviewer and a
SKILLS_DEPLOY_KEY secret (a new deploy key with write access); a ruleset skills on
refs/heads/skills (restrict updates, deletions, force pushes; bypass: deploy keys); and
skills tags on refs/tags/skills-v* (restrict creations, updates, deletions, force pushes;
bypass: deploy keys). A ruleset's deploy-key bypass covers every deploy key, the website's too.

The first version of this PR published to a separate TraxSharp/skills repository through a GitHub
App, from the release itself. Once the internal marker turned out to be needed anyway, main serves
nothing, so the second repository only bought a shorter install URL; and publishing from the release
left nobody approving what changed. ADR 0056 records all three.

Checked: the Trax.Docs suite and ADR guard green; actionlint, shellcheck and zizmor clean on the
workflows and script; the prepare script run against a stubbed gh for a first run (creating
skills-next), a later run (resetting it), an unchanged tree and a missing skills; a staged tree
passing claude plugin validate --strict and listing and installing with the skills CLI; the root
marketplace passing claude plugin validate --strict.

Architectural hinges

  • Trax.Docs/adr/0056-agent-skills-are-written-in-the-docs-and-published-from-a-release.md:
    the decision, the measurements behind the internal marker, and the options rejected (a separate
    repository, publishing from the release, skills on main, plugin-only, copies in every template).
  • ⬛ Trax.Docs/tools/Trax.Skills.Publish/SkillRules.cs (SkillRules.CheckOne): every rule a
    skill must pass, shared by the guard test and the stage.
  • ⬛ Trax.Docs/tools/Trax.Skills.Publish/Stager.cs (Stager.Stage, MarketplaceManifest): what
    a consumer actually installs, and the marketplace that points Claude Code at it.
  • ⬛ .github/workflows/skills.yml: the three jobs, what each holds, and the approval between the
    signed commit and the live branch.
  • ⬛ .github/scripts/publish-skills.sh: the signed commit on skills-next and the change list the
    approver reads.

ADR 0056. Skills live in Trax.Docs/skills/, are marked internal so the
skills CLI never serves them from main, and each release publishes them
to TraxSharp/skills as a Claude Code plugin and marketplace, stamped
with the release version.
Trax.Docs/skills/ holds the skills, kept off the site. Trax.Skills.Publish
checks them (name, description, the internal marker, markdown only,
links inside the skill) and stages the tree a release publishes;
SkillsAreWellFormedTests runs the same rules and resolves every docs
link a skill makes. The contributor skill is marked internal so
npx skills add TraxSharp/Trax does not offer it.
build stages the skills at the release version; a new skills job, after
publish, verifies the staged digests and commits them to TraxSharp/skills
through createCommitOnBranch with a GitHub App token scoped to that
repository, then tags the commit. Both stay off until the SKILLS_APP_ID
repository variable is set.
EveryCSharpFenceIsMarkedTests and SampleExcerptsMatchSourceTests read
published pages only, and skills/ is not published, so a skill's fences
went unchecked apart from compile. Both now read the skills too.
A skill is not a site page and links to the pages it routes to inline.
@vercel

vercel Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
trax-website Ignored Ignored Preview Oct 20, 2026 8:06pm UTC

@Theauxm
Theauxm added this pull request to stack #104 October 20, 2026 19:29
@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…ository

The release's skills job now commits the staged tree to the skills branch
of this repository with its own token and tags it skills-v<version>; no
GitHub App and no second repository. A static marketplace on main points
Claude Code's trax plugin at the branch, and SkillsAreWellFormedTests
holds it to the one the stager expects. ADR 0056 records why the
separate repository was dropped.
@Theauxm Theauxm changed the title ci(tooling): publish agent skills from each release ci(tooling): publish agent skills from each release to a skills branch Oct 20, 2026
The release no longer touches skills. A Skills workflow, dispatched with a
release tag like website.yml, stages the tree, commits it with the
workflow token on top of skills to skills-next (GitHub signs it), then
waits on the skills-deploy environment; once approved it fast-forwards
skills to that commit and tags it skills-v<version> with a deploy key,
the only bypass the skills rulesets allow.
@Theauxm Theauxm changed the title ci(tooling): publish agent skills from each release to a skills branch ci(tooling): publish agent skills to a skills branch by an approved deploy Oct 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant