Improve CLI options

.github/workflows/reusable.yml

@@ -66,9 +66,10 @@ on:
         description: >-
           The skills to grade: a directory, or a glob matching several, one per
           line or comma-separated, relative to the repository root. Each one is
-          graded on a runner of its own.
+          graded on a runner of its own. The default is every directory at the
+          root; a repo that keeps its skills together says so with `skills/*`.
         type: string
-        default: "skills/*"
+        default: "./*"
 
       structural:
         description: >-
@@ -297,7 +298,7 @@ jobs:
           # by default; `min_accuracy` lowers that bar and `routing: optional`
           # removes it.
           args: >-
-            --routing-skills '${{ matrix.skill }}'
+            --routing-room '${{ matrix.skill }}'
             --min-accuracy '${{ inputs.min_accuracy }}'
             --output routing-report.json
             --keep-logs routing-logs

.github/workflows/selftest.yml

@@ -51,18 +51,21 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
+      # Laid out the way the default glob expects one: skills at the root, so
+      # not a step below passes `skills`, which is the configuration a repo
+      # starting from nothing has.
       - name: Build a repo to test
         shell: bash
         run: |
           set -euo pipefail
-          mkdir -p fixture/skills/demo-skill/evals
-          cat > fixture/skills/demo-skill/SKILL.md <<'EOF'
+          mkdir -p fixture/demo-skill/evals
+          cat > fixture/demo-skill/SKILL.md <<'EOF'
           ---
           name: demo-skill
           description: Does demonstrable things, for a test that needs a skill.
           ---
           EOF
-          cat > fixture/skills/demo-skill/skill-card.md <<'EOF'
+          cat > fixture/demo-skill/skill-card.md <<'EOF'
           # Skill Card
 
           ## Description
@@ -77,7 +80,7 @@ jobs:
 
           MIT
           EOF
-          cat > fixture/skills/demo-skill/evals/evals.json <<'EOF'
+          cat > fixture/demo-skill/evals/evals.json <<'EOF'
           {
             "evaluations": [
               {"id": "demo-a", "skill_should_trigger": true, "prompt": "do the demo thing"},
@@ -105,8 +108,9 @@ jobs:
 
       # This repo has no skills, so what is left is its own prose -- which is
       # exactly the half a harness for checking references should not be
-      # allowed to get wrong about itself. Offline, so nothing here depends on
-      # the network being polite today.
+      # allowed to get wrong about itself. `--docs` is also what makes a run
+      # that finds no skill a real run rather than one that graded nothing.
+      # Offline, so nothing here depends on the network being polite today.
       - name: Check this repo's own references
         uses: ./
         with:
@@ -119,7 +123,7 @@ jobs:
         uses: ./
         with:
           command: select
-          args: --all --routing-skills demo-skill
+          args: --all --routing-room demo-skill
           repo: fixture
           source: .
 
@@ -137,7 +141,7 @@ jobs:
         uses: ./
         with:
           command: select
-          args: --all --routing-skills none
+          args: --all --routing-room none
           repo: fixture
           source: .
 

.github/workflows/skill-evals.yml

@@ -20,7 +20,7 @@ name: skill-evals
 #       secrets: inherit
 #       with:
 #         skill_globs: skills/*
-#         routing_skills: my-skill,its-neighbour
+#         routing_room: my-skill,its-neighbour
 #
 # The caller's `with:` block is the whole configuration. There is no config file
 # in the repo under test: a repo that runs these evals already has a workflow
@@ -32,7 +32,7 @@ name: skill-evals
 # in YAML cannot be tested and cannot be run locally. What is left here is job
 # topology, which is the one thing that has to be YAML.
 #
-#   * routing installs the skills named in `routing_skills` side by side and
+#   * routing installs the skills named in `routing_room` side by side and
 #     grades which one fires. Who a skill competes against is what the score
 #     means, so wherever there is a choice it is a decision a human makes
 #     rather than one this workflow guesses; a repo with a single skill has no
@@ -64,10 +64,12 @@ on:
       # --- what this repo looks like ---------------------------------------
       skill_globs:
         description: >-
-          Globs naming the directories that hold skills, comma-separated.
+          Globs naming the directories that are skills, comma-separated. The
+          default is every directory at the repository root; a repo that keeps
+          its skills together says so with `skills/*`.
         type: string
-        default: "skills/*"
-      routing_skills:
+        default: "./*"
+      routing_room:
         description: >-
           Skills to install side by side for the routing run, comma-separated,
           `all` for every skill with a dataset, or `none` to skip routing.
@@ -305,7 +307,7 @@ jobs:
           IGNORE_GATES: ${{ inputs.ignore_gates && '--ignore-gates' || '' }}
           BASE: ${{ github.event.pull_request.base.sha }}
           HEAD: ${{ github.event.pull_request.head.sha }}
-          ROUTING_SKILLS: ${{ inputs.routing_skills }}
+          ROUTING_ROOM: ${{ inputs.routing_room }}
           INFRA_PATHS: ${{ inputs.infra_paths }}
           BEHAVIOR_RUNNER: ${{ inputs.behavior_runner }}
           BEHAVIOR_OS: ${{ inputs.behavior_os }}
@@ -340,10 +342,10 @@ jobs:
               args += ["--ignore-gates", extended]
 
           # What this repo looks like. Planning is the only step that needs all
-          # of it, so this is the one place that spells it out; `--skills` goes
-          # to the action instead, because the launcher needs it too.
+          # of it, so this is the one place that spells it out; `--skills-dir`
+          # goes to the action instead, because the launcher needs it too.
           for flag, name in (
-              ("--routing-skills", "ROUTING_SKILLS"),
+              ("--routing-room", "ROUTING_ROOM"),
               ("--infra-paths", "INFRA_PATHS"),
               ("--behavior-runner", "BEHAVIOR_RUNNER"),
               ("--behavior-os", "BEHAVIOR_OS"),
@@ -508,7 +510,7 @@ jobs:
           # the way a shell would, and a comma-separated list a caller wrote
           # with spaces in it would otherwise arrive as several arguments.
           args: >-
-            --routing-skills '${{ inputs.routing_skills }}'
+            --routing-room '${{ inputs.routing_room }}'
             --min-accuracy '${{ inputs.min_accuracy }}'
             ${{ needs.discover.outputs.extended }}
             ${{ inputs.only && format('--only ''{0}''', inputs.only) || '' }}

README.md

@@ -36,7 +36,7 @@ uv tool install git+https://github.com/amd/skillscope
 skillscope structural                        # no agent, no tokens
 skillscope structural --external             # the same, plus checking external URLs
 skillscope behavioral --skill my-skill       # needs an authenticated `claude` CLI
-skillscope routing --routing-skills my-skill,its-neighbour
+skillscope routing --routing-room my-skill,its-neighbour
 ```
 
 ## Run it from your CI

action.yml

@@ -44,9 +44,10 @@ inputs:
     default: ""
   skills:
     description: >-
-      Globs naming the directories that hold skills, comma-separated. The
+      Globs naming the directories that are skills, comma-separated. The
       launcher needs these to find a skill's version pin before it has fetched
-      anything, and passes them on to the harness. Default: skills/*.
+      anything, and passes them on to the harness. Default: ./*, every
+      directory at the repo root.
     required: false
     default: ""
   repo:

bootstrap/launch.py

@@ -24,7 +24,7 @@
     SKILLSCOPE_COMMAND   the subcommand, e.g. "structural"
     SKILLSCOPE_ARGS      further arguments, shell-quoted
     SKILLSCOPE_REPO      root of the repo under test (default ".")
-    SKILLSCOPE_SKILLS    globs naming the directories that hold skills
+    SKILLSCOPE_SKILLS    globs naming the directories that are skills
     SKILLSCOPE_SOURCE    owner/repo (or a local path) to install from
     SKILLSCOPE_REQUESTED an explicit version, which wins outright
     SKILLSCOPE_VERSION   a version from the environment

bootstrap/resolve_version.py

@@ -36,7 +36,9 @@
 import sys
 from pathlib import Path
 
-DEFAULT_SKILL_GLOBS = ["skills/*"]
+# Kept in step with skillscope.config by hand: this script is deliberately
+# ignorant of the package it is about to fetch, so it cannot import the value.
+DEFAULT_SKILL_GLOBS = ["./*"]
 DATASET_RELPATH = "evals/evals.json"
 VERSION_KEY = "skillscope_version"
 
@@ -114,17 +116,17 @@ def main(argv: list[str] | None = None) -> int:
         "--default", default="", help="Fallback when nothing else pins a version."
     )
     parser.add_argument(
-        "--skills",
+        "--skills-dir",
         default="",
         help=(
-            "Comma-separated globs naming the directories that hold skills, "
-            "so a repo that keeps them somewhere unusual is still searched for "
-            f"the pin. Default: {','.join(DEFAULT_SKILL_GLOBS)}."
+            "Comma-separated globs naming the directories that are skills, so "
+            "a repo that keeps them somewhere other than its root is still "
+            f"searched for the pin. Default: {','.join(DEFAULT_SKILL_GLOBS)}."
         ),
     )
     args = parser.parse_args(argv)
 
-    globs = [g.strip() for g in args.skills.split(",") if g.strip()]
+    globs = [g.strip() for g in args.skills_dir.split(",") if g.strip()]
     version = resolve(
         root=Path(args.repo).expanduser().resolve(),
         requested=args.version,

docs/authoring-evals.md

@@ -49,7 +49,7 @@ skills in one workspace, so it is the same assertion either way, and filing it
 under the neighbour keeps `false` meaning "nothing fires".
 
 Your dataset's prompts are graded in a routing run only when your skill is in
-the room — one the workflow lists in `routing_skills`, or the only skill the
+the room — one the workflow lists in `routing_room`, or the only skill the
 repo has, which needs no listing. How the room is chosen is in
 [usage.md](usage.md#who-a-skill-competes-against-is-listed-not-inferred). A
 prompt expecting a skill that is not in the room could only ever lose, and a
@@ -207,7 +207,7 @@ holds.
 ```bash
 skillscope structural                               # your folder, prose, and dataset
 skillscope behavioral --skill <your-skill>          # your skill, end to end
-skillscope routing --routing-skills <your-skill>,<a-neighbour>
+skillscope routing --routing-room <your-skill>,<a-neighbour>
 skillscope routing --only <case-id> --keep-logs logs # one case, keeping the transcript
 ```
 

docs/usage.md

@@ -39,8 +39,8 @@ places to disagree.
 
 | Flag | Workflow input | Default | What it decides |
 | --- | --- | --- | --- |
-| `--skills` | `skill_globs` | `skills/*` | Globs naming the directories that hold skills. A skill is a directory with a `SKILL.md`, and its directory name is its identity. |
-| `--routing-skills` | `routing_skills` | the only skill, if there is one | The skills a routing run installs side by side. `all` means every skill with a dataset, `none` means no routing run, and blank means a repo with one skill runs that skill while a repo with several has to choose. |
+| `--skills-dir` | `skill_globs` | the directory you are in | Globs naming the directories that *are* skills, relative to the repo root. A skill is a directory with a `SKILL.md`, and its directory name is its identity. A repo that keeps its skills together passes `skills/*`. |
+| `--routing-room` | `routing_room` | `--skill` if given, else the only skill, if there is one | The skills a routing run installs side by side. `all` means every skill with a dataset, `none` means no routing run, and blank means `--skill` is the room, a repo with one skill runs that skill, and a repo with several and no `--skill` has to choose. |
 | `--infra-paths` | `infra_paths` | none | Paths that change the harness rather than one skill, so touching one re-runs every skill instead of guessing at the blast radius. Your own workflow file belongs here. |
 | `--docs` | `doc_globs` | none | Markdown outside the skills whose references should be checked too: a README, a docs tree. The skills themselves are always checked. |
 | `--exclude-url` | `excluded_urls` | none | Regexes matching URLs the external reference check leaves alone. For hosts that are auth-gated or that answer a runner's IP with a 403. |
@@ -60,23 +60,44 @@ whole command. Routing adds `--jobs`, `--case-timeout`, `--max-tool-calls`,
 `--max-budget-usd`, `--keep-logs`, and `--min-accuracy`. `--help` is the
 authority on all of them.
 
+### Where the skills are
+
+Every path in the table is relative to the repository root, which is the only
+base a workflow input, a line of `git diff --name-only`, and a root-relative
+markdown link can all agree on. `skills/*` in a workflow means the same thing
+wherever its runner happens to have started.
+
+The one thing measured from somewhere else is `--skills-dir` when you do not
+pass it: then it is every directory in the one you ran the command from.
+Standing in a tree of skills and typing `skillscope structural` can only mean
+these ones, and a repo that keeps them a level down works with a `cd` rather
+than a flag. Under CI the two bases coincide, because the launcher runs from
+the repo root — so a run with no `--skills-dir` grades the directories at the
+root, and a repo whose skills live anywhere else names them.
+
+Either way it looks one level down and no further. Searching a whole tree for
+every `SKILL.md` finds vendored copies, fixtures, and a contributor's local
+install, and each of those silently changes a routing score.
+
 ### Who a skill competes against is listed, not inferred
 
-`routing_skills` is the one input with no useful default, because the answer is
+`routing_room` is the one input with no useful default, because the answer is
 what the score *means*. Install every skill on disk and a work-in-progress
 directory drops everyone's number; install only the skill under review and it
 wins every prompt by walkover. A skill you leave off the list still gets its
 dataset checked and its behavioral cases run — it just does not move anybody's
 routing score. Listing them also makes the change visible: a skill joining or
 leaving the room moves every other skill's number, and that deserves a diff.
 
-A repo with one skill has no such choice, so leave `routing_skills` blank and
+A repo with one skill has no such choice, so leave `routing_room` blank and
 its only skill is the room. The score is then the half of the question that can
 be answered alone — does the skill fire on its own prompts, and does it stay
 quiet on its near misses and the shared negatives — and it stops meaning that
 the moment a second skill shows up, at which point the flag becomes required
-again rather than quietly picking a room for you. To turn routing off instead,
-say so: `routing_skills: none`.
+again rather than quietly picking a room for you. Naming skills with `--skill`
+and leaving `--routing-room` off is still a listing, not a guess: those skills
+are the room. The workflow input has no `--skill`, so a repo with several still
+has to choose. To turn routing off instead, say so: `routing_room: none`.
 
 ## What the structural check asserts
 
@@ -95,8 +116,11 @@ agent that simply never uses the skill. So every `SKILL.md` is read first:
 | `description` | non-empty, at most 1024 characters |
 | body | at most 500 lines — past that it is reference material, and an agent reads it in full every time the skill loads |
 
-A directory your skill globs match that holds no `SKILL.md` is reported too.
-Either the file is missing or the glob is too wide, and both are worth one line.
+A directory that holds no `SKILL.md` is simply not a skill, and is passed over
+without a word. Matching *no* skill at all is the case that is reported, since
+a run that graded nothing and called itself green is the one way this harness
+can lie about a repo; a run given `--docs` is exempt, having been asked to
+check a repo's own prose.
 
 Whatever else your repo asks of a skill is policy rather than format, so it is
 configuration:
@@ -222,7 +246,7 @@ Naming several, and holding them to different bars:
 
 | Input | Default | What it decides |
 | --- | --- | --- |
-| `skills` | `skills/*` | The skills to grade: a directory, or a glob matching several, one per line or comma-separated. |
+| `skills` | `./*` | The skills to grade: a directory, or a glob matching several, one per line or comma-separated. The default is every directory at the repo root. |
 | `structural` | `required` | `required`, `optional`, or `off`. |
 | `routing` | `required` | `required`, `optional`, or `off`. |
 | `behavioral` | `required` | `required`, `optional`, or `off`. |
@@ -260,7 +284,7 @@ jobs:
     uses: amd/skillscope/.github/workflows/skill-evals.yml@main
     secrets: inherit
     with:
-      routing_skills: my-skill,its-neighbour
+      routing_room: my-skill,its-neighbour
       api_key_secret: MY_MODEL_API_KEY
 ```