[ruromero]

Summary

• Add domain-neutral guidance to compare eval assertions against explicit requirements and fixture semantics.
• Add repeated-CPE coverage without asserting uniqueness.
• Clarify the project contract: every image row needs a CPE, but CPE values may repeat unless a specific contract requires uniqueness.

Implements TC-7147

Summary by Sourcery

Align implementation-task eval assertions with authoritative requirements and fixture semantics.

New Features:

• Add eval coverage for repeated CPE values to ensure assertions follow supplied fixture semantics.

Bug Fixes:

• Prevent eval assertions from requiring CPE uniqueness when the contract only requires every image row to have a CPE.

Enhancements:

• Add guidance for validating eval assertions against explicit requirements and fixture semantics.
• Clarify that CPE values may repeat across Container Images rows unless uniqueness is explicitly required.

[sourcery-ai]

Reviewer's Guide

This PR aligns implement-task eval behavior with authoritative requirements and fixture semantics by clarifying that every image row needs a CPE but repeated CPEs are valid, adding coverage for that case, and documenting a review workflow that prevents assertions from introducing unsupported uniqueness constraints.

Entity relationship diagram for repeated image CPE values

erDiagram
    IMAGE_ROW {
        string product_version
        string component
        string cpe
    }
    IMAGE_ROW }o--|| CPE_VALUE : includes
    CPE_VALUE {
        string value
    }

Loading

Flow diagram for eval assertion and fixture consistency

flowchart TD
    A[Review eval case] --> B[Read explicit requirements and authoritative contract]
    B --> C[Inspect supplied fixture values and relationships]
    C --> D{Does the assertion match both sources?}
    D -->|Yes| E[Keep or add the assertion]
    D -->|No| F{Is the source of truth unclear?}
    F -->|Yes| G[Ask the user]
    F -->|No| H[Correct the assertion]
    H --> E
    C --> I[Repeated values are valid unless uniqueness is explicitly required]
    I --> D

Loading

File-Level Changes

Change	Details	Files
Document the Container Images CPE contract to require presence without imposing uniqueness.	Require a non-empty CPE for every image row. Explicitly allow repeated CPE values unless uniqueness is contractually required.	docs/project-config-contract.md
Add an eval case and synthetic fixture that expose assertion errors caused by conflating presence with uniqueness.	Add repeated-CPE fixture data for two image rows. Add a task scenario whose proposed assertion incorrectly requires unique CPEs.	evals/implement-task/evals.json evals/implement-task/files/task-eval-repeated-values.md
Add implementation guidance for validating eval assertions against requirements and fixture semantics.	Require comparison with task requirements and authoritative contracts. Require inspection of fixture relationships and prohibit unsupported constraints. Define correction or escalation behavior when assertions conflict with requirements or fixtures.	plugins/sdlc-workflow/skills/implement-task/SKILL.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've reviewed your changes and they look great!

Sourcery assessment

Approved.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

[github-actions]

Eval Results

Source head: 462aac2
Merge: 827cfdf

Eval Results: implement-task

Eval	Passed	Failed	Pass Rate
eval-1	11/11	0	100%
eval-10	5/5	0	100%
eval-11	5/5	0	100%
eval-12	5/5	0	100%
eval-13	5/5	0	100%
eval-14	6/6	0	100%
eval-2	5/5	0	100%
eval-3	6/6	0	100%
eval-4	5/6	1	83%
eval-5	7/7	0	100%
eval-6	4/4	0	100%
eval-7	5/5	0	100%
eval-8	5/5	0	100%
eval-9	5/5	0	100%

Failed Assertions

eval-4: 1 failing assertion

• Assertion: "The plan scopes changes to the files listed in Files to Modify and Files to Create — no files outside those sections are modified (constraint 1.4, 5.1)"
  Evidence: "plan.md includes a 'Module Registration' section (lines 79-80) that adds an additional file modification: 'Update modules/fundamental/src/sbom/model/mod.rs to add pub mod export;'. This file is NOT listed in the plan's numbered Files to Modify (sections 1-2) or Files to Create (sections 3-5) sections. While the plan justifies this as being 'within the modules/fundamental/src/sbom/ scope', it is an additional file beyond the explicitly listed sections, violating the strict scoping constraint."

Pass rate: 99% · Tokens: 40,917 · Duration: 105s

Baseline (6922b59c): 97% · 39,597 tokens · 126s

---

Generated by sdlc-workflow/run-evals v0.13.10

[fullsend-ai-review]

🤖 Finished Verify Pr · ✅ Success · Started 10:36 PM UTC · Completed 10:46 PM UTC

Commit: 462aac2 · View workflow run →

Runtime: claude · Model: claude-opus-4-8 · Effort: high · Cost: $4.88

[fullsend-ai-review]

Verification Report for TC-7147 (commit 462aac2)

Check	Result	Details
Review Feedback	N/A	No inline comments or substantive review bodies on the PR (sole review is a bot approval).
Root-Cause Investigation	N/A	No sub-tasks created; nothing to investigate.
Scope Containment	PASS	All 4 task files present; no out-of-scope files.
Diff Size	PASS	87 insertions / 0 deletions across 4 files - proportionate to scope.
Commit Traceability	PASS	All 3 commits reference TC-7147.
Sensitive Patterns	PASS	No secrets detected; only synthetic CPE example identifiers.
CI Status	PASS	All 7 check-runs + the Eval PR Run commit status report success.
Acceptance Criteria	PASS	4 of 4 criteria met.
Test Quality	WARN	Eval Quality WARN: 1 pre-existing eval-4 failure (no regressions); overall 99% (up from 97% baseline); new eval-14 passed 6/6. Repetitive/Doc: N/A.
Test Change Classification	ADDITIVE	1 new eval case + 1 new fixture; nothing removed or weakened.
Verification Commands	PASS	skillsaw / plugin-validate equivalents green on head SHA (Skill Lint + Plugin Validation); not re-runnable in sandbox.

Overall: PASS

All blocking checks pass. The PR cleanly implements TC-7147: it adds implement-task guidance for checking eval assertions against both explicit requirements and fixture semantics, adds a focused eval case (eval-14, 6/6) plus a SYNTHETIC TEST DATA-annotated fixture, and documents per-row CPE presence without implying uniqueness. Scope, commit traceability, security, CI, and all 4 acceptance criteria are satisfied.

The single WARN (Test Quality) is informational and does not affect the overall result: one eval-4 assertion fails, but it is pre-existing - the identical assertion fails at the base-branch baseline (5/6) with the same root cause - and is unrelated to this PR's subject. No regression, so no sub-tasks were created. Overall eval pass rate improved 97% -> 99%.

---

This comment was AI-generated by sdlc-workflow/verify-pr v0.13.10.

[mrizzi]

Sourcery wrote your changes are great, evals are fine and verify-pr gave PASS verdict, who am I to stop this!? 😉