Skip to content

DM-14/DM-16 model fixes and File Data Use Codes (14.1.0) - #277

Merged
Bankso merged 6 commits into
mainfrom
review-fixes
Oct 28, 2026
Merged

Bankso merged 6 commits into
mainfrom
review-fixes

Conversation

@Bankso

@Bankso Bankso commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Changes following review of PR #262. Releases as 14.1.0 (see plans/release_14.1.0_notes.md).

  • DM-14: qc_model/qc_attribute_mapping.csv DatasetView attribute Data Use Codes renamed Dataset Data Use Codes.
  • DM-16: model fixes: duplicate Not-Applicable tumor type removed, DOI description (https://doi.org/{doi}), DUO shorthands aligned to official DUO, DUO ID pattern widened to DUO:\d{7,8}, trailing whitespace removed from source values, Dataset Doi no longer required. DataCatalog dataType stays lowercase immunoassay.
  • New File Data Use Codes: a pattern-validated DUO list with no controlled values and no conditional companion columns. File View and all 18 file-level assay templates use it instead of Data Use Codes (breaking for file manifests: the column header changes). Data Use Codes is kept for Study; DSP Data Use Codes for DataDSP; Dataset Data Use Codes for Dataset.
  • Regenerated mc2.model.csv, mc2.model.jsonld, json_schemas/, templates/.

Deferred: kg-pipeline still maps the live annotation key DataUseCodes to Data Use Codes (and its generated LinkML schema/fixtures); to be updated once portal tables are re-annotated.

Review: independent review found no correctness bugs.

Update qc_model/qc_attribute_mapping.csv to change the DatasetView attribute name from 'Data Use Codes' to 'Dataset Data Use Codes' for clarity/consistency.
Bankso and others added 3 commits October 26, 2026 12:00
…taset Doi)

Source edits in modules/** with mc2.model.csv, mc2.model.jsonld,
json_schemas, templates and all_valid_values.csv regenerated via make all.
- Remove duplicate Tumor Type value Not-Applicable
- Immunoassay casing in DataCatalog dataType
- Publication Doi description uses https://doi.org/{doi}
- DUO shorthands to official (US, RS, TS, IS, GS, DS)
- DUO ID pattern widened to DUO:\d{7,8}
- Strip leading/trailing whitespace from values
- Dataset Doi not required in DatasetView

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner decision: dataType values stay lowercase. Revert the
immunoassay -> Immunoassay rename in dataCatalog_data_type.csv and
regenerate. The shared bts:Immunoassay JSON-LD node displayName may
remain lowercase (known quirk, noted in the 14.0.1 release notes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Bankso

Bankso commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Added the 14.0.1 patch items (DM-16) on top of DM-14: Tumor Type Not-Applicable removed; Publication Doi description now https://doi.org/{doi}; official DUO shorthands in duo.csv (US, RS, TS, IS, GS, DS); DUO ID pattern widened to DUO:\d{7,8} (DUO mints NPOA as DUO:00000044); Dataset Doi no longer required; trailing whitespace stripped from 4 valid values and one Parent value. Release notes: plans/release_14.0.1_notes.md. Checks: scripts/check_json_schemas.py 0 failures, pytest 22 passed. Once this merges, tag v14.0.1 at the merge commit.

🤖 Generated with Claude Code

File View and all file-level assay templates now use File Data Use Codes
(pattern-validated DUO list, no conditional companion columns). Data Use
Codes is retained for Study. Release notes renamed 14.0.1 -> 14.1.0.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@Bankso Bankso changed the title Changes following review (pre v14.0.0 release) DM-14/DM-16 model fixes and File Data Use Codes (14.1.0) Oct 26, 2026
@Bankso Bankso added the minor PR label for a minor update label Oct 26, 2026
@Bankso
Bankso marked this pull request as ready for review October 26, 2026 23:34
@Bankso
Bankso requested a review from a team as a code owner October 26, 2026 23:34

@aditya-nath-sage aditya-nath-sage left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

…investigators) and regenerate mc2_model from 14.1.0 (#278)

* Split Tool/EducationalResource license and Grant investigator as lists, with legacy comma fallback

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Regenerate cckp_portal.ttl: license and investigator no longer single-valued

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Regenerate mc2_model LinkML and Turtle from mc2.model.csv 14.1.0

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* kg-pipeline: split a legacy comma-joined Grant.investigator with the person-name heuristic

A plain comma split cut "Last, First" names (PEROU, CHARLES M) into two
people. legacy_comma_separated takes a mode: "true" (comma) for the licenses,
"person_names" (split_person_names, now in harmonize.py and shared with
link_scdm) for Grant.investigator. On the extracted Grant table the result
equals link_scdm.split_investigators for every row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* kg-pipeline: keep each investigator as one item, same text in the comma and "|" forms

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* kg-pipeline: map File View data use codes to File Data Use Codes and keep resolving DUO terms

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* kg-pipeline: drop the tumorType crosswalk row for the removed Not-Applicable term

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* kg-pipeline: test build_triples for multivalued license and investigator cells

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* kg-pipeline: license and investigator stay comma-joined STRING columns (CCKP D-Q51)

The CCKP cutover keeps these portal columns as text (license may become
STRING_LIST later), so the annotation is renamed comma_separated and the
descriptions say comma-joined text. The splitting rule is unchanged: a "|"
cell splits on "|", otherwise on commas (licenses) or with the person-name
heuristic (investigators).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@Bankso
Bankso merged commit 2b22302 into main Oct 28, 2026
1 check passed
@Bankso
Bankso deleted the review-fixes branch October 28, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

minor PR label for a minor update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants