From bfc0f696d549d633f1287dec3197d9d2133e59b7 Mon Sep 17 00:00:00 2001 From: "K.Filippopolitis" <56073635+KFilippopolitis@users.noreply.github.com> Date: Fri, 23 Oct 2026 22:27:52 +0300 Subject: [PATCH 01/31] refactor(data-handling): rename statistic-analysis-panel to DataHandlingComponent Moves src/app/pages/experiment-studio/statistic-analysis-panel to data-handling/, renames StatisticAnalysisPanelComponent to DataHandlingComponent (selector app-data-handling) and updates the experiment studio wiring, specs and docs. No behavior change. --- AGENTS.md | 2 +- docs/context/conventions.md | 2 +- docs/context/module-index.md | 2 +- docs/context/risk-register.md | 4 ++-- .../data-handling.component.css} | 0 .../data-handling.component.html} | 0 .../data-handling.component.spec.ts} | 12 ++++++------ .../data-handling.component.ts} | 12 ++++++------ .../data-handling.pipeline.spec.ts} | 12 ++++++------ .../data-handling.preprocessing.css} | 8 ++++---- .../data-handling.results.css} | 4 ++-- .../kmeans-cluster-source.component.css | 0 .../kmeans-cluster-source.component.html | 0 .../kmeans-cluster-source.component.spec.ts | 0 .../kmeans-cluster-source.component.ts | 0 .../experiment-studio.component.html | 2 +- .../experiment-studio.component.spec.ts | 14 +++++++------- .../experiment-studio.component.ts | 12 ++++++------ .../variables-panel/variables-panel.component.css | 2 +- 19 files changed, 44 insertions(+), 44 deletions(-) rename src/app/pages/experiment-studio/{statistic-analysis-panel/statistic-analysis-panel.component.css => data-handling/data-handling.component.css} (100%) rename src/app/pages/experiment-studio/{statistic-analysis-panel/statistic-analysis-panel.component.html => data-handling/data-handling.component.html} (100%) rename src/app/pages/experiment-studio/{statistic-analysis-panel/statistic-analysis-panel.component.spec.ts => data-handling/data-handling.component.spec.ts} (99%) rename src/app/pages/experiment-studio/{statistic-analysis-panel/statistic-analysis-panel.component.ts => data-handling/data-handling.component.ts} (99%) rename src/app/pages/experiment-studio/{statistic-analysis-panel/statistic-analysis-panel.pipeline.spec.ts => data-handling/data-handling.pipeline.spec.ts} (99%) rename src/app/pages/experiment-studio/{statistic-analysis-panel/statistic-analysis-panel.preprocessing.css => data-handling/data-handling.preprocessing.css} (98%) rename src/app/pages/experiment-studio/{statistic-analysis-panel/statistic-analysis-panel.results.css => data-handling/data-handling.results.css} (99%) rename src/app/pages/experiment-studio/{statistic-analysis-panel => data-handling}/kmeans-cluster-source/kmeans-cluster-source.component.css (100%) rename src/app/pages/experiment-studio/{statistic-analysis-panel => data-handling}/kmeans-cluster-source/kmeans-cluster-source.component.html (100%) rename src/app/pages/experiment-studio/{statistic-analysis-panel => data-handling}/kmeans-cluster-source/kmeans-cluster-source.component.spec.ts (100%) rename src/app/pages/experiment-studio/{statistic-analysis-panel => data-handling}/kmeans-cluster-source/kmeans-cluster-source.component.ts (100%) diff --git a/AGENTS.md b/AGENTS.md index 3e969c33b..8e08c81d0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,7 +10,7 @@ This repository contains `fl-platform`, the Angular 22 standalone frontend for t - `src/app/services/`: auth/session, experiment orchestration, dashboard data access, algorithm rules, labeling, runtime env, theme, errors, and PDF/CSV exports. - `src/app/models/`: frontend and backend DTOs/interfaces for users, algorithms, experiments, filters, and data models. - `src/app/core/`: algorithm mapping, result enum mapping, constants, and result utility logic. -- `src/app/pages/experiment-studio/`: step views switched via a sticky horizontal stepper — `experiment-studio.component` shows one step view at a time (Datasets & Variables · Data review & preprocessing · Algorithm); all views stay mounted (hidden via CSS) so state is preserved. On the Data review step the stepper expands a sub-step row (Filtering, Raw Summary, Preprocessing, Processed Summary, Transformation) that drives the `statistic-analysis-panel` sections (`goToSection`). In-panel section targets use `scroll-margin-top` so `scrollIntoView` clears sticky chrome. Also contains variable/filter selection, QueryBuilder filter UI, algorithm configuration with variable role assignment (outcome y / covariates x on the algorithm panel), run/edit flows, statistics, visualizations, and result rendering. +- `src/app/pages/experiment-studio/`: step views switched via a sticky horizontal stepper — `experiment-studio.component` shows one step view at a time (Datasets & Variables · Data review & preprocessing · Algorithm); all views stay mounted (hidden via CSS) so state is preserved. On the Data review step the stepper expands a sub-step row (Filtering, Raw Summary, Preprocessing, Processed Summary, Transformation) that drives the `data-handling` (`DataHandlingComponent`) sections (`goToSection`). In-panel section targets use `scroll-margin-top` so `scrollIntoView` clears sticky chrome. Also contains variable/filter selection, QueryBuilder filter UI, algorithm configuration with variable role assignment (outcome y / covariates x on the algorithm panel), run/edit flows, statistics, visualizations, and result rendering. - `src/app/pages/experiments-dashboard/`: experiment list/search/pagination, detail view, compare mode, sharing, delete/edit/name updates, and result export. - `src/app/pages/terms-page/`: NDA/TOS display and acceptance flow. - `src/app/pages/account-page/`: account/profile view and logout entry. diff --git a/docs/context/conventions.md b/docs/context/conventions.md index ee621fc5c..63f763f80 100644 --- a/docs/context/conventions.md +++ b/docs/context/conventions.md @@ -30,7 +30,7 @@ ### UI and Styling - Use component CSS for feature-specific styles. - Use `src/styles.css` for app-wide styles and QueryBuilder theming. -- A component may list several stylesheets in `styleUrls`; Angular concatenates them **in array order**, so that order is the cascade. Splitting a file must preserve the original order and must not move a rule across another rule that shares its selector — `statistic-analysis-panel` is split this way (`.component.css` shell + pipeline canvas, then `.preprocessing.css`, then `.results.css`). +- A component may list several stylesheets in `styleUrls`; Angular concatenates them **in array order**, so that order is the cascade. Splitting a file must preserve the original order and must not move a rule across another rule that shares its selector — `data-handling` (`DataHandlingComponent`) is split this way (`.component.css` shell + pipeline canvas, then `.preprocessing.css`, then `.results.css`). - Consult `DESIGN.md` before visual changes. - Existing branding assets live under `src/assets`. diff --git a/docs/context/module-index.md b/docs/context/module-index.md index 3e025bd56..40f3edcac 100644 --- a/docs/context/module-index.md +++ b/docs/context/module-index.md @@ -42,7 +42,7 @@ Notes: Stored historical experiment payloads may depend on legacy algorithm keys ## `src/app/pages/experiment-studio` Purpose: Experiment composition workflow. -Key files: `experiment-studio.component.*`, `variables-panel/*`, `statistic-analysis-panel/*`, `algorithm-panel/*`, `execution-panel/*` (result view + docked setup summary), `stepper/*`, `shared/*`, `guide/*`. +Key files: `experiment-studio.component.*`, `variables-panel/*`, `data-handling/*`, `algorithm-panel/*`, `execution-panel/*` (result view + docked setup summary), `stepper/*`, `shared/*`, `guide/*`. Used by: `/experiment-studio` route and dashboard edit flows. Rules: Keep page-specific UI here; use `ExperimentStudioService` for shared selection/run state. Tests: Component specs live near components. diff --git a/docs/context/risk-register.md b/docs/context/risk-register.md index 17e0eaa4e..04f668176 100644 --- a/docs/context/risk-register.md +++ b/docs/context/risk-register.md @@ -15,7 +15,7 @@ | Docker image publishing | Release workflow uses registry credentials and pushes images. | `.github/workflows/publish_images.yml`. | Never print secrets; review registry/tag changes carefully. | Yes | | EBRAINS mirroring | Workflow pushes code/tags to an external GitLab mirror. | `.github/workflows/ebrains.yml`. | Avoid changing mirror refs or credentials handling without owner review. | Yes | | Dependencies | Updates can affect Angular build, bundle output, and chart/export libraries. | `package.json`, `package-lock.json`. | Explain need, review lockfile, run build/tests. | Yes | -| Cohort / category-rule filters (edit mode) | Stored filter trees can be re-sent in a shape the engine rejects, or silently rewritten, so an experiment that runs from the API fails from Studio. | `core/filter-logic.utils.ts`, `filter-config-modal.component.ts` (`backendToGroup`/`normalizeCondition`), `statistic-analysis-panel.component.ts` (`commitTransformationRuleFilters`). | Treat backend filter trees as untrusted input: hydrate every shape, never default an unknown operator to `equal`, never overwrite a loaded rule with an empty builder. Verify edit-mode runs against an experiment whose filters use `in`/`is_not_null`. | Yes | -| Derived-column stage (ungated) | Preview and Apply run on whatever the cards can express, so an unfinished card contributes nothing to the describe and to the payload, and a duplicate column name persists its first occurrence only. | `statistic-analysis-panel.component.ts` (`commitTransformationRuleFilters`, `buildTransformationConfigForDraft`, `transformationDraftStatus`). | Do not reintroduce a Preview/Apply gate over loaded data: it strands experiments whose category rules were authored outside this UI. Keep the commit's rule-safety check — never overwrite a rule its builder could not read — and let that rule's own note (`.transformation-saved-filter-note`, driven by `unloadedInput()`) plus the `Pending`/`Duplicate name` chips stay the only warning. `transformationCanApply` may dim Apply while no card forms a config, which must stay a statement about an empty payload and never a verdict on loaded rules; and Apply must keep the stage open while `transformationHasPendingChange` is true, or the chip that reports unfinished work folds away with it. Manual QA must cover Apply with a duplicate column name, and Apply on an experiment whose stored category rules this UI cannot author. | Yes | +| Cohort / category-rule filters (edit mode) | Stored filter trees can be re-sent in a shape the engine rejects, or silently rewritten, so an experiment that runs from the API fails from Studio. | `core/filter-logic.utils.ts`, `filter-config-modal.component.ts` (`backendToGroup`/`normalizeCondition`), `data-handling.component.ts` (`commitTransformationRuleFilters`). | Treat backend filter trees as untrusted input: hydrate every shape, never default an unknown operator to `equal`, never overwrite a loaded rule with an empty builder. Verify edit-mode runs against an experiment whose filters use `in`/`is_not_null`. | Yes | +| Derived-column stage (ungated) | Preview and Apply run on whatever the cards can express, so an unfinished card contributes nothing to the describe and to the payload, and a duplicate column name persists its first occurrence only. | `data-handling.component.ts` (`commitTransformationRuleFilters`, `buildTransformationConfigForDraft`, `transformationDraftStatus`). | Do not reintroduce a Preview/Apply gate over loaded data: it strands experiments whose category rules were authored outside this UI. Keep the commit's rule-safety check — never overwrite a rule its builder could not read — and let that rule's own note (`.transformation-saved-filter-note`, driven by `unloadedInput()`) plus the `Pending`/`Duplicate name` chips stay the only warning. `transformationCanApply` may dim Apply while no card forms a config, which must stay a statement about an empty payload and never a verdict on loaded rules; and Apply must keep the stage open while `transformationHasPendingChange` is true, or the chip that reports unfinished work folds away with it. Manual QA must cover Apply with a duplicate column name, and Apply on an experiment whose stored category rules this UI cannot author. | Yes | | Dashboard history cap | A filter the backend cannot express is answered from one ownership-scoped snapshot of the history, capped at `fullHistoryCap` runs; a search past it silently misses older runs. | `experiments-dashboard.service.ts` (`loadAllExperiments`, `historyTruncated`). | The cap is stated in the list summary now, not hidden; removing the signal without a backend substring/date filter brings the silent miss back. | Yes | | Design and branding | UI changes can violate MIP brand guidelines. | `DESIGN.md`, logos/assets. | Follow approved colors, logo rules, and UI hierarchy. | For visible brand changes | diff --git a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.component.css b/src/app/pages/experiment-studio/data-handling/data-handling.component.css similarity index 100% rename from src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.component.css rename to src/app/pages/experiment-studio/data-handling/data-handling.component.css diff --git a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.component.html b/src/app/pages/experiment-studio/data-handling/data-handling.component.html similarity index 100% rename from src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.component.html rename to src/app/pages/experiment-studio/data-handling/data-handling.component.html diff --git a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.component.spec.ts b/src/app/pages/experiment-studio/data-handling/data-handling.component.spec.ts similarity index 99% rename from src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.component.spec.ts rename to src/app/pages/experiment-studio/data-handling/data-handling.component.spec.ts index db6e47442..f891f56b7 100644 --- a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.component.spec.ts +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.spec.ts @@ -1,7 +1,7 @@ import type { MockedObject } from 'vitest'; import { ComponentFixture, TestBed } from '@angular/core/testing'; import { QueryList } from '@angular/core'; -import { StatisticAnalysisPanelComponent } from './statistic-analysis-panel.component'; +import { DataHandlingComponent } from './data-handling.component'; import { ExperimentStudioService } from '../../../services/experiment-studio.service'; import { ExperimentsDashboardService } from '../../../services/experiments-dashboard.service'; import { ExperimentStudioNavigationService } from '../../../services/experiment-studio-navigation.service'; @@ -13,9 +13,9 @@ import { provideZonelessChangeDetection, signal } from '@angular/core'; import { provideEchartsCore } from 'ngx-echarts'; import { createSpyObj } from '../../../testing/spy-obj.testing'; -describe('StatisticAnalysisPanelComponent', () => { - let component: StatisticAnalysisPanelComponent; - let fixture: ComponentFixture; +describe('DataHandlingComponent', () => { + let component: DataHandlingComponent; + let fixture: ComponentFixture; let mockExpService: MockedObject; let mockChartBuilder: MockedObject; let mockPdfService: MockedObject; @@ -87,7 +87,7 @@ describe('StatisticAnalysisPanelComponent', () => { mockPdfService = createSpyObj('PdfExportService', ['exportDescriptiveStatisticsPdf']); await TestBed.configureTestingModule({ - imports: [StatisticAnalysisPanelComponent], // Standalone component + imports: [DataHandlingComponent], // Standalone component providers: [ provideZonelessChangeDetection(), provideEchartsCore({ @@ -100,7 +100,7 @@ describe('StatisticAnalysisPanelComponent', () => { ] }).compileComponents(); - fixture = TestBed.createComponent(StatisticAnalysisPanelComponent); + fixture = TestBed.createComponent(DataHandlingComponent); component = fixture.componentInstance; fixture.detectChanges(); }); diff --git a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.component.ts b/src/app/pages/experiment-studio/data-handling/data-handling.component.ts similarity index 99% rename from src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.component.ts rename to src/app/pages/experiment-studio/data-handling/data-handling.component.ts index 166a630a8..758121fea 100644 --- a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.component.ts +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.ts @@ -244,22 +244,22 @@ export interface DescriptiveProgressState { } @Component({ - selector: 'app-statistic-analysis-panel', + selector: 'app-data-handling', imports: [ChartRendererComponent, HistogramComponent, FormsModule, NgTemplateOutlet, FilterConfigModalComponent, StationActionBarComponent, StationCardComponent, KMeansClusterSourceComponent], - templateUrl: './statistic-analysis-panel.component.html', + templateUrl: './data-handling.component.html', // Order is the cascade: shell + pipeline canvas, then preprocessing stations, // then shared controls / result surfaces + responsive overrides. Concatenated in this order. styleUrls: [ - './statistic-analysis-panel.component.css', - './statistic-analysis-panel.preprocessing.css', - './statistic-analysis-panel.results.css', + './data-handling.component.css', + './data-handling.preprocessing.css', + './data-handling.results.css', ], host: { '(document:click)': 'closeBatchMenuOnOutsideClick($event)', }, changeDetection: ChangeDetectionStrategy.OnPush, }) -export class StatisticAnalysisPanelComponent implements OnDestroy { +export class DataHandlingComponent implements OnDestroy { readonly processedDataInput = input([], { alias: 'processedData' }); processedData: PivotBlock[] = []; readonly variables = input([]); diff --git a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.pipeline.spec.ts b/src/app/pages/experiment-studio/data-handling/data-handling.pipeline.spec.ts similarity index 99% rename from src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.pipeline.spec.ts rename to src/app/pages/experiment-studio/data-handling/data-handling.pipeline.spec.ts index 46cfac2e0..5eabb4062 100644 --- a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.pipeline.spec.ts +++ b/src/app/pages/experiment-studio/data-handling/data-handling.pipeline.spec.ts @@ -3,7 +3,7 @@ import { ComponentFixture, TestBed } from '@angular/core/testing'; import { of, throwError } from 'rxjs'; import { provideZonelessChangeDetection, signal } from '@angular/core'; import { provideEchartsCore } from 'ngx-echarts'; -import { StatisticAnalysisPanelComponent } from './statistic-analysis-panel.component'; +import { DataHandlingComponent } from './data-handling.component'; import { ExperimentStudioService } from '../../../services/experiment-studio.service'; import { ExperimentsDashboardService } from '../../../services/experiments-dashboard.service'; import { ChartBuilderService } from '../visualisations/charts/chart-builder.service'; @@ -15,9 +15,9 @@ import { createSpyObj } from '../../../testing/spy-obj.testing'; * visit or present in the persisted request config. Opening a stage must never * write that config. */ -describe('StatisticAnalysisPanelComponent pipeline presence', () => { - let component: StatisticAnalysisPanelComponent; - let fixture: ComponentFixture; +describe('DataHandlingComponent pipeline presence', () => { + let component: DataHandlingComponent; + let fixture: ComponentFixture; let mockExpService: MockedObject; const age = { code: 'age', label: 'Age', type: 'real' }; @@ -129,7 +129,7 @@ describe('StatisticAnalysisPanelComponent pipeline presence', () => { }); await TestBed.configureTestingModule({ - imports: [StatisticAnalysisPanelComponent], + imports: [DataHandlingComponent], providers: [ provideZonelessChangeDetection(), provideEchartsCore({ echarts: () => import('echarts') }), @@ -140,7 +140,7 @@ describe('StatisticAnalysisPanelComponent pipeline presence', () => { ] }).compileComponents(); - fixture = TestBed.createComponent(StatisticAnalysisPanelComponent); + fixture = TestBed.createComponent(DataHandlingComponent); component = fixture.componentInstance; fixture.detectChanges(); }); diff --git a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.preprocessing.css b/src/app/pages/experiment-studio/data-handling/data-handling.preprocessing.css similarity index 98% rename from src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.preprocessing.css rename to src/app/pages/experiment-studio/data-handling/data-handling.preprocessing.css index edc9a2a19..acf235d7b 100644 --- a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.preprocessing.css +++ b/src/app/pages/experiment-studio/data-handling/data-handling.preprocessing.css @@ -1,13 +1,13 @@ /* Preprocessing stations of the Data review step: station shell, preview bar, glass feedback cards, the documentation panel, the missing/outlier/longitudinal controls and the preprocessing variable browser. - Split from statistic-analysis-panel.component.css to keep every component + Split from data-handling.component.css to keep every component stylesheet inside the anyComponentStyle budget. Angular concatenates the component's styleUrl array IN ORDER, and this file is the second of three — the order below is the cascade, so do not reorder or move rules across files: - 1. statistic-analysis-panel.component.css panel shell + node-pipeline canvas - 2. statistic-analysis-panel.preprocessing.css (this file) - 3. statistic-analysis-panel.results.css shared controls + result surfaces + 1. data-handling.component.css panel shell + node-pipeline canvas + 2. data-handling.preprocessing.css (this file) + 3. data-handling.results.css shared controls + result surfaces Shared/one-off overrides that must win over styles.css live in the file that owns the rule's section, never duplicated. */ diff --git a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.results.css b/src/app/pages/experiment-studio/data-handling/data-handling.results.css similarity index 99% rename from src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.results.css rename to src/app/pages/experiment-studio/data-handling/data-handling.results.css index afc6634f6..c71ac8212 100644 --- a/src/app/pages/experiment-studio/statistic-analysis-panel/statistic-analysis-panel.results.css +++ b/src/app/pages/experiment-studio/data-handling/data-handling.results.css @@ -2,8 +2,8 @@ switch, the federated-data note, the descriptive/data tables, the statistics browser, the chart-export-only containers, the transformation station and the responsive overrides for all three files. - Second split of statistic-analysis-panel.component.css (see - statistic-analysis-panel.preprocessing.css for the reason and the cascade + Second split of data-handling.component.css (see + data-handling.preprocessing.css for the reason and the cascade order). This file is LAST in the styleUrl array, which is why the responsive overrides live here. */ diff --git a/src/app/pages/experiment-studio/statistic-analysis-panel/kmeans-cluster-source/kmeans-cluster-source.component.css b/src/app/pages/experiment-studio/data-handling/kmeans-cluster-source/kmeans-cluster-source.component.css similarity index 100% rename from src/app/pages/experiment-studio/statistic-analysis-panel/kmeans-cluster-source/kmeans-cluster-source.component.css rename to src/app/pages/experiment-studio/data-handling/kmeans-cluster-source/kmeans-cluster-source.component.css diff --git a/src/app/pages/experiment-studio/statistic-analysis-panel/kmeans-cluster-source/kmeans-cluster-source.component.html b/src/app/pages/experiment-studio/data-handling/kmeans-cluster-source/kmeans-cluster-source.component.html similarity index 100% rename from src/app/pages/experiment-studio/statistic-analysis-panel/kmeans-cluster-source/kmeans-cluster-source.component.html rename to src/app/pages/experiment-studio/data-handling/kmeans-cluster-source/kmeans-cluster-source.component.html diff --git a/src/app/pages/experiment-studio/statistic-analysis-panel/kmeans-cluster-source/kmeans-cluster-source.component.spec.ts b/src/app/pages/experiment-studio/data-handling/kmeans-cluster-source/kmeans-cluster-source.component.spec.ts similarity index 100% rename from src/app/pages/experiment-studio/statistic-analysis-panel/kmeans-cluster-source/kmeans-cluster-source.component.spec.ts rename to src/app/pages/experiment-studio/data-handling/kmeans-cluster-source/kmeans-cluster-source.component.spec.ts diff --git a/src/app/pages/experiment-studio/statistic-analysis-panel/kmeans-cluster-source/kmeans-cluster-source.component.ts b/src/app/pages/experiment-studio/data-handling/kmeans-cluster-source/kmeans-cluster-source.component.ts similarity index 100% rename from src/app/pages/experiment-studio/statistic-analysis-panel/kmeans-cluster-source/kmeans-cluster-source.component.ts rename to src/app/pages/experiment-studio/data-handling/kmeans-cluster-source/kmeans-cluster-source.component.ts diff --git a/src/app/pages/experiment-studio/experiment-studio.component.html b/src/app/pages/experiment-studio/experiment-studio.component.html index 44147606f..288d3fd48 100644 --- a/src/app/pages/experiment-studio/experiment-studio.component.html +++ b/src/app/pages/experiment-studio/experiment-studio.component.html @@ -95,7 +95,7 @@ [class.is-visible]="activeSection() === 'statistics-section'" [class.is-complete]="descriptiveProgress().preprocessingStatus === 'applied'" > - + @if (!isDataReviewReady()) {
diff --git a/src/app/pages/experiment-studio/experiment-studio.component.spec.ts b/src/app/pages/experiment-studio/experiment-studio.component.spec.ts index 67fb37f46..343b08194 100644 --- a/src/app/pages/experiment-studio/experiment-studio.component.spec.ts +++ b/src/app/pages/experiment-studio/experiment-studio.component.spec.ts @@ -210,7 +210,7 @@ describe('ExperimentStudioComponent view switching', () => { const sectionOpen = signal>({ raw: false, setup: false, filters: false, processed: false, transformation: false, }); - (component as any).statisticPanel = signal({ goToSection, sectionOpen }); + (component as any).dataHandling = signal({ goToSection, sectionOpen }); component.activateGuideTarget('Analysis', 'analysis-filtering'); expect(goToSection).toHaveBeenCalledWith('filters'); @@ -233,14 +233,14 @@ describe('ExperimentStudioComponent view switching', () => { const sectionOpen = signal>({ raw: false, setup: false, filters: false, processed: false, transformation: false, }); - const statisticPanel = { goToSection, sectionOpen }; - (component as any).statisticPanel = signal(statisticPanel); + const dataHandling = { goToSection, sectionOpen }; + (component as any).dataHandling = signal(dataHandling); vi.spyOn(router, 'navigate').mockResolvedValue(true); component.goToDescriptiveStep('setup'); expect(component.activeSection()).toBe('variables-top'); - expect(statisticPanel.goToSection).not.toHaveBeenCalled(); + expect(dataHandling.goToSection).not.toHaveBeenCalled(); expect(router.navigate).not.toHaveBeenCalled(); }); @@ -249,8 +249,8 @@ describe('ExperimentStudioComponent view switching', () => { const sectionOpen = signal>({ raw: false, setup: false, filters: false, processed: false, transformation: false, }); - const statisticPanel = { goToSection, sectionOpen }; - (component as any).statisticPanel = signal(statisticPanel); + const dataHandling = { goToSection, sectionOpen }; + (component as any).dataHandling = signal(dataHandling); vi.spyOn(router, 'navigate').mockReturnValue(Promise.resolve(true)); vi.spyOn(window, 'requestAnimationFrame').mockImplementation((callback: FrameRequestCallback): number => { queueMicrotask(() => callback(0)); @@ -267,7 +267,7 @@ describe('ExperimentStudioComponent view switching', () => { expect(component.activeSection()).toBe('statistics-section'); await Promise.resolve(); await Promise.resolve(); - expect(statisticPanel.goToSection).toHaveBeenCalledWith('processed'); + expect(dataHandling.goToSection).toHaveBeenCalledWith('processed'); }); describe('stepper navigation service state publication', () => { diff --git a/src/app/pages/experiment-studio/experiment-studio.component.ts b/src/app/pages/experiment-studio/experiment-studio.component.ts index b8f0356ac..f1195e46e 100644 --- a/src/app/pages/experiment-studio/experiment-studio.component.ts +++ b/src/app/pages/experiment-studio/experiment-studio.component.ts @@ -10,8 +10,8 @@ import { ErrorService } from '../../services/error.service'; import { StudioStepperComponent } from './stepper/studio-stepper.component'; import { DescriptiveProgressState, - StatisticAnalysisPanelComponent, -} from './statistic-analysis-panel/statistic-analysis-panel.component'; + DataHandlingComponent, +} from './data-handling/data-handling.component'; import { countFilterRules } from '../../core/filter-display.utils'; import { Subject, takeUntil } from 'rxjs'; import { ExperimentStudioGuideComponent } from './guide/experiment-studio-guide.component'; @@ -37,7 +37,7 @@ type AlgorithmSubstepKey = 'setup' | 'parameters'; VariablesPanelComponent, AlgorithmPanelComponent, ExecutionPanelComponent, - StatisticAnalysisPanelComponent, + DataHandlingComponent, ExperimentStudioGuideComponent, StudioStepperComponent, ], @@ -68,7 +68,7 @@ export class ExperimentStudioComponent implements OnInit, OnDestroy { }); readonly algorithmPanel = viewChild(AlgorithmPanelComponent); readonly runResult = this.expStudioService.runResult; - readonly statisticPanel = viewChild(StatisticAnalysisPanelComponent); + readonly dataHandling = viewChild(DataHandlingComponent); // --- Step views ------------------------------------------------- activeSection = signal('variables-top'); @@ -349,7 +349,7 @@ export class ExperimentStudioComponent implements OnInit, OnDestroy { if (target === 'statistics-section' && stepId) { const section = this.guideStepToSection(stepId); if (section) { - this.statisticPanel()?.goToSection(section); + this.dataHandling()?.goToSection(section); } } }; @@ -363,7 +363,7 @@ export class ExperimentStudioComponent implements OnInit, OnDestroy { } private scrollToDescriptiveStep(section: DescriptiveStep): void { - requestAnimationFrame(() => this.statisticPanel()?.goToSection(section)); + requestAnimationFrame(() => this.dataHandling()?.goToSection(section)); } private runCurrentExperiment(): void { diff --git a/src/app/pages/experiment-studio/variables-panel/variables-panel.component.css b/src/app/pages/experiment-studio/variables-panel/variables-panel.component.css index 9ad8b813f..c16b5eaa9 100644 --- a/src/app/pages/experiment-studio/variables-panel/variables-panel.component.css +++ b/src/app/pages/experiment-studio/variables-panel/variables-panel.component.css @@ -822,7 +822,7 @@ flex: 0 0 auto; } -/* Same pill as the summary toolbar Export in statistic-analysis-panel. */ +/* Same pill as the summary toolbar Export in data-handling. */ .histogram-export-group .studio-export-btn { min-height: 34px; padding: 0 14px; From 6d60d7eb709ea00e69b4a3630152e3490eab5019 Mon Sep 17 00:00:00 2001 From: "K.Filippopolitis" <56073635+KFilippopolitis@users.noreply.github.com> Date: Fri, 23 Oct 2026 22:40:23 +0300 Subject: [PATCH 02/31] refactor(data-handling): move shared types and pure helpers out of the component data-handling.models.ts holds the pipeline types and SUMMARY_GUIDE_ANCHORS; utils/ holds the number formatting, the preprocessing documentation builder and the transformation draft factory, each with a focused spec. No behavior change. --- .../data-handling/data-handling.component.ts | 341 +++--------------- .../data-handling/data-handling.models.ts | 161 +++++++++ .../preprocessing-documentation.utils.spec.ts | 22 ++ .../preprocessing-documentation.utils.ts | 84 +++++ .../utils/statistic-format.utils.spec.ts | 36 ++ .../utils/statistic-format.utils.ts | 43 +++ .../utils/transformation-draft.utils.ts | 13 + .../experiment-studio.component.ts | 6 +- 8 files changed, 413 insertions(+), 293 deletions(-) create mode 100644 src/app/pages/experiment-studio/data-handling/data-handling.models.ts create mode 100644 src/app/pages/experiment-studio/data-handling/utils/preprocessing-documentation.utils.spec.ts create mode 100644 src/app/pages/experiment-studio/data-handling/utils/preprocessing-documentation.utils.ts create mode 100644 src/app/pages/experiment-studio/data-handling/utils/statistic-format.utils.spec.ts create mode 100644 src/app/pages/experiment-studio/data-handling/utils/statistic-format.utils.ts create mode 100644 src/app/pages/experiment-studio/data-handling/utils/transformation-draft.utils.ts diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.ts b/src/app/pages/experiment-studio/data-handling/data-handling.component.ts index 758121fea..6a811e4bc 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.ts +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.ts @@ -20,14 +20,12 @@ import { FormsModule } from '@angular/forms'; import { EChartsOption } from 'echarts'; import { Subject, Subscription, takeUntil } from 'rxjs'; import { ExperimentStudioService, PreprocessingConfig } from '../../../services/experiment-studio.service'; -import { escapeHtml } from '../../../core/html.utils'; import { ChartBuilderService } from '../visualisations/charts/chart-builder.service'; import { ChartRendererComponent } from '../visualisations/charts/charts-renderer/charts-renderer.component'; import { HistogramComponent } from '../visualisations/histogram/histogram.component'; import { DATASET_VARIABLE_CODE, HISTOGRAM_SPLIT_NONE, - HistogramGroupSeries, HistogramSplitOption, buildHistogramSplitOptions, buildStackedHistogramSeries, @@ -45,7 +43,7 @@ import { FilterConfigModalComponent } from '../shared/filter-config-modal/filter import { StationActionBarComponent } from '../shared/station-action-bar/station-action-bar.component'; import { StationCardComponent, StationStatus } from '../shared/station-card/station-card.component'; import { KMeansClusterSourceComponent } from './kmeans-cluster-source/kmeans-cluster-source.component'; -import { BackendFilter, BackendRule } from '../../../models/filters.model'; +import { BackendFilter } from '../../../models/filters.model'; import { CsvExportService } from '../../../services/csv-export.service'; import { ExperimentStudioNavigationService } from '../../../services/experiment-studio-navigation.service'; import { countFilterRules, formatFilterExpression } from '../../../core/filter-display.utils'; @@ -72,176 +70,44 @@ import { validateOutlierRule, } from '../../../core/outlier-rules'; -type TabKey = 'Statistics' | 'Charts' | 'Histogram'; -type SummaryKind = 'source' | 'raw' | 'processed'; -type SectionKey = 'source' | 'raw' | 'setup' | 'filters' | 'processed' | 'transformation'; -type PipelineStep = 'source' | 'filters' | 'setup' | 'transformation'; -type PrepStep = 'missing' | 'outlier' | 'longitudinal'; -type DistributionSubTab = 'Numeric' | 'Nominal'; -type StatisticVariableType = 'numeric' | 'nominal'; -type PreprocessingStatus = 'none' | 'pending' | 'applied'; -type PrepKind = 'missing' | 'outlier' | 'longitudinal'; - -type SummaryAnchorKey = 'charts' | 'details' | 'export'; - -/** - * Where each summary surface wears a `data-guide` anchor. They are not uniform: the Raw - * surface owns the overlay-tab and export anchors, the Processed surface only its details - * panel, and step 0 is a read-only duplicate of the Raw surface, so it wears none — the - * same anchor on a hidden node would hand the tour or a QA selector the wrong surface. - */ -/** Backend step docs arrive as prose whose bullets are '-', '*' or '\u2022'; all of them - become real list items, so the block reads like the algorithm documentation. */ -const DOCUMENTATION_BULLET = /^[-*\u2022\u00b7]\s+/; - -const SUMMARY_GUIDE_ANCHORS: Record> = { - source: { charts: null, details: null, export: null }, - raw: { charts: 'analysis-charts', details: 'analysis-raw-details', export: 'analysis-export' }, - processed: { charts: null, details: 'analysis-processed-details', export: null }, -}; - -type MissingAction = 'no_action' | 'drop' | 'mean' | 'median' | 'constant'; -type LongitudinalStrategy = 'first' | 'second' | 'diff'; - -/** - * The record-level rule behind one category. exaflow takes either a group of rules or a - * single condition, and saved experiments hold both shapes - so the draft keeps whichever - * it was given instead of pretending a bare condition cannot exist. - */ -type CategoryFilter = BackendFilter | BackendRule; - -interface TransformationRule { - value: string; - filter: CategoryFilter | null; -} - -/** One "Create new categorical column" card. The stage holds an ordered list. */ -interface TransformationColumnDraft { - id: number; - code: string; - defaultEnumeration: string; - rules: TransformationRule[]; - open: boolean; - /** Rule index whose filter editor is expanded; at most one open per card. */ - openRuleIndex?: number | null; - /** Picked from the type chooser (or restored from the store). The stage keeps one blank - * draft as its model; until it is chosen it stays off screen and the chooser shows. */ - chosen?: boolean; -} - -/** A summary cell as a number: tolerates grouping commas and a trailing %; null when not numeric. */ -function parseNumber(value: unknown): number | null { - const text = String(value ?? '').replace(/[,%]/g, '').trim(); - const number = Number(text); - return text !== '' && Number.isFinite(number) ? number : null; -} - -let transformationDraftSeq = 0; -function emptyTransformationDraft(): TransformationColumnDraft { - return { id: ++transformationDraftSeq, code: '', defaultEnumeration: '', rules: [], open: true, openRuleIndex: null }; -} - -/** One sub-step of a pipeline stage, rendered on the pipeline-overview rail. */ -interface PipelineSubNode { - // Preprocessing stations use fixed ids; transformation sub-nodes carry a - // `transformation:` id so the rail can list one card per derived column. - id: string; - icon: string; - title: string; - subtitle: string; - statusLabel: string; - statusTone: 'applied' | 'default' | 'pending'; - /** Optional consequence line (e.g. rows the default drop-NaN would remove). */ - impact?: string; -} - -type MetricKey = - | 'num_dtps' | 'num_na' | 'num_total' - | 'mean' | 'std' | 'min' | 'q1' | 'q2' | 'q3' | 'max'; - -interface SummaryView { - activeTab: TabKey; - data: PivotBlock[]; - featurewiseRows: any[]; - isLoading: boolean; - showDistributions: boolean; - distributionSubTab: DistributionSubTab; - nonNominalVariables: VariableRow[]; - nominalVariables: VariableRow[]; - chartsForBoxPlot: EChartsOption[][]; - chartsForNominal: EChartsOption[][]; - histogramDataByVariable: Record; - histogramLoadingByVariable: Record; - histogramErrorByVariable: Record; - histogramSubscriptionsByVariable: Record; - histogramRequestIdByVariable: Record; - histogramSplitByVariable: Record; - activeBoxPlotIndex: number; - activeNominalIndex: number; - selectedStatisticKey: string | null; -} - -interface PivotBlock { - code?: string; - name: string; - columns: string[]; - rows: Array<{ metric: string; values: Record }>; -} - -interface VariableRow { - code: string; - name?: string; - label?: string; - type?: string; - enumerations?: Array<{ code?: string; label?: string; name?: string }>; -} - -interface EnumOption { - code: string; - label: string; -} - -interface HistogramPreviewData { - bins: string[]; - counts: Array; - variableName: string; - variableType?: string; - groups?: HistogramGroupSeries[]; - splitLabel?: string; -} - -interface PreprocessingRule { - variableCode: string; - action: MissingAction; - value: string; - enabled: boolean; -} - -interface PreprocessingGroup { - key: 'pending' | 'applied' | 'default' | 'not-applied'; - title: string; - variables: VariableRow[]; -} - -interface OutlierPreviewRow { - variable: string; - dataset: string; - strategy: string; - tail: string; - fold: string; - lowerBound: string; - upperBound: string; - lowerOutliers: string; - upperOutliers: string; - totalOutliers: string; - outlierPercentage: string; -} - -export interface DescriptiveProgressState { - pendingChangeCount: number; - preprocessingStatus: PreprocessingStatus; - transformationStatusLabel: string; -} +import { + TabKey, + SummaryKind, + SectionKey, + PipelineStep, + PrepStep, + StatisticVariableType, + PreprocessingStatus, + PrepKind, + SummaryAnchorKey, + SUMMARY_GUIDE_ANCHORS, + MissingAction, + LongitudinalStrategy, + CategoryFilter, + TransformationRule, + TransformationColumnDraft, + PipelineSubNode, + MetricKey, + SummaryView, + PivotBlock, + VariableRow, + EnumOption, + HistogramPreviewData, + PreprocessingRule, + PreprocessingGroup, + OutlierPreviewRow, + DescriptiveProgressState, +} from './data-handling.models'; +import { buildPreprocessingDocumentationHtml } from './utils/preprocessing-documentation.utils'; +import { emptyTransformationDraft, nextTransformationDraftId } from './utils/transformation-draft.utils'; +import { + formatInteger, + formatNumber, + formatOutlierBound, + formatOutlierPercentage, + formatPercent, + parseNumber, +} from './utils/statistic-format.utils'; @Component({ selector: 'app-data-handling', @@ -1996,82 +1862,9 @@ export class DataHandlingComponent implements OnDestroy { } formatPreprocessingDocumentationHtml(stepName: string): string { - return this.buildPreprocessingDocumentationHtml(this.preprocessingStepDocumentation(stepName)); + return buildPreprocessingDocumentationHtml(this.preprocessingStepDocumentation(stepName)); } - private buildPreprocessingDocumentationHtml(text: string): string { - const trimmed = text.trim(); - if (!trimmed) { - return ''; - } - - const lines = trimmed.includes('\n') - ? trimmed.split(/\r?\n/) - : trimmed.split(/\s+(?:[-*\u2022\u00b7])\s+(?=[A-Za-z])/); - - const parts: string[] = []; - let listItems: string[] = []; - let isIntroParagraph = true; - - const flushList = (): void => { - if (!listItems.length) { - return; - } - parts.push('
    ' + listItems.join('') + '
'); - listItems = []; - }; - - const pushParagraph = (line: string): void => { - const escaped = escapeHtml(line); - if (line.endsWith(':') && line.length <= 120) { - parts.push('

' + escaped + '

'); - isIntroParagraph = false; - return; - } - - const classes = isIntroParagraph - ? 'preprocessing-doc-paragraph preprocessing-doc-intro' - : 'preprocessing-doc-paragraph'; - parts.push('

' + escaped + '

'); - isIntroParagraph = false; - }; - - for (const rawLine of lines) { - const line = rawLine.trim(); - if (!line) { - continue; - } - - const isBullet = DOCUMENTATION_BULLET.test(line); - let itemText = isBullet ? line.replace(DOCUMENTATION_BULLET, '').trim() : line; - const colonIndex = itemText.indexOf(':'); - const looksLikeListItem = - isBullet || - (colonIndex > 0 && colonIndex < 40 && itemText.slice(colonIndex + 1).trim().length > 0); - - if (looksLikeListItem && colonIndex > 0) { - const term = escapeHtml(itemText.slice(0, colonIndex).trim()); - const description = escapeHtml(itemText.slice(colonIndex + 1).trim()); - // The term keeps its colon: the item is read as one line of prose, not a table. - listItems.push( - '
  • ' + term + ': ' + - '' + description + '
    • ' - ); - continue; - } - - if (isBullet) { - listItems.push('
  • ' + escapeHtml(itemText) + '
    • '); - continue; - } - - flushList(); - pushParagraph(line); - } - - flushList(); - return parts.join(''); - } // ── Preprocessing evidence ──────────────────────────────────────────────── // A preprocessing form states a decision but never priced it. These read the @@ -2158,7 +1951,7 @@ export class DataHandlingComponent implements OnDestroy { } coveragePercent(coverage: { share: number }): string { - return this.formatPercent(coverage.share * 100); + return formatPercent(coverage.share * 100); } /** Bar length for a share scaled up so small shares stay visible (×1000: 5% is a half @@ -3623,13 +3416,13 @@ export class DataHandlingComponent implements OnDestroy { dataset: String(item.dataset ?? '').trim() || '-', strategy: outlierStrategyLabel(String(data['strategy'] ?? '')), tail: outlierTailLabel(String(data['tail'] ?? '')), - fold: this.formatOutlierBound(data['fold']), - lowerBound: this.formatOutlierBound(data['lower_bound']), - upperBound: this.formatOutlierBound(data['upper_bound']), - lowerOutliers: this.formatInteger(data['lower_outlier_count'], '-'), - upperOutliers: this.formatInteger(data['upper_outlier_count'], '-'), - totalOutliers: this.formatInteger(data['total_outlier_count'], '-'), - outlierPercentage: this.formatOutlierPercentage(data['total_outlier_percentage']), + fold: formatOutlierBound(data['fold']), + lowerBound: formatOutlierBound(data['lower_bound']), + upperBound: formatOutlierBound(data['upper_bound']), + lowerOutliers: formatInteger(data['lower_outlier_count'], '-'), + upperOutliers: formatInteger(data['upper_outlier_count'], '-'), + totalOutliers: formatInteger(data['total_outlier_count'], '-'), + outlierPercentage: formatOutlierPercentage(data['total_outlier_percentage']), }; }); } @@ -3639,40 +3432,10 @@ export class DataHandlingComponent implements OnDestroy { return variable ? this.variableLabel(variable) : code; } - /** Canonical (painted + exported) pivot cell: fixed 2 decimals, no grouping. */ - private formatNumber(value: unknown): string { - if (value === null || value === undefined || value === '') return 'N/A'; - const n = typeof value === 'number' ? value : Number(String(value).replace(/,/g, '')); - if (!Number.isFinite(n)) return String(value); - return n.toFixed(2); - } - /** Outlier preview bounds keep the old 6-decimal preview precision. */ - private formatOutlierBound(value: unknown): string { - if (value === null || value === undefined || value === '') return '-'; - const n = typeof value === 'number' ? value : Number(String(value).replace(/,/g, '')); - if (!Number.isFinite(n)) return String(value); - return n.toLocaleString(undefined, { maximumFractionDigits: 6 }); - } - private formatInteger(value: unknown, empty = 'N/A'): string { - if (value === null || value === undefined || value === '') return empty; - const n = typeof value === 'number' ? value : Number(value); - if (!Number.isFinite(n)) return String(value); - return String(Math.round(n)); - } - private formatPercent(pct: number, digits?: number): string { - const d = digits ?? (pct >= 10 || pct === 0 ? 0 : 1); - return `${pct.toFixed(d)}%`; - } - private formatOutlierPercentage(value: unknown): string { - if (value === null || value === undefined || value === '') return '-'; - const num = typeof value === 'number' ? value : parseFloat(String(value)); - if (!Number.isFinite(num)) return '-'; - return this.formatPercent(num, 2); - } /** Copy pending → applied for one card or all of them. */ private commitPendingToApplied(scope: PrepStep | 'all'): void { @@ -4315,7 +4078,7 @@ export class DataHandlingComponent implements OnDestroy { }): TransformationColumnDraft { const rules = creator.rules ?? {}; return { - id: ++transformationDraftSeq, + id: nextTransformationDraftId(), code: creator.code == null ? '' : String(creator.code), defaultEnumeration: creator.default_enumeration == null ? '' : String(creator.default_enumeration), rules: Object.entries(rules).map(([value, filter]) => ({ @@ -4717,7 +4480,7 @@ export class DataHandlingComponent implements OnDestroy { metric.key === 'num_datapoints' ? byDataset[ds]?.num_dtps : metric.key === 'num_missing' ? byDataset[ds]?.num_na : byDataset[ds]?.num_total; - values[ds] = this.formatInteger(raw); + values[ds] = formatInteger(raw); } return { metric: metric.label, values }; }); @@ -4728,7 +4491,7 @@ export class DataHandlingComponent implements OnDestroy { const label = enumMap.get(key) ?? key; const values: Record = {}; for (const ds of datasetOrder) { - values[ds] = this.formatInteger(byDataset[ds]?.counts?.[key]); + values[ds] = formatInteger(byDataset[ds]?.counts?.[key]); } rows.push({ metric: label, values }); }); @@ -4748,7 +4511,7 @@ export class DataHandlingComponent implements OnDestroy { metric.key === 'std' ? byDataset[ds]?.std : metric.key === 'q2' ? byDataset[ds]?.q2 : byDataset[ds]?.[metric.key]; - values[ds] = countKeys.has(metric.key) ? this.formatInteger(raw) : this.formatNumber(raw); + values[ds] = countKeys.has(metric.key) ? formatInteger(raw) : formatNumber(raw); } return { metric: metric.label, values }; }); diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.models.ts b/src/app/pages/experiment-studio/data-handling/data-handling.models.ts new file mode 100644 index 000000000..6f36dabd9 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/data-handling.models.ts @@ -0,0 +1,161 @@ +import { EChartsOption } from 'echarts'; +import { Subscription } from 'rxjs'; +import { BackendFilter, BackendRule } from '../../../models/filters.model'; +import { HistogramGroupSeries } from '../visualisations/histogram/histogram-data'; + +/** Shared types of the Data Handling pipeline and its subcomponents. */ + +export type TabKey = 'Statistics' | 'Charts' | 'Histogram'; +export type SummaryKind = 'source' | 'raw' | 'processed'; +export type SectionKey = 'source' | 'raw' | 'setup' | 'filters' | 'processed' | 'transformation'; +export type PipelineStep = 'source' | 'filters' | 'setup' | 'transformation'; +export type PrepStep = 'missing' | 'outlier' | 'longitudinal'; +export type DistributionSubTab = 'Numeric' | 'Nominal'; +export type StatisticVariableType = 'numeric' | 'nominal'; +export type PreprocessingStatus = 'none' | 'pending' | 'applied'; +export type PrepKind = 'missing' | 'outlier' | 'longitudinal'; + +export type SummaryAnchorKey = 'charts' | 'details' | 'export'; + +/** + * Where each summary surface wears a `data-guide` anchor. They are not uniform: the Raw + * surface owns the overlay-tab and export anchors, the Processed surface only its details + * panel, and step 0 is a read-only duplicate of the Raw surface, so it wears none — the + * same anchor on a hidden node would hand the tour or a QA selector the wrong surface. + */ +export const SUMMARY_GUIDE_ANCHORS: Record> = { + source: { charts: null, details: null, export: null }, + raw: { charts: 'analysis-charts', details: 'analysis-raw-details', export: 'analysis-export' }, + processed: { charts: null, details: 'analysis-processed-details', export: null }, +}; + +export type MissingAction = 'no_action' | 'drop' | 'mean' | 'median' | 'constant'; +export type LongitudinalStrategy = 'first' | 'second' | 'diff'; + +/** + * The record-level rule behind one category. exaflow takes either a group of rules or a + * single condition, and saved experiments hold both shapes - so the draft keeps whichever + * it was given instead of pretending a bare condition cannot exist. + */ +export type CategoryFilter = BackendFilter | BackendRule; + +export interface TransformationRule { + value: string; + filter: CategoryFilter | null; +} + +/** One "Create new categorical column" card. The stage holds an ordered list. */ +export interface TransformationColumnDraft { + id: number; + code: string; + defaultEnumeration: string; + rules: TransformationRule[]; + open: boolean; + /** Rule index whose filter editor is expanded; at most one open per card. */ + openRuleIndex?: number | null; + /** Picked from the type chooser (or restored from the store). The stage keeps one blank + * draft as its model; until it is chosen it stays off screen and the chooser shows. */ + chosen?: boolean; +} + +/** One sub-step of a pipeline stage, rendered on the pipeline-overview rail. */ +export interface PipelineSubNode { + // Preprocessing stations use fixed ids; transformation sub-nodes carry a + // `transformation:` id so the rail can list one card per derived column. + id: string; + icon: string; + title: string; + subtitle: string; + statusLabel: string; + statusTone: 'applied' | 'default' | 'pending'; + /** Optional consequence line (e.g. rows the default drop-NaN would remove). */ + impact?: string; +} + +export type MetricKey = + | 'num_dtps' | 'num_na' | 'num_total' + | 'mean' | 'std' | 'min' | 'q1' | 'q2' | 'q3' | 'max'; + +export interface SummaryView { + activeTab: TabKey; + data: PivotBlock[]; + featurewiseRows: any[]; + isLoading: boolean; + showDistributions: boolean; + distributionSubTab: DistributionSubTab; + nonNominalVariables: VariableRow[]; + nominalVariables: VariableRow[]; + chartsForBoxPlot: EChartsOption[][]; + chartsForNominal: EChartsOption[][]; + histogramDataByVariable: Record; + histogramLoadingByVariable: Record; + histogramErrorByVariable: Record; + histogramSubscriptionsByVariable: Record; + histogramRequestIdByVariable: Record; + histogramSplitByVariable: Record; + activeBoxPlotIndex: number; + activeNominalIndex: number; + selectedStatisticKey: string | null; +} + +export interface PivotBlock { + code?: string; + name: string; + columns: string[]; + rows: Array<{ metric: string; values: Record }>; +} + +export interface VariableRow { + code: string; + name?: string; + label?: string; + type?: string; + enumerations?: Array<{ code?: string; label?: string; name?: string }>; +} + +export interface EnumOption { + code: string; + label: string; +} + +export interface HistogramPreviewData { + bins: string[]; + counts: Array; + variableName: string; + variableType?: string; + groups?: HistogramGroupSeries[]; + splitLabel?: string; +} + +export interface PreprocessingRule { + variableCode: string; + action: MissingAction; + value: string; + enabled: boolean; +} + +export interface PreprocessingGroup { + key: 'pending' | 'applied' | 'default' | 'not-applied'; + title: string; + variables: VariableRow[]; +} + +export interface OutlierPreviewRow { + variable: string; + dataset: string; + strategy: string; + tail: string; + fold: string; + lowerBound: string; + upperBound: string; + lowerOutliers: string; + upperOutliers: string; + totalOutliers: string; + outlierPercentage: string; +} + +export interface DescriptiveProgressState { + pendingChangeCount: number; + preprocessingStatus: PreprocessingStatus; + transformationStatusLabel: string; +} diff --git a/src/app/pages/experiment-studio/data-handling/utils/preprocessing-documentation.utils.spec.ts b/src/app/pages/experiment-studio/data-handling/utils/preprocessing-documentation.utils.spec.ts new file mode 100644 index 000000000..014da7538 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/utils/preprocessing-documentation.utils.spec.ts @@ -0,0 +1,22 @@ +import { buildPreprocessingDocumentationHtml } from './preprocessing-documentation.utils'; + +describe('buildPreprocessingDocumentationHtml', () => { + it('returns nothing for empty documentation', () => { + expect(buildPreprocessingDocumentationHtml(' ')).toBe(''); + }); + + it('turns intro, section titles and term bullets into paragraphs and a list', () => { + const html = buildPreprocessingDocumentationHtml('Caps values.\nStrategies:\n- Fold: how far out\n- plain bullet'); + expect(html).toBe( + '

    Caps values.

    ' + + '

    Strategies:

    ' + + '
    • Fold: ' + + 'how far out
      • ' + + '
    • plain bullet
    ', + ); + }); + + it('escapes backend text before it reaches innerHTML', () => { + expect(buildPreprocessingDocumentationHtml('x')).toContain('<b>x</b>'); + }); +}); diff --git a/src/app/pages/experiment-studio/data-handling/utils/preprocessing-documentation.utils.ts b/src/app/pages/experiment-studio/data-handling/utils/preprocessing-documentation.utils.ts new file mode 100644 index 000000000..2f4d0a478 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/utils/preprocessing-documentation.utils.ts @@ -0,0 +1,84 @@ +import { escapeHtml } from '../../../../core/html.utils'; + +/** Backend step docs arrive as prose whose bullets are '-', '*' or '\u2022'; all of them + become real list items, so the block reads like the algorithm documentation. */ +const DOCUMENTATION_BULLET = /^[-*\u2022\u00b7]\s+/; + +/** + * The "How this works" panel of a preprocessing card: backend prose turned into + * paragraphs, section titles and term/description list items. Every piece of text is + * escaped, so the result is safe to bind with `[innerHTML]`. + */ +export function buildPreprocessingDocumentationHtml(text: string): string { + const trimmed = text.trim(); + if (!trimmed) { + return ''; + } + + const lines = trimmed.includes('\n') + ? trimmed.split(/\r?\n/) + : trimmed.split(/\s+(?:[-*\u2022\u00b7])\s+(?=[A-Za-z])/); + + const parts: string[] = []; + let listItems: string[] = []; + let isIntroParagraph = true; + + const flushList = (): void => { + if (!listItems.length) { + return; + } + parts.push('
      ' + listItems.join('') + '
    '); + listItems = []; + }; + + const pushParagraph = (line: string): void => { + const escaped = escapeHtml(line); + if (line.endsWith(':') && line.length <= 120) { + parts.push('

    ' + escaped + '

    '); + isIntroParagraph = false; + return; + } + + const classes = isIntroParagraph + ? 'preprocessing-doc-paragraph preprocessing-doc-intro' + : 'preprocessing-doc-paragraph'; + parts.push('

    ' + escaped + '

    '); + isIntroParagraph = false; + }; + + for (const rawLine of lines) { + const line = rawLine.trim(); + if (!line) { + continue; + } + + const isBullet = DOCUMENTATION_BULLET.test(line); + let itemText = isBullet ? line.replace(DOCUMENTATION_BULLET, '').trim() : line; + const colonIndex = itemText.indexOf(':'); + const looksLikeListItem = + isBullet || + (colonIndex > 0 && colonIndex < 40 && itemText.slice(colonIndex + 1).trim().length > 0); + + if (looksLikeListItem && colonIndex > 0) { + const term = escapeHtml(itemText.slice(0, colonIndex).trim()); + const description = escapeHtml(itemText.slice(colonIndex + 1).trim()); + // The term keeps its colon: the item is read as one line of prose, not a table. + listItems.push( + '
  • ' + term + ': ' + + '' + description + '
    • ' + ); + continue; + } + + if (isBullet) { + listItems.push('
  • ' + escapeHtml(itemText) + '
    • '); + continue; + } + + flushList(); + pushParagraph(line); + } + + flushList(); + return parts.join(''); +} diff --git a/src/app/pages/experiment-studio/data-handling/utils/statistic-format.utils.spec.ts b/src/app/pages/experiment-studio/data-handling/utils/statistic-format.utils.spec.ts new file mode 100644 index 000000000..099d0a3b2 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/utils/statistic-format.utils.spec.ts @@ -0,0 +1,36 @@ +import { + formatInteger, + formatNumber, + formatOutlierBound, + formatOutlierPercentage, + formatPercent, + parseNumber, +} from './statistic-format.utils'; + +describe('statistic-format utils', () => { + it('parses summary cells with grouping commas and a trailing percent', () => { + expect(parseNumber('1,234.5')).toBe(1234.5); + expect(parseNumber('12%')).toBe(12); + expect(parseNumber('')).toBeNull(); + expect(parseNumber('N/A')).toBeNull(); + expect(parseNumber(null)).toBeNull(); + }); + + it('formats canonical pivot cells with two decimals and no grouping', () => { + expect(formatNumber(1234.5)).toBe('1234.50'); + expect(formatNumber('1,234.5')).toBe('1234.50'); + expect(formatNumber('')).toBe('N/A'); + expect(formatNumber('abc')).toBe('abc'); + }); + + it('formats integers, bounds and percentages with their own empty markers', () => { + expect(formatInteger(4.6)).toBe('5'); + expect(formatInteger(undefined, '-')).toBe('-'); + expect(formatOutlierBound(null)).toBe('-'); + expect(formatPercent(5)).toBe('5.0%'); + expect(formatPercent(0)).toBe('0%'); + expect(formatPercent(42.4)).toBe('42%'); + expect(formatOutlierPercentage('3.14159')).toBe('3.14%'); + expect(formatOutlierPercentage('x')).toBe('-'); + }); +}); diff --git a/src/app/pages/experiment-studio/data-handling/utils/statistic-format.utils.ts b/src/app/pages/experiment-studio/data-handling/utils/statistic-format.utils.ts new file mode 100644 index 000000000..7b4b7535f --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/utils/statistic-format.utils.ts @@ -0,0 +1,43 @@ +/** Pure number formatting shared by the Data Handling summaries, evidence panes and exports. */ + +/** A summary cell as a number: tolerates grouping commas and a trailing %; null when not numeric. */ +export function parseNumber(value: unknown): number | null { + const text = String(value ?? '').replace(/[,%]/g, '').trim(); + const number = Number(text); + return text !== '' && Number.isFinite(number) ? number : null; +} + +/** Canonical (painted + exported) pivot cell: fixed 2 decimals, no grouping. */ +export function formatNumber(value: unknown): string { + if (value === null || value === undefined || value === '') return 'N/A'; + const n = typeof value === 'number' ? value : Number(String(value).replace(/,/g, '')); + if (!Number.isFinite(n)) return String(value); + return n.toFixed(2); +} + +/** Outlier preview bounds keep the old 6-decimal preview precision. */ +export function formatOutlierBound(value: unknown): string { + if (value === null || value === undefined || value === '') return '-'; + const n = typeof value === 'number' ? value : Number(String(value).replace(/,/g, '')); + if (!Number.isFinite(n)) return String(value); + return n.toLocaleString(undefined, { maximumFractionDigits: 6 }); +} + +export function formatInteger(value: unknown, empty = 'N/A'): string { + if (value === null || value === undefined || value === '') return empty; + const n = typeof value === 'number' ? value : Number(value); + if (!Number.isFinite(n)) return String(value); + return String(Math.round(n)); +} + +export function formatPercent(pct: number, digits?: number): string { + const d = digits ?? (pct >= 10 || pct === 0 ? 0 : 1); + return `${pct.toFixed(d)}%`; +} + +export function formatOutlierPercentage(value: unknown): string { + if (value === null || value === undefined || value === '') return '-'; + const num = typeof value === 'number' ? value : parseFloat(String(value)); + if (!Number.isFinite(num)) return '-'; + return formatPercent(num, 2); +} diff --git a/src/app/pages/experiment-studio/data-handling/utils/transformation-draft.utils.ts b/src/app/pages/experiment-studio/data-handling/utils/transformation-draft.utils.ts new file mode 100644 index 000000000..2cf97eabf --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/utils/transformation-draft.utils.ts @@ -0,0 +1,13 @@ +import { TransformationColumnDraft } from '../data-handling.models'; + +let transformationDraftSeq = 0; + +/** Ids are unique per page load; they key the stage's cards and their template tracking. */ +export function nextTransformationDraftId(): number { + return ++transformationDraftSeq; +} + +/** A blank "Create new categorical column" card with a fresh id, open on its editor. */ +export function emptyTransformationDraft(): TransformationColumnDraft { + return { id: nextTransformationDraftId(), code: '', defaultEnumeration: '', rules: [], open: true, openRuleIndex: null }; +} diff --git a/src/app/pages/experiment-studio/experiment-studio.component.ts b/src/app/pages/experiment-studio/experiment-studio.component.ts index f1195e46e..6f3a6a2d5 100644 --- a/src/app/pages/experiment-studio/experiment-studio.component.ts +++ b/src/app/pages/experiment-studio/experiment-studio.component.ts @@ -8,10 +8,8 @@ import { AuthService } from '../../services/auth.service'; import { ExperimentsDashboardService } from '../../services/experiments-dashboard.service'; import { ErrorService } from '../../services/error.service'; import { StudioStepperComponent } from './stepper/studio-stepper.component'; -import { - DescriptiveProgressState, - DataHandlingComponent, -} from './data-handling/data-handling.component'; +import { DataHandlingComponent } from './data-handling/data-handling.component'; +import { DescriptiveProgressState } from './data-handling/data-handling.models'; import { countFilterRules } from '../../core/filter-display.utils'; import { Subject, takeUntil } from 'rxjs'; import { ExperimentStudioGuideComponent } from './guide/experiment-studio-guide.component'; From 42f7a7abe0ea6614a5a6586faddfc9ba2a14e5e9 Mon Sep 17 00:00:00 2001 From: "K.Filippopolitis" <56073635+KFilippopolitis@users.noreply.github.com> Date: Fri, 23 Oct 2026 22:46:06 +0300 Subject: [PATCH 03/31] refactor(data-handling): extract the "How to use" strip into HowToStripComponent details[app-how-to-strip] replaces the #howToStrip ng-template. It attaches to the
    element itself, so the DOM keeps the same shape and parent rules such as `.workflow-section-body.open > *` still reach it. Its CSS moved with its markup; the station-card margin uses :host-context. No behavior or visual change. --- .../data-handling/data-handling.component.css | 105 ----------------- .../data-handling.component.html | 28 +---- .../data-handling/data-handling.component.ts | 6 +- .../data-handling/data-handling.models.ts | 5 + .../how-to-strip/how-to-strip.component.css | 110 ++++++++++++++++++ .../how-to-strip/how-to-strip.component.html | 11 ++ .../how-to-strip/how-to-strip.component.ts | 19 +++ 7 files changed, 154 insertions(+), 130 deletions(-) create mode 100644 src/app/pages/experiment-studio/data-handling/how-to-strip/how-to-strip.component.css create mode 100644 src/app/pages/experiment-studio/data-handling/how-to-strip/how-to-strip.component.html create mode 100644 src/app/pages/experiment-studio/data-handling/how-to-strip/how-to-strip.component.ts diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.css b/src/app/pages/experiment-studio/data-handling/data-handling.component.css index d747ae42c..4db3f47b1 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.css +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.css @@ -847,7 +847,6 @@ } } - .workflow-section { background: transparent; border: none; @@ -948,104 +947,6 @@ font-size: 0.62rem; } -/* "How to use" strip directly under an expanded stage header; a
    folded by - default, so only the summary line shows until someone asks for the steps. */ -.how-to-strip { - background: color-mix(in srgb, var(--primary-color) 4%, white); - border-bottom: 1px solid rgba(43, 51, 233, 0.08); -} - -.how-to-summary { - display: flex; - align-items: center; - gap: 10px; - padding: 8px 18px; - list-style: none; - cursor: pointer; -} - -.how-to-summary::-webkit-details-marker { - display: none; -} - -.how-to-summary:hover .how-to-peek { - color: var(--primary-color); -} - -.how-to-summary:focus-visible { - outline: 2px solid var(--primary-color); - outline-offset: -2px; -} - -.how-to-label { - font-size: 0.66rem; - font-weight: 700; - letter-spacing: 0.06em; - text-transform: uppercase; - color: var(--primary-color); -} - -.how-to-peek { - font-size: 0.74rem; - color: var(--text-muted); -} - -.how-to-chevron { - font-size: 0.6rem; - color: var(--primary-color); - transition: transform 0.15s ease; -} - -.how-to-strip[open] .how-to-chevron { - transform: rotate(180deg); -} - -.how-to-strip[open] .how-to-peek { - display: none; -} - -.how-to-body { - display: flex; - flex-wrap: wrap; - align-items: center; - gap: 8px 14px; - padding: 0 18px 10px; -} - -.how-to-step { - display: inline-flex; - align-items: center; - gap: 7px; - font-size: 0.78rem; - color: #334155; -} - -.how-to-number { - display: inline-flex; - align-items: center; - justify-content: center; - flex: none; - width: 18px; - height: 18px; - border-radius: 50%; - background: var(--primary-color); - color: #fff; - font-size: 0.68rem; - font-weight: 700; -} - -.how-to-note { - flex-basis: 100%; - font-size: 0.76rem; - color: var(--text-muted); -} - -/* Projected into a station card, the strip spans the card body edge to edge. - `app-station-card` is the host this template owns, so emulated styles reach it. */ -app-station-card .how-to-strip { - margin: -14px -14px 14px; -} - /* Live match chip under the cohort builder, filled by Preview. */ .filter-match-row { display: flex; @@ -1065,12 +966,6 @@ app-station-card .how-to-strip { background: color-mix(in srgb, var(--success) 12%, white); } -@media (max-width: 768px) { - app-station-card .how-to-strip { - margin: -14px -12px 14px; - } -} - @media (max-width: 720px) { .pipeline-records-out { display: none; diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.html b/src/app/pages/experiment-studio/data-handling/data-handling.component.html index 620ebca38..9e222bf45 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.html +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.html @@ -313,24 +313,6 @@

    {{ selectedBlock.name }} @if (selectedBlock.code) { - -
    - - How to use - {{ guide.steps.length }} steps - - -
    - @for (step of guide.steps; track step; let i = $index) { - {{ i + 1 }}{{ step }} - } - {{ guide.note }} -
    -
    -     - @@ -466,7 +448,7 @@

    - - -

    - - } -
    - The next run will use -
    - @if (cohortRetention; as r) { - {{ r.processed.toLocaleString() }} of {{ r.raw.toLocaleString() }} records - } - {{ preprocessingVariables.length }} variables · {{ expStudioService.selectedDatasets().length }} datasets - {{ terminalPreprocessingLine }} -
    -
    - - +
    diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.ts b/src/app/pages/experiment-studio/data-handling/data-handling.component.ts index 693960ea7..85f13822e 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.ts +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.ts @@ -44,6 +44,7 @@ import { StationActionBarComponent } from '../shared/station-action-bar/station- import { StationCardComponent, StationStatus } from '../shared/station-card/station-card.component'; import { KMeansClusterSourceComponent } from './kmeans-cluster-source/kmeans-cluster-source.component'; import { HowToStripComponent } from './how-to-strip/how-to-strip.component'; +import { PipelineTerminalComponent } from './pipeline-terminal/pipeline-terminal.component'; import { BackendFilter } from '../../../models/filters.model'; import { CsvExportService } from '../../../services/csv-export.service'; import { ExperimentStudioNavigationService } from '../../../services/experiment-studio-navigation.service'; @@ -113,7 +114,7 @@ import { @Component({ selector: 'app-data-handling', - imports: [ChartRendererComponent, HistogramComponent, FormsModule, NgTemplateOutlet, FilterConfigModalComponent, StationActionBarComponent, StationCardComponent, KMeansClusterSourceComponent, HowToStripComponent], + imports: [ChartRendererComponent, HistogramComponent, FormsModule, NgTemplateOutlet, FilterConfigModalComponent, StationActionBarComponent, StationCardComponent, KMeansClusterSourceComponent, HowToStripComponent, PipelineTerminalComponent], templateUrl: './data-handling.component.html', // Order is the cascade: shell + pipeline canvas, then preprocessing stations, // then shared controls / result surfaces + responsive overrides. Concatenated in this order. diff --git a/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.css b/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.css new file mode 100644 index 000000000..a416d6bc1 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.css @@ -0,0 +1,157 @@ +/* The panel nudges every icon 1px down (`.stats-panel-shell i`); the rule is + scoped to the panel's own template, so it is repeated here for this one. */ +:host-context(.stats-panel-shell) i { + transform: translateY(1px); +} + +/* Centred and capped like a collapsed pipeline step (`.pipeline-node.is-compact`). */ +:host(.pipeline-terminal-card) { + align-self: center; + width: 100%; + max-width: var(--studio-pipeline-overview-max, 860px); +} + +/* Terminal Card: this component's host. */ +:host(.pipeline-terminal-card) { + display: grid; + grid-template-columns: minmax(0, 1fr) auto; + align-items: center; + padding: 16px 20px; + background: var(--studio-card-header-bg); + border: 1px solid var(--border-color, rgba(43, 51, 233, 0.15)); + border-radius: var(--radius-lg, 12px); + box-shadow: none; + gap: 20px; + margin-top: 4px; +} + +.pipeline-terminal-summary { + display: flex; + flex-direction: column; + gap: 8px; + min-width: 0; +} + +.pipeline-terminal-kicker { + color: var(--text-muted); + font-size: 0.72rem; + font-weight: 700; + letter-spacing: var(--letter-spacing-config); + text-transform: uppercase; +} + +.pipeline-terminal-facts { + display: flex; + flex-wrap: wrap; + align-items: baseline; + gap: 6px 22px; + font-size: 0.84rem; + color: var(--text-muted); +} + +.pipeline-terminal-facts strong { + font-size: 1.05rem; + font-weight: 700; + color: var(--text-main); +} + +.btn-finish-pipeline { + display: inline-flex; + align-items: center; + gap: 8px; + padding: 10px 22px; + background: var(--primary-color); + color: #ffffff; + border: none; + border-radius: 999px; + font-family: inherit; + font-size: 0.86rem; + font-weight: 600; + cursor: pointer; + box-shadow: var(--shadow-sm); + transition: all 0.2s ease; + white-space: nowrap; +} + +/* Unapplied-changes guard */ +.unapplied-warning-card { + grid-column: 1 / -1; + flex: 1 1 100%; + display: flex; + align-items: center; + gap: 14px; + padding: 12px 16px; + background: color-mix(in srgb, #f59e0b 8%, white); + border: 1px solid color-mix(in srgb, #f59e0b 30%, transparent); + border-radius: var(--radius-md, 8px); +} + +.unapplied-warning-icon { + color: #78350f; + font-size: 1.1rem; + flex-shrink: 0; +} + +.unapplied-warning-text h4 { + font-size: 0.88rem; + font-weight: 700; + color: var(--text-main); + margin: 0 0 2px; +} + +.unapplied-warning-text p { + font-size: 0.78rem; + color: var(--text-muted); + margin: 0; +} + +.unapplied-warning-actions { + margin-left: auto; + display: flex; + gap: 8px; + align-items: center; +} + +/* Secondary and ghost siblings of the primary pill: same shape, no fill. The frame is + an inset shadow so the box stays exactly as tall as .btn-finish-pipeline (border: none). */ +.unapplied-warning-btn.secondary { + background: var(--card-bg); + color: var(--primary-color); + box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--primary-color) 35%, transparent); +} + +.unapplied-warning-btn.ghost { + background: transparent; + color: var(--text-muted); + box-shadow: none; +} + +.unapplied-warning-btn.secondary:hover, +.unapplied-warning-btn.ghost:hover { + transform: none; +} + +.unapplied-warning-btn.secondary:hover { + background: color-mix(in srgb, var(--primary-color) 8%, white); +} + +.unapplied-warning-btn.ghost:hover { + background: transparent; + color: var(--text-main); +} + +.btn-finish-pipeline:hover { + background: var(--primary-dark, #1b21a3); + transform: none; + box-shadow: var(--shadow-md); +} + +@media (max-width: 768px) { + :host(.pipeline-terminal-card) { + grid-template-columns: minmax(0, 1fr); + align-items: stretch; + } + .btn-finish-pipeline { + justify-content: center; + } +} diff --git a/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.html b/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.html new file mode 100644 index 000000000..4763de382 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.html @@ -0,0 +1,35 @@ +@if (showUnappliedWarning()) { + +} +
    + The next run will use +
    + @if (retention(); as r) { + {{ r.processed.toLocaleString() }} of {{ r.raw.toLocaleString() }} records + } + {{ variableCount() }} variables · {{ datasetCount() }} datasets + {{ preprocessingLine() }} +
    +
    + diff --git a/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.spec.ts b/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.spec.ts new file mode 100644 index 000000000..214dbeb31 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.spec.ts @@ -0,0 +1,58 @@ +import { Component, signal } from '@angular/core'; +import { TestBed } from '@angular/core/testing'; +import { PipelineTerminalComponent } from './pipeline-terminal.component'; + +@Component({ + imports: [PipelineTerminalComponent], + template: `
    `, +}) +class HostComponent { + readonly retention = signal<{ raw: number; processed: number } | null>({ raw: 1200, processed: 1000 }); + readonly warn = signal(false); + readonly log: string[] = []; +} + +describe('PipelineTerminalComponent', () => { + async function render() { + const fixture = TestBed.createComponent(HostComponent); + await fixture.whenStable(); + return { fixture, host: fixture.componentInstance, el: fixture.nativeElement as HTMLElement }; + } + + it('renders on the card element itself with the run summary', async () => { + const { el } = await render(); + const card = el.querySelector('div.pipeline-terminal-card')!; + expect(card).toBeTruthy(); + expect(card.textContent).toContain('1,000 of 1,200 records'); + expect(card.textContent).toContain('3 variables · 2 datasets'); + expect(card.textContent).toContain('No preprocessing'); + expect(card.querySelector('.unapplied-warning-card')).toBeNull(); + }); + + it('hides the records line until retention is known', async () => { + const { fixture, host, el } = await render(); + host.retention.set(null); + await fixture.whenStable(); + expect(el.textContent).not.toContain('records'); + }); + + it('emits continue from the hand-off button', async () => { + const { host, el } = await render(); + (el.querySelector('.pipeline-terminal-card > button.btn-finish-pipeline') as HTMLButtonElement).click(); + expect(host.log).toEqual(['continue']); + }); + + it('shows the unapplied-changes guard and emits each choice', async () => { + const { fixture, host, el } = await render(); + host.warn.set(true); + await fixture.whenStable(); + const buttons = el.querySelectorAll('.unapplied-warning-actions button'); + expect(el.querySelector('[role="alert"]')?.textContent).toContain('You have unapplied changes'); + buttons.forEach((b) => b.click()); + expect(host.log).toEqual(['apply', 'discard', 'stay']); + }); +}); diff --git a/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.ts b/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.ts new file mode 100644 index 000000000..3e210333e --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/pipeline-terminal/pipeline-terminal.component.ts @@ -0,0 +1,27 @@ +import { ChangeDetectionStrategy, Component, input, output } from '@angular/core'; + +/** + * End of the Data Handling pipeline: what the next run will use, the hand-off to + * Algorithm Selection, and the guard shown when edits are still unapplied. + * Attached to the card element itself, so the pipeline column lays it out as before. + */ +@Component({ + selector: 'div[app-pipeline-terminal]', + host: { class: 'pipeline-terminal-card' }, + templateUrl: './pipeline-terminal.component.html', + styleUrl: './pipeline-terminal.component.css', + changeDetection: ChangeDetectionStrategy.OnPush, +}) +export class PipelineTerminalComponent { + /** Records kept after preprocessing out of the raw cohort; null until both are known. */ + readonly retention = input<{ raw: number; processed: number } | null>(null); + readonly variableCount = input(0); + readonly datasetCount = input(0); + readonly preprocessingLine = input(''); + readonly showUnappliedWarning = input(false); + + readonly continue = output(); + readonly applyAndContinue = output(); + readonly discardAndContinue = output(); + readonly stay = output(); +} From 6ac7665bda883af0521de18918a9cd32bbde1029 Mon Sep 17 00:00:00 2001 From: "K.Filippopolitis" <56073635+KFilippopolitis@users.noreply.github.com> Date: Fri, 23 Oct 2026 23:08:18 +0300 Subject: [PATCH 05/31] refactor(data-handling): extract the summary workspace into SummaryWorkspaceComponent The variable browser / statistics / charts / histogram / export workspace that the Raw data snapshot, the Raw preview and the Processed preview all render moves from an ng-template outlet to data-handling/summary-workspace/. It attaches to each section's body element (div.workflow-section-body[app-summary-workspace]), so the DOM, the studio reveal animation, data-guide anchors and spec selectors are unchanged. The workspace's state stays on DataHandlingComponent; sections reach it through the new DATA_HANDLING token (data-handling.context.ts) and use Eager change detection, so they refresh exactly when the inline template did. Cleanups in the moved code: - the duplicated Numerical / Categorical variable groups and the three tab buttons are rendered from typed constants (VARIABLE_GROUPS, SUMMARY_TABS); - coverage is computed once per variable button (@let) instead of twice; - the skeleton grid uses named cell arrays instead of inline number literals; - createEmptySummary becomes the pure utils/summary-view.utils emptySummaryView. CSS: rules for elements that only live in the workspace moved with it; shared rules (tables, empty states, search, button:disabled) are repeated in the panel's cascade order. The node-frame rule becomes :host-context(.pipeline-node). Computed styles and screenshots match master in all harness scenarios. Specs: SummaryWorkspaceComponent (10) and emptySummaryView (2). --- .../data-handling/data-handling.component.css | 108 ---- .../data-handling.component.html | 350 +---------- .../data-handling/data-handling.component.ts | 64 +- .../data-handling/data-handling.context.ts | 13 + .../data-handling.pipeline.spec.ts | 3 +- .../data-handling.preprocessing.css | 107 ---- .../data-handling/data-handling.results.css | 192 +----- .../summary-workspace.component.css | 594 ++++++++++++++++++ .../summary-workspace.component.html | 271 ++++++++ .../summary-workspace.component.spec.ts | 143 +++++ .../summary-workspace.component.ts | 53 ++ .../utils/summary-view.utils.spec.ts | 18 + .../data-handling/utils/summary-view.utils.ts | 26 + 13 files changed, 1169 insertions(+), 773 deletions(-) create mode 100644 src/app/pages/experiment-studio/data-handling/data-handling.context.ts create mode 100644 src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.css create mode 100644 src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.html create mode 100644 src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.spec.ts create mode 100644 src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.ts create mode 100644 src/app/pages/experiment-studio/data-handling/utils/summary-view.utils.spec.ts create mode 100644 src/app/pages/experiment-studio/data-handling/utils/summary-view.utils.ts diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.css b/src/app/pages/experiment-studio/data-handling/data-handling.component.css index 38969f625..d76e12df5 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.css +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.css @@ -29,92 +29,6 @@ transform: translateY(1px); } -/* One toolbar row over the results pane: variable identity on the left, the - Table / Charts / Histogram tray and one Export menu on the right. */ -.summary-detail-toolbar { - display: flex; - align-items: center; - justify-content: space-between; - gap: 12px; - padding: 14px 20px; - border-bottom: 1px solid var(--studio-card-header-border); -} - -.summary-detail-actions { - display: flex; - align-items: center; - flex-wrap: wrap; - gap: 10px; - flex-shrink: 0; -} - -.histogram-split { - display: inline-flex; - align-items: center; - gap: 8px; - min-height: 34px; - color: var(--text-muted); - font-size: 0.78rem; - font-weight: 600; - letter-spacing: 0.01em; -} - -.histogram-split select { - appearance: none; - -webkit-appearance: none; - width: auto; - min-height: 34px; - max-width: 180px; - padding: 0 34px 0 14px; - border: 0; - border-radius: 999px; - background-color: var(--card-bg); - background-image: - linear-gradient(45deg, transparent 50%, var(--primary-color) 50%), - linear-gradient(135deg, var(--primary-color) 50%, transparent 50%); - background-position: - calc(100% - 17px) 50%, - calc(100% - 12px) 50%; - background-size: 5px 5px, 5px 5px; - background-repeat: no-repeat; - box-shadow: inset 0 0 0 1px var(--border-color), var(--shadow-sm); - color: var(--text-main); - font: inherit; - font-weight: 600; -} - -.histogram-split select:focus-visible { - outline: 2px solid var(--primary-color); - outline-offset: 2px; -} - -/* Beside the .modal-tabs tray, Export takes the tray's geometry (34px pill, - 1px inset edge, 0.78rem type) and reads like its active pill. */ -.summary-detail-actions .studio-export-btn { - min-height: 34px; - padding: 0 14px; - border: 0; - border-radius: 999px; - background: var(--card-bg); - box-shadow: inset 0 0 0 1px var(--border-color), var(--shadow-sm); - font-size: 0.78rem; - letter-spacing: 0.01em; -} - -.summary-detail-actions .studio-export-btn:hover:not(:disabled), -.summary-detail-actions .studio-export-btn[aria-expanded="true"]:not(:disabled) { - background: color-mix(in srgb, var(--primary-color) 6%, var(--card-bg)); - box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--primary-color) 35%, transparent), var(--shadow-sm); -} - -.statistics-panel-heading h4 { - display: flex; - align-items: center; - gap: 8px; - margin: 2px 0 0; - font-size: 1rem; -} - .statistics-panel-code { padding: 1px 6px; border-radius: var(--radius-sm); @@ -125,20 +39,6 @@ font-weight: 500; } -.summary-export-group { - position: relative; - display: inline-flex; -} - -.summary-export-group .summary-export-caret { - font-size: 0.58rem; - transition: transform 0.18s ease; -} - -.summary-export-group .summary-export-caret.open { - transform: rotate(180deg); -} - .statistics-panel { padding: 16px 20px 20px; } @@ -729,14 +629,6 @@ padding: 0; } -/* Summary workspaces nested inside a pipeline node inherit the node's frame; - strip the inner card chrome so only one border is visible. */ -.pipeline-node .workflow-section .statistics-workspace { - border: none; - border-radius: 0; - box-shadow: none; -} - /* Stage header extras: the step kicker above the title and the "Records out" figure before the status badge. */ .pipeline-node-kicker { diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.html b/src/app/pages/experiment-studio/data-handling/data-handling.component.html index 82ec42aca..2d3c51545 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.html +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.html @@ -8,311 +8,6 @@

    Data Handling Pipeline

    - - - @if (kind === 'processed' && cohortRetention; as retention) { -
    - - - Cohort retention: {{ retention.processed.toLocaleString() }} / {{ retention.raw.toLocaleString() }} records - @if (retention.dropped > 0) { - ({{ retention.dropped.toLocaleString() }} rows removed by preprocessing) - } - · {{ retention.percentage.toFixed(1) }}% retained - -
    - } - - @if (kind !== 'source') { -
    - -
    - } - - @if (summary.isLoading) { -
    - - {{ summaryScopeRunLabel() }}. The summary is calculated on every node and waits for the slowest one. -
    -
    - - - -
    - } - - @if (!summary.isLoading) { -
    - @if (summary.data.length > 0) { - @if (filteredStatisticBlocks(kind).length > 0) { -
    - - -
    - @if (selectedStatisticBlock(kind); as selectedBlock) { -
    -
    - {{ statisticBlockTypeLabel(kind, selectedBlock) }} variable -

    {{ selectedBlock.name }} @if (selectedBlock.code) {{{ selectedBlock.code }}}

    -
    -
    -
    - - - -
    - @if (summary.activeTab === 'Histogram' && canSplitSummaryHistogram(kind, selectedBlock)) { - - } -
    - - @if (summaryExportMenuKind() === kind) { - - } -
    -
    -
    - -
    - @if (summary.activeTab === 'Statistics') { -
    - - - - - @for (ds of selectedBlock.columns; track ds) { - - } - - - - @for (r of selectedBlock.rows; track r.metric; let ri = $index) { - @if (metricGroupLabel(selectedBlock.rows, ri); as groupLabel) { - - - - } - - - @for (ds of selectedBlock.columns; track ds) { - - } - - } - -
    Metric - {{ datasetLabel(ds) }} -
    {{ groupLabel }}
    {{ r.metric }}{{ displayNumber(r.values[ds]) }}@if (missingShareNote(selectedBlock, r, ds); as share) {{{ share }}} -
    -
    - } - - @if (summary.activeTab === 'Charts') { - @if (selectedSummaryChartOptions(kind, selectedBlock).length > 0) { -
    - -
    - } @else { -
    - -

    No chart available for {{ selectedBlock.name }}.

    - Select another variable or review its statistics. -
    - } - } - - @if (summary.activeTab === 'Histogram') { - @if (summaryHistogramLoading(kind, selectedBlock)) { - -
    - - {{ summaryHistogramRunLabel(selectedBlock) }}. Numerical histograms are not part of the summary payload, so the first run for each variable is computed live and waits for the slowest node. -
    -
    - - -
    - } @else if (summaryHistogramError(kind, selectedBlock)) { -
    {{ summaryHistogramError(kind, selectedBlock) }}
    - } @else if (selectedSummaryHistogramData(kind, selectedBlock)) { -
    - -
    - } @else { -
    - -

    No histogram available for {{ selectedBlock.name }}.

    - Select another variable or review its statistics. -
    - } - } -
    - } -
    -
    - } @else { -
    - -

    No variables match the current search.

    - Clear the search field to show all statistics. -
    - } - } - @else { -
    - -

    No data available.

    - {{ emptyHint }} -
    - } -
    - } -     - @@ -444,16 +139,17 @@

    + + } + + @if (summary().isLoading) { +
    + + {{ dh.summaryScopeRunLabel() }}. The summary is calculated on every node and waits for the slowest one. +
    +
    + + + +
    + } + + @if (!summary().isLoading) { +
    + @if (summary().data.length > 0) { + @if (dh.filteredStatisticBlocks(kind()).length > 0) { +
    + + +
    + @if (dh.selectedStatisticBlock(kind()); as selectedBlock) { +
    +
    + {{ dh.statisticBlockTypeLabel(kind(), selectedBlock) }} variable +

    {{ selectedBlock.name }} @if (selectedBlock.code) {{{ selectedBlock.code }}}

    +
    +
    +
    + @for (tab of tabs; track tab.key) { + + } +
    + @if (summary().activeTab === 'Histogram' && dh.canSplitSummaryHistogram(kind(), selectedBlock)) { + + } +
    + + @if (dh.summaryExportMenuKind() === kind()) { + + } +
    +
    +
    + +
    + @if (summary().activeTab === 'Statistics') { +
    + + + + + @for (ds of selectedBlock.columns; track ds) { + + } + + + + @for (r of selectedBlock.rows; track r.metric; let ri = $index) { + @if (dh.metricGroupLabel(selectedBlock.rows, ri); as groupLabel) { + + + + } + + + @for (ds of selectedBlock.columns; track ds) { + + } + + } + +
    Metric + {{ dh.datasetLabel(ds) }} +
    {{ groupLabel }}
    {{ r.metric }}{{ dh.displayNumber(r.values[ds]) }}@if (dh.missingShareNote(selectedBlock, r, ds); as share) {{{ share }}} +
    +
    + } + + @if (summary().activeTab === 'Charts') { + @if (dh.selectedSummaryChartOptions(kind(), selectedBlock).length > 0) { +
    + +
    + } @else { +
    + +

    No chart available for {{ selectedBlock.name }}.

    + Select another variable or review its statistics. +
    + } + } + + @if (summary().activeTab === 'Histogram') { + @if (dh.summaryHistogramLoading(kind(), selectedBlock)) { + +
    + + {{ dh.summaryHistogramRunLabel(selectedBlock) }}. Numerical histograms are not part of the summary payload, so the first run for each variable is computed live and waits for the slowest node. +
    +
    + + +
    + } @else if (dh.summaryHistogramError(kind(), selectedBlock)) { +
    {{ dh.summaryHistogramError(kind(), selectedBlock) }}
    + } @else if (dh.selectedSummaryHistogramData(kind(), selectedBlock)) { +
    + +
    + } @else { +
    + +

    No histogram available for {{ selectedBlock.name }}.

    + Select another variable or review its statistics. +
    + } + } +
    + } +
    +
    + } @else { +
    + +

    No variables match the current search.

    + Clear the search field to show all statistics. +
    + } + } + @else { +
    + +

    No data available.

    + {{ emptyHint() }} +
    + } +
    + } +} diff --git a/src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.spec.ts b/src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.spec.ts new file mode 100644 index 000000000..d08722dfc --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.spec.ts @@ -0,0 +1,143 @@ +import { Component, signal } from '@angular/core'; +import { TestBed } from '@angular/core/testing'; +import { vi } from 'vitest'; +import { DATA_HANDLING } from '../data-handling.context'; +import type { DataHandlingComponent } from '../data-handling.component'; +import { PivotBlock, SummaryKind, SummaryView } from '../data-handling.models'; +import { emptySummaryView } from '../utils/summary-view.utils'; +import { SummaryWorkspaceComponent } from './summary-workspace.component'; + +const age: PivotBlock = { code: 'age', name: 'Age', columns: ['ds1'], rows: [{ metric: 'Mean', values: { ds1: '61.2' } }] }; +const sex: PivotBlock = { code: 'sex', name: 'Sex', columns: ['ds1'], rows: [{ metric: 'Count', values: { ds1: '40' } }] }; + +/** The slice of the pipeline the workspace reads, with just enough behaviour to render. */ +function pipelineStub(selected: PivotBlock | null) { + return { + cohortRetention: { raw: 120, processed: 100, dropped: 20, percentage: 83.333 }, + statisticsSearch: { source: '', raw: '', processed: '' } as Record, + summaryExportMenuKind: signal(null), + filteredStatisticBlocks: () => [age, sex], + statisticBlocks: (_kind: SummaryKind, type: string) => (type === 'numeric' ? [age] : [sex]), + summaryKindLabel: (kind: SummaryKind) => (kind === 'raw' ? 'Raw data' : kind === 'processed' ? 'Processed data' : 'Data'), + summaryScopeRunLabel: () => 'Running', + summaryGuideAnchor: () => null, + blockCoverage: () => null, + selectionCoverage: () => null, + coverageSeverity: () => 'none', + coverageTitle: () => '', + coveragePercent: () => '', + barWidth: () => 0, + isStatisticBlockSelected: (_kind: SummaryKind, block: PivotBlock) => block === selected, + selectedStatisticBlock: () => selected, + statisticBlockTypeLabel: () => 'Numerical', + canSplitSummaryHistogram: () => false, + selectedSummaryChartType: () => 'numeric', + metricGroupLabel: () => null, + isRollupColumn: () => false, + datasetLabel: (code: string) => code.toUpperCase(), + displayNumber: (value: string | null | undefined) => value ?? '', + missingShareNote: () => null, + selectStatisticBlock: vi.fn(), + setSummaryTab: vi.fn(), + toggleSummaryWorkspace: vi.fn(), + toggleSummaryExportMenu: vi.fn(), + }; +} + +@Component({ + imports: [SummaryWorkspaceComponent], + template: `
    `, +}) +class HostComponent { + readonly kind = signal('raw'); + readonly summary = signal({ ...emptySummaryView(false), data: [age, sex] }); + readonly rendered = signal(true); +} + +describe('SummaryWorkspaceComponent', () => { + async function render(selected: PivotBlock | null = age) { + const dh = pipelineStub(selected); + TestBed.configureTestingModule({ + providers: [{ provide: DATA_HANDLING, useValue: dh as unknown as DataHandlingComponent }], + }); + const fixture = TestBed.createComponent(HostComponent); + await fixture.whenStable(); + return { fixture, dh, host: fixture.componentInstance, el: fixture.nativeElement as HTMLElement }; + } + + it('renders into the section body itself', async () => { + const { el } = await render(); + expect(el.querySelector('.workflow-section-body > section.summary-workspace')).toBeTruthy(); + }); + + it('renders nothing while not rendered', async () => { + const { fixture, host, el } = await render(); + host.rendered.set(false); + await fixture.whenStable(); + expect(el.querySelector('.workflow-section-body')!.children.length).toBe(0); + }); + + it('lists numerical and categorical variables in their own groups', async () => { + const { el } = await render(); + const titles = [...el.querySelectorAll('.statistics-variable-group-title')].map((t) => t.textContent!.replace(/\s+/g, ' ').trim()); + expect(titles).toEqual(['Numerical 1', 'Categorical 1']); + expect(el.querySelector('.statistics-variable-btn.active')?.textContent).toContain('Age'); + }); + + it('selects a variable through the pipeline', async () => { + const { dh, el } = await render(); + (el.querySelectorAll('.statistics-variable-btn')[1] as HTMLButtonElement).click(); + expect(dh.selectStatisticBlock).toHaveBeenCalledWith('raw', sex); + }); + + it('shows the Table, Charts and Histogram views and marks the active one', async () => { + const { dh, el } = await render(); + const tabs = [...el.querySelectorAll('.summary-tabs button')]; + expect(tabs.map((t) => t.textContent!.trim())).toEqual(['Table', 'Charts', 'Histogram']); + expect(tabs.map((t) => t.getAttribute('aria-pressed'))).toEqual(['true', 'false', 'false']); + expect(tabs[1].querySelector('i')?.className).toBe('fas fa-chart-simple'); + tabs[2].click(); + expect(dh.setSummaryTab).toHaveBeenCalledWith('raw', 'Histogram'); + }); + + it('renders the selected variable as a statistics table', async () => { + const { el } = await render(); + expect(el.querySelector('.statistics-table thead')?.textContent).toContain('DS1'); + expect(el.querySelector('.statistics-table td.metric-cell')?.textContent).toBe('Mean'); + }); + + it('offers Close on previews but not on the Raw data snapshot', async () => { + const { fixture, host, dh, el } = await render(); + (el.querySelector('.station-preview-close') as HTMLButtonElement).click(); + expect(dh.toggleSummaryWorkspace).toHaveBeenCalledWith('raw'); + host.kind.set('source'); + await fixture.whenStable(); + expect(el.querySelector('.station-preview-close')).toBeNull(); + }); + + it('shows cohort retention on the processed preview only', async () => { + const { fixture, host, el } = await render(); + expect(el.querySelector('.cohort-retention-strip')).toBeNull(); + host.kind.set('processed'); + await fixture.whenStable(); + expect(el.querySelector('.cohort-retention-strip')?.textContent).toContain('100 / 120 records'); + expect(el.querySelector('.cohort-retention-strip')?.textContent).toContain('83.3% retained'); + }); + + it('shows a loading skeleton with a header row and three body rows', async () => { + const { fixture, host, el } = await render(); + host.summary.set(emptySummaryView(true)); + await fixture.whenStable(); + expect(el.querySelector('[aria-label="Raw data summary loading"]')).toBeTruthy(); + expect(el.querySelectorAll('.studio-skeleton-cell.head').length).toBe(4); + expect(el.querySelectorAll('.studio-skeleton-cell:not(.head)').length).toBe(12); + }); + + it('shows the surface-specific hint when there is no data', async () => { + const { fixture, host, el } = await render(); + host.summary.set(emptySummaryView(false)); + await fixture.whenStable(); + expect(el.querySelector('.empty-state-block')?.textContent).toContain('Pick some variables.'); + }); +}); diff --git a/src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.ts b/src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.ts new file mode 100644 index 000000000..b3d56ce3a --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.ts @@ -0,0 +1,53 @@ +import { ChangeDetectionStrategy, Component, inject, input } from '@angular/core'; +import { FormsModule } from '@angular/forms'; +import { ChartRendererComponent } from '../../visualisations/charts/charts-renderer/charts-renderer.component'; +import { HistogramComponent } from '../../visualisations/histogram/histogram.component'; +import { DATA_HANDLING } from '../data-handling.context'; +import { HowToGuide, StatisticVariableType, SummaryKind, SummaryView, TabKey } from '../data-handling.models'; +import { HowToStripComponent } from '../how-to-strip/how-to-strip.component'; + +/** The variable browser's groups, in display order. */ +const VARIABLE_GROUPS: ReadonlyArray<{ type: StatisticVariableType; title: string }> = [ + { type: 'numeric', title: 'Numerical' }, + { type: 'nominal', title: 'Categorical' }, +]; + +/** The detail pane's views; the first one is the default. */ +const SUMMARY_TABS: ReadonlyArray<{ key: TabKey; label: string; /** Font Awesome solid glyph. */ icon: string }> = [ + { key: 'Statistics', label: 'Table', icon: 'fa-table' }, + { key: 'Charts', label: 'Charts', icon: 'fa-chart-simple' }, + { key: 'Histogram', label: 'Histogram', icon: 'fa-chart-column' }, +]; + +/** + * The shared summary workspace: variable browser, statistics table, charts, histograms + * and export for one Data Handling surface (the Raw data snapshot, the Raw preview or + * the Processed preview). Attached to the section body element itself, so the studio's + * reveal animation and every selector see the same DOM as before. + */ +@Component({ + selector: 'div[app-summary-workspace]', + imports: [FormsModule, ChartRendererComponent, HistogramComponent, HowToStripComponent], + templateUrl: './summary-workspace.component.html', + styleUrl: './summary-workspace.component.css', + // Reads the pipeline's plain fields, so it refreshes with the pipeline (see DATA_HANDLING). + changeDetection: ChangeDetectionStrategy.Eager, +}) +export class SummaryWorkspaceComponent { + protected readonly dh = inject(DATA_HANDLING); + + readonly kind = input.required(); + readonly summary = input.required(); + /** The empty-state line shown when the summary has no data. */ + readonly emptyHint = input.required(); + /** False keeps the workspace out of the DOM (the Raw data snapshot renders only while open). */ + readonly rendered = input(true); + /** Optional "How to use" strip shown above the workspace. */ + readonly howTo = input(null); + + protected readonly variableGroups = VARIABLE_GROUPS; + protected readonly tabs = SUMMARY_TABS; + /** Placeholder grid shown while the summary loads: one header row, three body rows. */ + protected readonly skeletonHeadCells = Array.from({ length: 4 }); + protected readonly skeletonBodyCells = Array.from({ length: 12 }); +} diff --git a/src/app/pages/experiment-studio/data-handling/utils/summary-view.utils.spec.ts b/src/app/pages/experiment-studio/data-handling/utils/summary-view.utils.spec.ts new file mode 100644 index 000000000..043731de3 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/utils/summary-view.utils.spec.ts @@ -0,0 +1,18 @@ +import { emptySummaryView } from './summary-view.utils'; + +describe('emptySummaryView', () => { + it('starts on the Table view with no data', () => { + const view = emptySummaryView(true); + expect(view.isLoading).toBe(true); + expect(view.activeTab).toBe('Statistics'); + expect(view.data).toEqual([]); + expect(view.selectedStatisticKey).toBeNull(); + }); + + it('never shares caches between views', () => { + const a = emptySummaryView(false); + const b = emptySummaryView(false); + expect(a.histogramDataByVariable).not.toBe(b.histogramDataByVariable); + expect(a.data).not.toBe(b.data); + }); +}); diff --git a/src/app/pages/experiment-studio/data-handling/utils/summary-view.utils.ts b/src/app/pages/experiment-studio/data-handling/utils/summary-view.utils.ts new file mode 100644 index 000000000..cab36f0c4 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/utils/summary-view.utils.ts @@ -0,0 +1,26 @@ +import { SummaryView } from '../data-handling.models'; + +/** A summary surface with nothing loaded yet: the Table view, no variables, no per-variable caches. */ +export function emptySummaryView(isLoading: boolean): SummaryView { + return { + activeTab: 'Statistics', + data: [], + featurewiseRows: [], + isLoading, + showDistributions: false, + distributionSubTab: 'Numeric', + nonNominalVariables: [], + nominalVariables: [], + chartsForBoxPlot: [], + chartsForNominal: [], + histogramDataByVariable: {}, + histogramLoadingByVariable: {}, + histogramErrorByVariable: {}, + histogramSubscriptionsByVariable: {}, + histogramRequestIdByVariable: {}, + histogramSplitByVariable: {}, + activeBoxPlotIndex: 0, + activeNominalIndex: 0, + selectedStatisticKey: null, + }; +} From 701ed12e23db26c911b66255d21650604ca654c1 Mon Sep 17 00:00:00 2001 From: "K.Filippopolitis" <56073635+KFilippopolitis@users.noreply.github.com> Date: Fri, 23 Oct 2026 23:12:07 +0300 Subject: [PATCH 06/31] refactor(data-handling): type and extract the describe-summary pivot Moves the pure part of the statistic summary out of DataHandlingComponent into utils/statistic-summary.utils.ts and types it with the existing VariableStats model instead of any: - pivotByDataset: one table per variable from describe's featurewise rows. The metric list (SUMMARY_METRICS) now uses the payload's own keys, which replaces a six-level ternary that translated num_datapoints/num_missing back to num_dtps/num_na; the counts / categories / full-metrics branches share one row builder. - variableLabel, isCategoricalVariable, enumLabelMap, categoryCounts and pooledTotalRows: small helpers the component re-implemented inline (the public variableLabel / isCategoricalVariable methods delegate, so templates and specs are unchanged). - SummaryView.featurewiseRows is VariableStats[] (was any[]), which removes the remaining `any` casts in buildSummaryFromResponse, buildDistributionState, buildHistogramFromDescribeCounts and getSummaryTotalRows. Rendered tables are unchanged (harness: identical DOM, styles and screenshots). Spec: statistic-summary.utils (9). --- .../data-handling.component.spec.ts | 4 +- .../data-handling/data-handling.component.ts | 144 +++--------------- .../data-handling/data-handling.models.ts | 4 +- .../utils/statistic-summary.utils.spec.ts | 80 ++++++++++ .../utils/statistic-summary.utils.ts | 118 ++++++++++++++ 5 files changed, 220 insertions(+), 130 deletions(-) create mode 100644 src/app/pages/experiment-studio/data-handling/utils/statistic-summary.utils.spec.ts create mode 100644 src/app/pages/experiment-studio/data-handling/utils/statistic-summary.utils.ts diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.spec.ts b/src/app/pages/experiment-studio/data-handling/data-handling.component.spec.ts index f891f56b7..24cd4a15b 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.spec.ts +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.spec.ts @@ -1055,7 +1055,7 @@ describe('DataHandlingComponent', () => { }); fixture.detectChanges(); - expect(component.processedSummary.featurewiseRows[0].data.mean).toBe(82); + expect(component.processedSummary.featurewiseRows[0].data).toMatchObject({ mean: 82 }); first.complete(); second.complete(); @@ -1093,7 +1093,7 @@ describe('DataHandlingComponent', () => { fixture.detectChanges(); expect(component.sourceSummary.featurewiseRows.length).toBe(1); - expect(component.sourceSummary.featurewiseRows[0].data.mean).toBe(82); + expect(component.sourceSummary.featurewiseRows[0].data).toMatchObject({ mean: 82 }); expect(component.sourceSummary.isLoading).toBe(false); stale.complete(); diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.ts b/src/app/pages/experiment-studio/data-handling/data-handling.component.ts index ebf3e696d..38d4f42e6 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.ts +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.ts @@ -48,8 +48,10 @@ import { HowToStripComponent } from './how-to-strip/how-to-strip.component'; import { SummaryWorkspaceComponent } from './summary-workspace/summary-workspace.component'; import { DATA_HANDLING } from './data-handling.context'; import { emptySummaryView } from './utils/summary-view.utils'; +import { categoryCounts, enumLabelMap, isCategoricalVariable, pivotByDataset, pooledTotalRows, variableLabel } from './utils/statistic-summary.utils'; import { PipelineTerminalComponent } from './pipeline-terminal/pipeline-terminal.component'; import { BackendFilter } from '../../../models/filters.model'; +import { VariableStats } from '../../../models/algorithm-results.model'; import { CsvExportService } from '../../../services/csv-export.service'; import { ExperimentStudioNavigationService } from '../../../services/experiment-studio-navigation.service'; import { countFilterRules, formatFilterExpression } from '../../../core/filter-display.utils'; @@ -109,7 +111,6 @@ import { buildPreprocessingDocumentationHtml } from './utils/preprocessing-docum import { emptyTransformationDraft, nextTransformationDraftId } from './utils/transformation-draft.utils'; import { formatInteger, - formatNumber, formatOutlierBound, formatOutlierPercentage, formatPercent, @@ -774,19 +775,6 @@ export class DataHandlingComponent implements OnDestroy { readonly outlierStrategies = OUTLIER_STRATEGIES; readonly outlierTails = OUTLIER_TAILS; - readonly metricOrder: Array<{ key: string; label: string }> = [ - { key: 'num_datapoints', label: 'Datapoints' }, - { key: 'num_missing', label: 'Missing' }, - { key: 'num_total', label: 'Total' }, - { key: 'mean', label: 'Mean' }, - { key: 'std', label: 'Standard Deviation' }, - { key: 'min', label: 'Min' }, - { key: 'q1', label: 'Q1' }, - { key: 'q2', label: 'Median' }, - { key: 'q3', label: 'Q3' }, - { key: 'max', label: 'Max' }, - ]; - readonly metricLabel: Record = { num_dtps: 'Datapoints', num_na: 'Missing', @@ -2849,7 +2837,7 @@ export class DataHandlingComponent implements OnDestroy { } variableLabel(variable: VariableRow): string { - return variable.name ?? variable.label ?? variable.code; + return variableLabel(variable); } datasetLabel(datasetCode: string): string { @@ -2984,13 +2972,13 @@ export class DataHandlingComponent implements OnDestroy { const current = this.getSummary(kind); const featurewise = getFeaturewiseDescribeRows(response); const dsFromPayload = Array.from( - new Set((featurewise ?? []).map((item: any) => String(item.dataset))) + new Set(featurewise.map((item) => String(item.dataset))) ) as string[]; const datasetOrder = dsFromPayload .filter((dataset) => dataset && dataset !== 'all datasets') .concat('all datasets'); const variableList = this.preprocessingVariables; - const data = this.pivotByDataset(featurewise, variableList, datasetOrder); + const data = pivotByDataset(featurewise, variableList, datasetOrder); const distributionState = this.buildDistributionState(featurewise, response); const summary: SummaryView = { ...current, @@ -3198,12 +3186,10 @@ export class DataHandlingComponent implements OnDestroy { block: PivotBlock ): HistogramPreviewData | null { const featurewise = this.getSummary(kind).featurewiseRows; - const row = featurewise.find( - (item: { variable?: string; dataset?: string }) => - item?.variable === code && String(item?.dataset ?? '') === 'all datasets' - ) ?? featurewise.find((item: { variable?: string }) => item?.variable === code); - const counts = (row as { data?: { counts?: Record } })?.data?.counts; - if (!counts || typeof counts !== 'object' || !Object.keys(counts).length) { + const row = featurewise.find((item) => item?.variable === code && String(item?.dataset ?? '') === 'all datasets') + ?? featurewise.find((item) => item?.variable === code); + const counts = categoryCounts(row?.data); + if (!Object.keys(counts).length) { return null; } @@ -3318,12 +3304,12 @@ export class DataHandlingComponent implements OnDestroy { } const selected = (this.expStudioService.selectedVariables() as VariableRow[]) .find((variable) => variable.code === groupingVar); - const mapped = this.getEnumLabelMap(selected ?? this.findModelVariable(groupingVar) ?? undefined) + const mapped = enumLabelMap(selected ?? this.findModelVariable(groupingVar) ?? undefined) .get(groupingEnum); return mapped ?? groupingEnum; } - private buildDistributionState(featurewise: any[], response: unknown): Pick { @@ -3332,7 +3318,7 @@ export class DataHandlingComponent implements OnDestroy { const varsWithCounts = new Set(); for (const item of featurewise) { - if (item.data && item.data.counts && Object.keys(item.data.counts).length > 0) { + if (Object.keys(categoryCounts(item.data)).length > 0) { varsWithCounts.add(item.variable); } } @@ -3354,7 +3340,7 @@ export class DataHandlingComponent implements OnDestroy { result: { ...((response as { result?: Record })?.result ?? {}), featurewise: featurewise.filter( - (row: any) => row.variable === v.code && row.dataset !== 'all datasets' + (row) => row.variable === v.code && row.dataset !== 'all datasets' ), dataset_labels: datasetLabels, }, @@ -3363,9 +3349,9 @@ export class DataHandlingComponent implements OnDestroy { }); const chartsForNominal = nominalVariables.map((v) => { - const varData = featurewise.filter((row: any) => row.variable === v.code); + const varData = featurewise.filter((row) => row.variable === v.code); const varLabel = this.variableLabel(v); - const enumMap = this.getEnumLabelMap(v); + const enumMap = enumLabelMap(v); return buildGroupedBarChart(varData, varLabel, enumMap); }); @@ -4300,10 +4286,7 @@ export class DataHandlingComponent implements OnDestroy { } private getSummaryTotalRows(summary: SummaryView): number | null { - const all = (summary.featurewiseRows ?? []).find((item: any) => item.dataset === 'all datasets'); - if (!all) return null; - const total = Number(all.data?.num_total); - return Number.isFinite(total) ? total : null; + return pooledTotalRows(summary.featurewiseRows ?? []); } get cohortRetention(): { raw: number; processed: number; dropped: number; percentage: number } | null { @@ -4331,7 +4314,7 @@ export class DataHandlingComponent implements OnDestroy { } isCategoricalVariable(variable: VariableRow): boolean { - return String(variable.type ?? '').toLowerCase() === 'nominal' || !!variable.enumerations?.length; + return isCategoricalVariable(variable); } private enumCodes(variable: VariableRow): string[] { @@ -4397,19 +4380,6 @@ export class DataHandlingComponent implements OnDestroy { return mapped > 0 ? mappedBins : bins; } - private getEnumLabelMap(variable: VariableRow | undefined): Map { - const enums = Array.isArray(variable?.enumerations) ? variable.enumerations : []; - const map = new Map(); - enums.forEach((item) => { - const raw = item?.code ?? item?.label ?? item?.name; - if (raw === null || raw === undefined) return; - const key = String(raw); - const label = String(item?.label ?? item?.name ?? raw); - map.set(key, label); - }); - return map; - } - private pivotBlocksWithDatasetLabels(blocks: PivotBlock[]): PivotBlock[] { return blocks.map((block) => { const labeledColumns = block.columns.map((code) => this.datasetLabel(code)); @@ -4426,86 +4396,6 @@ export class DataHandlingComponent implements OnDestroy { }); } - private pivotByDataset( - items: any[], - variableList: VariableRow[], - datasetOrder: string[] - ): PivotBlock[] { - const byVar: Record = {}; - for (const item of items || []) { - const arr = byVar[item.variable] || (byVar[item.variable] = []); - arr.push(item); - } - - const result: PivotBlock[] = []; - for (const [varCode, arr] of Object.entries(byVar)) { - const matched = variableList.find((v) => v.code === varCode); - const varName = matched ? this.variableLabel(matched) : varCode; - const enumMap = this.getEnumLabelMap(matched); - const byDataset: Record = {}; - for (const entry of arr) byDataset[entry.dataset] = entry.data || {}; - - const countsKeys = new Set(); - for (const ds of datasetOrder) { - const counts = byDataset[ds]?.counts ?? {}; - Object.keys(counts).forEach((key) => countsKeys.add(String(key))); - } - - let rows: Array<{ metric: string; values: Record }> = []; - if (countsKeys.size > 0) { - const baseMetrics = [ - { key: 'num_datapoints', label: 'Datapoints' }, - { key: 'num_missing', label: 'Missing' }, - { key: 'num_total', label: 'Total' }, - ]; - rows = baseMetrics.map((metric) => { - const values: Record = {}; - for (const ds of datasetOrder) { - const raw = - metric.key === 'num_datapoints' ? byDataset[ds]?.num_dtps : - metric.key === 'num_missing' ? byDataset[ds]?.num_na : - byDataset[ds]?.num_total; - values[ds] = formatInteger(raw); - } - return { metric: metric.label, values }; - }); - - const orderedKeys = Array.from(enumMap.keys()).filter((key) => countsKeys.has(key)); - const remainingKeys = Array.from(countsKeys).filter((key) => !enumMap.has(key)); - orderedKeys.concat(remainingKeys).forEach((key) => { - const label = enumMap.get(key) ?? key; - const values: Record = {}; - for (const ds of datasetOrder) { - values[ds] = formatInteger(byDataset[ds]?.counts?.[key]); - } - rows.push({ metric: label, values }); - }); - } else { - const isCategorical = !!matched && this.isCategoricalVariable(matched); - const countKeys = new Set(['num_datapoints', 'num_missing', 'num_total']); - const metrics = isCategorical - ? this.metricOrder.filter((metric) => countKeys.has(metric.key)) - : this.metricOrder; - rows = metrics.map((metric) => { - const values: Record = {}; - for (const ds of datasetOrder) { - const raw = - metric.key === 'num_datapoints' ? byDataset[ds]?.num_dtps : - metric.key === 'num_missing' ? byDataset[ds]?.num_na : - metric.key === 'num_total' ? byDataset[ds]?.num_total : - metric.key === 'std' ? byDataset[ds]?.std : - metric.key === 'q2' ? byDataset[ds]?.q2 : - byDataset[ds]?.[metric.key]; - values[ds] = countKeys.has(metric.key) ? formatInteger(raw) : formatNumber(raw); - } - return { metric: metric.label, values }; - }); - } - result.push({ code: varCode, name: varName, columns: datasetOrder, rows }); - } - return result; - } - private statisticBlockKey(block: PivotBlock): string { return block.code ?? block.name; } diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.models.ts b/src/app/pages/experiment-studio/data-handling/data-handling.models.ts index d680b6f7d..1a1783b9e 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.models.ts +++ b/src/app/pages/experiment-studio/data-handling/data-handling.models.ts @@ -1,5 +1,6 @@ import { EChartsOption } from 'echarts'; import { Subscription } from 'rxjs'; +import { VariableStats } from '../../../models/algorithm-results.model'; import { BackendFilter, BackendRule } from '../../../models/filters.model'; import { HistogramGroupSeries } from '../visualisations/histogram/histogram-data'; @@ -79,7 +80,8 @@ export type MetricKey = export interface SummaryView { activeTab: TabKey; data: PivotBlock[]; - featurewiseRows: any[]; + /** The describe payload's featurewise rows behind `data` (one per variable x dataset). */ + featurewiseRows: VariableStats[]; isLoading: boolean; showDistributions: boolean; distributionSubTab: DistributionSubTab; diff --git a/src/app/pages/experiment-studio/data-handling/utils/statistic-summary.utils.spec.ts b/src/app/pages/experiment-studio/data-handling/utils/statistic-summary.utils.spec.ts new file mode 100644 index 000000000..a958970d9 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/utils/statistic-summary.utils.spec.ts @@ -0,0 +1,80 @@ +import { VariableStats } from '../../../../models/algorithm-results.model'; +import { VariableRow } from '../data-handling.models'; +import { + categoryCounts, + enumLabelMap, + isCategoricalVariable, + pivotByDataset, + pooledTotalRows, + variableLabel, +} from './statistic-summary.utils'; + +const numeric = (dataset: string, mean: number | null): VariableStats => ({ + variable: 'age', + dataset, + data: { num_dtps: 1200, num_na: 3, num_total: 1203, mean, std: 1.234567, min: 1, q1: 2, q2: 3, q3: 4, max: 5 }, +}); +const nominal = (dataset: string, counts: Record): VariableStats => ({ + variable: 'sex', + dataset, + data: { num_dtps: 10, num_na: 0, num_total: 10, counts }, +}); +const sex: VariableRow = { code: 'sex', label: 'Sex', type: 'nominal', enumerations: [{ code: 'F', label: 'Female' }, { code: 'M', label: 'Male' }] }; + +describe('statistic-summary utils', () => { + it('labels a variable by name, then label, then code', () => { + expect(variableLabel({ code: 'a', name: 'N', label: 'L' })).toBe('N'); + expect(variableLabel({ code: 'a', label: 'L' })).toBe('L'); + expect(variableLabel({ code: 'a' })).toBe('a'); + }); + + it('treats nominal or enumerated variables as categorical', () => { + expect(isCategoricalVariable({ code: 'a', type: 'Nominal' })).toBe(true); + expect(isCategoricalVariable({ code: 'a', type: 'real', enumerations: [{ code: '1' }] })).toBe(true); + expect(isCategoricalVariable({ code: 'a', type: 'real' })).toBe(false); + }); + + it('maps enumeration codes to labels', () => { + expect([...enumLabelMap(sex)]).toEqual([['F', 'Female'], ['M', 'Male']]); + expect(enumLabelMap(undefined).size).toBe(0); + }); + + it('reads category counts only from nominal stats', () => { + expect(categoryCounts(nominal('d', { F: 1 }).data)).toEqual({ F: 1 }); + expect(categoryCounts(numeric('d', 1).data)).toEqual({}); + expect(categoryCounts(null)).toEqual({}); + }); + + it('reads the pooled record count', () => { + expect(pooledTotalRows([numeric('d1', 1), numeric('all datasets', 1)])).toBe(1203); + expect(pooledTotalRows([numeric('d1', 1)])).toBeNull(); + }); + + it('pivots a numeric variable into every metric, one column per dataset', () => { + const [block] = pivotByDataset([numeric('d1', 61.25), numeric('d2', null)], [{ code: 'age', name: 'Age' }], ['d1', 'd2']); + expect(block.code).toBe('age'); + expect(block.name).toBe('Age'); + expect(block.columns).toEqual(['d1', 'd2']); + expect(block.rows.map((row) => row.metric)).toEqual( + ['Datapoints', 'Missing', 'Total', 'Mean', 'Standard Deviation', 'Min', 'Q1', 'Median', 'Q3', 'Max']); + expect(block.rows[0].values).toEqual({ d1: '1200', d2: '1200' }); + expect(block.rows[3].values['d2']).toBe('N/A'); + }); + + it('pivots category counts in enumeration order, then unknown categories', () => { + const [block] = pivotByDataset([nominal('d1', { M: 4, X: 1, F: 5 })], [sex], ['d1', 'd2']); + expect(block.name).toBe('Sex'); + expect(block.rows.map((row) => row.metric)).toEqual(['Datapoints', 'Missing', 'Total', 'Female', 'Male', 'X']); + expect(block.rows[3].values).toEqual({ d1: '5', d2: 'N/A' }); + }); + + it('shows only record counts for a categorical variable without category counts', () => { + const rows: VariableStats[] = [{ variable: 'sex', dataset: 'd1', data: null }]; + const [block] = pivotByDataset(rows, [sex], ['d1']); + expect(block.rows.map((row) => row.metric)).toEqual(['Datapoints', 'Missing', 'Total']); + }); + + it('falls back to the code for variables it does not know', () => { + expect(pivotByDataset([numeric('d1', 1)], [], ['d1'])[0].name).toBe('age'); + }); +}); diff --git a/src/app/pages/experiment-studio/data-handling/utils/statistic-summary.utils.ts b/src/app/pages/experiment-studio/data-handling/utils/statistic-summary.utils.ts new file mode 100644 index 000000000..67cea44d9 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/utils/statistic-summary.utils.ts @@ -0,0 +1,118 @@ +import { VariableStats } from '../../../../models/algorithm-results.model'; +import { MetricKey, PivotBlock, VariableRow } from '../data-handling.models'; +import { formatInteger, formatNumber } from './statistic-format.utils'; + +/** The describe metrics a summary table shows, in row order, with their row labels. */ +export const SUMMARY_METRICS: ReadonlyArray<{ key: MetricKey; label: string }> = [ + { key: 'num_dtps', label: 'Datapoints' }, + { key: 'num_na', label: 'Missing' }, + { key: 'num_total', label: 'Total' }, + { key: 'mean', label: 'Mean' }, + { key: 'std', label: 'Standard Deviation' }, + { key: 'min', label: 'Min' }, + { key: 'q1', label: 'Q1' }, + { key: 'q2', label: 'Median' }, + { key: 'q3', label: 'Q3' }, + { key: 'max', label: 'Max' }, +]; + +/** Record counts: shown as integers, and the only rows a categorical variable gets. */ +const COUNT_METRICS: ReadonlySet = new Set(['num_dtps', 'num_na', 'num_total']); +const COUNT_ROWS = SUMMARY_METRICS.filter((metric) => COUNT_METRICS.has(metric.key)); + +type DescribeStats = NonNullable; +type PivotRow = PivotBlock['rows'][number]; + +/** The name a variable is shown under. */ +export function variableLabel(variable: VariableRow): string { + return variable.name ?? variable.label ?? variable.code; +} + +export function isCategoricalVariable(variable: VariableRow): boolean { + return String(variable.type ?? '').toLowerCase() === 'nominal' || !!variable.enumerations?.length; +} + +/** Category code -> display label, from the variable's enumerations (empty when it has none). */ +export function enumLabelMap(variable: VariableRow | null | undefined): Map { + const map = new Map(); + for (const item of variable?.enumerations ?? []) { + const raw = item?.code ?? item?.label ?? item?.name; + if (raw === null || raw === undefined) continue; + map.set(String(raw), String(item?.label ?? item?.name ?? raw)); + } + return map; +} + +/** The record count of the pooled "all datasets" row, or null when the payload has none. */ +export function pooledTotalRows(rows: readonly VariableStats[]): number | null { + const pooled = rows.find((row) => row.dataset === 'all datasets'); + if (!pooled) return null; + const total = Number(pooled.data?.num_total); + return Number.isFinite(total) ? total : null; +} + +function metricValue(stats: DescribeStats | undefined, key: MetricKey): unknown { + return (stats as Partial> | undefined)?.[key]; +} + +/** A nominal variable's per-category record counts; empty for numeric stats or missing data. */ +export function categoryCounts(stats: VariableStats['data'] | undefined): Record { + const counts: unknown = stats && 'counts' in stats ? stats.counts : undefined; + return counts && typeof counts === 'object' ? (counts as Record) : {}; +} + +/** + * Turns describe's featurewise rows (one per variable x dataset) into one table per + * variable: a column per dataset in `datasetOrder`, a row per metric. + * - Variables the payload reports category counts for get the three record counts + * plus one row per category (enumeration order first, then any others). + * - Other categorical variables get the record counts only; the rest get every metric. + * Variables come out in payload order and are labelled from `variables` when known. + */ +export function pivotByDataset( + rows: readonly VariableStats[], + variables: readonly VariableRow[], + datasetOrder: readonly string[], +): PivotBlock[] { + // A plain record on purpose: Object.entries order is the order tables have always had. + const rowsByVariable: Record = {}; + for (const row of rows) (rowsByVariable[row.variable] ??= []).push(row); + + return Object.entries(rowsByVariable).map(([code, variableRows]) => { + const variable = variables.find((item) => item.code === code); + const statsByDataset = new Map(); + for (const row of variableRows) statsByDataset.set(row.dataset, row.data ?? ({} as DescribeStats)); + + const metricRow = (metric: { key: MetricKey; label: string }): PivotRow => ({ + metric: metric.label, + values: Object.fromEntries(datasetOrder.map((ds) => { + const value = metricValue(statsByDataset.get(ds), metric.key); + return [ds, COUNT_METRICS.has(metric.key) ? formatInteger(value) : formatNumber(value)]; + })), + }); + + const categories = new Set(); + for (const ds of datasetOrder) Object.keys(categoryCounts(statsByDataset.get(ds))).forEach((key) => categories.add(key)); + + let tableRows: PivotRow[]; + if (categories.size > 0) { + const labels = enumLabelMap(variable); + const ordered = [ + ...Array.from(labels.keys()).filter((key) => categories.has(key)), + ...Array.from(categories).filter((key) => !labels.has(key)), + ]; + tableRows = [ + ...COUNT_ROWS.map(metricRow), + ...ordered.map((key) => ({ + metric: labels.get(key) ?? key, + values: Object.fromEntries(datasetOrder.map((ds) => [ds, formatInteger(categoryCounts(statsByDataset.get(ds))[key])])), + })), + ]; + } else { + const categorical = !!variable && isCategoricalVariable(variable); + tableRows = (categorical ? COUNT_ROWS : SUMMARY_METRICS).map(metricRow); + } + + return { code, name: variable ? variableLabel(variable) : code, columns: [...datasetOrder], rows: tableRows }; + }); +} From 2bf8e1d1f0698f6e17a8868c50bb6fb5445f7a72 Mon Sep 17 00:00:00 2001 From: "K.Filippopolitis" <56073635+KFilippopolitis@users.noreply.github.com> Date: Fri, 23 Oct 2026 23:14:51 +0300 Subject: [PATCH 07/31] refactor(data-handling): share the pending-change check between preprocessing steps hasPendingMissingChanges and hasPendingOutlierChanges each re-implemented the per-variable comparison that prepVariableHasPendingChange already did (and walked a code set built from three sources that always reduced to the current variables). Each step now has one per-variable check (missingRuleHasPendingChange, outlierRuleHasPendingChange); the step-level getters are `some()` over it, and the "edit, else step default while nothing is applied" resolution both steps used is the pure utils/preprocessing-state.utils pendingOrDefault. No behaviour change: same variables, same comparison, same implicit-default rule. Spec: pendingOrDefault (3). --- .../data-handling/data-handling.component.ts | 79 +++++++------------ .../utils/preprocessing-state.utils.spec.ts | 18 +++++ .../utils/preprocessing-state.utils.ts | 8 ++ 3 files changed, 53 insertions(+), 52 deletions(-) create mode 100644 src/app/pages/experiment-studio/data-handling/utils/preprocessing-state.utils.spec.ts create mode 100644 src/app/pages/experiment-studio/data-handling/utils/preprocessing-state.utils.ts diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.ts b/src/app/pages/experiment-studio/data-handling/data-handling.component.ts index 38d4f42e6..edde644b7 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.ts +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.ts @@ -48,6 +48,7 @@ import { HowToStripComponent } from './how-to-strip/how-to-strip.component'; import { SummaryWorkspaceComponent } from './summary-workspace/summary-workspace.component'; import { DATA_HANDLING } from './data-handling.context'; import { emptySummaryView } from './utils/summary-view.utils'; +import { pendingOrDefault } from './utils/preprocessing-state.utils'; import { categoryCounts, enumLabelMap, isCategoricalVariable, pivotByDataset, pooledTotalRows, variableLabel } from './utils/statistic-summary.utils'; import { PipelineTerminalComponent } from './pipeline-terminal/pipeline-terminal.component'; import { BackendFilter } from '../../../models/filters.model'; @@ -1029,26 +1030,7 @@ export class DataHandlingComponent implements OnDestroy { } get hasPendingMissingChanges(): boolean { - const currentCodes = this.currentPreprocessingCodeSet(); - const codes = new Set( - [ - ...Array.from(currentCodes), - ...Object.keys(this.pendingPreprocessingRules), - ...Object.keys(this.appliedPreprocessingRules), - ].filter((code) => currentCodes.has(code)) - ); - for (const code of codes) { - const appliedRule = this.appliedPreprocessingRules[code]; - const pendingRule = this.pendingPreprocessingRules[code] - ?? (appliedRule ? undefined : this.defaultRule(code)); - // Newly added variables sit on the implicit NA-removal default; that - // doesn't count as a pending change because no user action is required. - if (this.isDefaultMissingRule(pendingRule) && !this.hasAppliedPreprocessing(code)) continue; - if (this.serializeRule(pendingRule) !== this.serializeRule(appliedRule)) { - return true; - } - } - return false; + return this.preprocessingVariables.some((variable) => this.missingRuleHasPendingChange(variable.code)); } get pendingOutlierChangeCount(): number { @@ -1056,22 +1038,7 @@ export class DataHandlingComponent implements OnDestroy { } get hasPendingOutlierChanges(): boolean { - const currentCodes = this.currentOutlierPreprocessingCodeSet(); - const codes = new Set( - [ - ...Array.from(currentCodes), - ...Object.keys(this.pendingOutlierRules), - ...Object.keys(this.appliedOutlierRules), - ].filter((code) => currentCodes.has(code)) - ); - for (const code of codes) { - const pendingRule = this.pendingOutlierRules[code] - ?? (this.appliedOutlierRules[code] ? undefined : this.defaultOutlierRule(code)); - if (serializeOutlierRule(pendingRule) !== serializeOutlierRule(this.appliedOutlierRules[code])) { - return true; - } - } - return false; + return this.outlierPreprocessingVariables.some((variable) => this.outlierRuleHasPendingChange(variable.code)); } get pendingLongitudinalChangeCount(): number { @@ -1174,23 +1141,31 @@ export class DataHandlingComponent implements OnDestroy { return 'Not applied'; } - prepVariableHasPendingChange(kind: PrepKind, variable: VariableRow): boolean { + /** + * Whether a variable's missing-values edit differs from what the run uses. A variable + * with nothing applied is compared through the step default (drop rows), and sitting + * on that implicit default is not a change: no user action is required. + */ + private missingRuleHasPendingChange(code: string): boolean { + const applied = this.appliedPreprocessingRules[code]; + const pending = pendingOrDefault(this.pendingPreprocessingRules[code], applied, () => this.defaultRule(code)); + if (this.isDefaultMissingRule(pending) && !this.hasAppliedPreprocessing(code)) return false; + return this.serializeRule(pending) !== this.serializeRule(applied); + } + + /** Whether a variable's outlier edit differs from what the run uses (default: clipping off). */ + private outlierRuleHasPendingChange(code: string): boolean { + const applied = this.appliedOutlierRules[code]; + const pending = pendingOrDefault(this.pendingOutlierRules[code], applied, () => this.defaultOutlierRule(code)); + return serializeOutlierRule(pending) !== serializeOutlierRule(applied); + } + + prepVariableHasPendingChange(kind: PrepKind, variable: VariableRow): boolean { switch (kind) { - case 'missing': { - const appliedRule = this.appliedPreprocessingRules[variable.code]; - const pendingRule = this.pendingPreprocessingRules[variable.code] - ?? (appliedRule ? undefined : this.defaultRule(variable.code)); - if (this.isDefaultMissingRule(pendingRule) && !this.hasAppliedPreprocessing(variable.code)) { - return false; - } - return this.serializeRule(pendingRule) !== this.serializeRule(appliedRule); - } - case 'outlier': { - const appliedRule = this.appliedOutlierRules[variable.code]; - const pendingRule = this.pendingOutlierRules[variable.code] - ?? (appliedRule ? undefined : this.defaultOutlierRule(variable.code)); - return serializeOutlierRule(pendingRule) !== serializeOutlierRule(appliedRule); - } + case 'missing': + return this.missingRuleHasPendingChange(variable.code); + case 'outlier': + return this.outlierRuleHasPendingChange(variable.code); case 'longitudinal': { if (!this.isLongitudinalModel) return false; const pendingStrategy = this.pendingLongitudinalStrategies[variable.code] ?? this.defaultLongitudinalStrategy(variable); diff --git a/src/app/pages/experiment-studio/data-handling/utils/preprocessing-state.utils.spec.ts b/src/app/pages/experiment-studio/data-handling/utils/preprocessing-state.utils.spec.ts new file mode 100644 index 000000000..12288fa41 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/utils/preprocessing-state.utils.spec.ts @@ -0,0 +1,18 @@ +import { pendingOrDefault } from './preprocessing-state.utils'; + +describe('pendingOrDefault', () => { + const stepDefault = () => 'default'; + + it('prefers the edit', () => { + expect(pendingOrDefault('edit', 'applied', stepDefault)).toBe('edit'); + expect(pendingOrDefault('edit', undefined, stepDefault)).toBe('edit'); + }); + + it('falls back to the step default while nothing is applied', () => { + expect(pendingOrDefault(undefined, undefined, stepDefault)).toBe('default'); + }); + + it('has no pending rule when an applied rule was not edited', () => { + expect(pendingOrDefault(undefined, 'applied', stepDefault)).toBeUndefined(); + }); +}); diff --git a/src/app/pages/experiment-studio/data-handling/utils/preprocessing-state.utils.ts b/src/app/pages/experiment-studio/data-handling/utils/preprocessing-state.utils.ts new file mode 100644 index 000000000..0c35bb51c --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/utils/preprocessing-state.utils.ts @@ -0,0 +1,8 @@ +/** + * The rule a preprocessing step compares against what the run uses: the user's edit + * when there is one; otherwise the step default, but only while nothing is applied + * for the variable (an applied rule with no edit is simply unchanged). + */ +export function pendingOrDefault(pending: R | undefined, applied: R | undefined, stepDefault: () => R): R | undefined { + return pending ?? (applied ? undefined : stepDefault()); +} From 32a72759887e704f886eaf15ba372d02bfc04097 Mon Sep 17 00:00:00 2001 From: "K.Filippopolitis" <56073635+KFilippopolitis@users.noreply.github.com> Date: Fri, 23 Oct 2026 23:15:20 +0300 Subject: [PATCH 08/31] docs(data-handling): record the section-component split Decision log entry for how Data Handling is split (attribute-hosted sections, DATA_HANDLING context + Eager change detection, CSS repeated in cascade order, pure utils), and module-index / conventions pointers to the new pieces. --- docs/context/conventions.md | 2 +- docs/context/decision-log.md | 18 ++++++++++++++++++ docs/context/module-index.md | 2 +- 3 files changed, 20 insertions(+), 2 deletions(-) diff --git a/docs/context/conventions.md b/docs/context/conventions.md index 63f763f80..5a595fbaf 100644 --- a/docs/context/conventions.md +++ b/docs/context/conventions.md @@ -30,7 +30,7 @@ ### UI and Styling - Use component CSS for feature-specific styles. - Use `src/styles.css` for app-wide styles and QueryBuilder theming. -- A component may list several stylesheets in `styleUrls`; Angular concatenates them **in array order**, so that order is the cascade. Splitting a file must preserve the original order and must not move a rule across another rule that shares its selector — `data-handling` (`DataHandlingComponent`) is split this way (`.component.css` shell + pipeline canvas, then `.preprocessing.css`, then `.results.css`). +- A component may list several stylesheets in `styleUrls`; Angular concatenates them **in array order**, so that order is the cascade. Splitting a file must preserve the original order and must not move a rule across another rule that shares its selector — `data-handling` (`DataHandlingComponent`) is split this way (`.component.css` shell + pipeline canvas, then `.preprocessing.css`, then `.results.css`). Its section components attach to the element they replace (attribute selectors such as `div[app-pipeline-terminal]`) and repeat the shared rules they need in the same order; see the decision log entry on Data Handling sections. - Consult `DESIGN.md` before visual changes. - Existing branding assets live under `src/assets`. diff --git a/docs/context/decision-log.md b/docs/context/decision-log.md index 4977b0c20..b10751b9b 100644 --- a/docs/context/decision-log.md +++ b/docs/context/decision-log.md @@ -201,3 +201,21 @@ Consequences: Breaking: this UI needs a backend with PR #34 and does not work ag Files affected: `src/app/services/experiment-studio.service.ts`, `src/app/services/experiment-studio.service.spec.ts`, `e2e/experiment-studio-api.ts`, `e2e/experiment-studio.flows.spec.ts`, `README.md`, `docs/context/*.md`, `docs/llm-wiki/indexes/backend-endpoints.md`. Date: 2026-10-15. + +### Decision: Data Handling sections are attribute-hosted components over the pipeline's state + +Status: Accepted + +Context: `statistic-analysis-panel` grew to one ~4.5k-line component with a ~1.8k-line template. It was renamed `data-handling` (`DataHandlingComponent`, `app-data-handling`) and is being split into section components. Behaviour, visuals, `data-guide` anchors, e2e selectors, payloads and sessionStorage keys must not change, and the studio's global reveal rule (`.experiment-studio-page .workflow-section-body.open > *`) animates the direct children of a section body. + +Decision: +- A section component attaches to the element it replaces (`details[app-how-to-strip]`, `div[app-pipeline-terminal]`, `div.workflow-section-body[app-summary-workspace]`) instead of adding a wrapper, so the DOM, the reveal animation and every selector stay as they were. `display: contents` wrappers were tried and rejected: they break the reveal animation. +- Pipeline state stays on `DataHandlingComponent`, whose public API (`goToSection`, `progressStateChange`) and fields the studio and specs use. Narrow sections take `input()`/`output()` (how-to strip, terminal). Sections that read much of the pipeline inject `DATA_HANDLING` (`data-handling.context.ts`, provided by the panel with `useExisting`) and use `ChangeDetectionStrategy.Eager`, so they refresh exactly when the inline template did. +- Styles move with the elements that only a section renders. Rules shared with the panel (tables, empty states, search, `button:disabled`) are repeated in the section's stylesheet in the panel's cascade order. The panel's icon nudge (`.stats-panel-shell i`) is repeated as `:host-context(.stats-panel-shell) i`; ancestor selectors become `:host-context(...)`. +- Pure logic moves to `data-handling/utils/*` with specs (formatting, summary pivot, empty summary view, preprocessing state, documentation HTML, transformation drafts); shared types live in `data-handling.models.ts`. + +Consequences: Each extraction was checked against master with a local DOM + computed-style + screenshot comparison of the pipeline in seven scenarios. Sections that `@ViewChildren` queries reach into (the categorical transformation card's `ruleModal`s) stay in the panel until that seam is redesigned. Repeated shared CSS is a known cost; a shared `@import` partial is the follow-up if more sections need it. + +Files affected: `src/app/pages/experiment-studio/data-handling/**`, `src/app/pages/experiment-studio/experiment-studio.component.*`, `AGENTS.md`, `docs/context/module-index.md`, `docs/context/conventions.md`, `docs/context/risk-register.md`, `docs/context/decision-log.md`. + +Date: 2026-10-23. diff --git a/docs/context/module-index.md b/docs/context/module-index.md index 40f3edcac..43887aba9 100644 --- a/docs/context/module-index.md +++ b/docs/context/module-index.md @@ -42,7 +42,7 @@ Notes: Stored historical experiment payloads may depend on legacy algorithm keys ## `src/app/pages/experiment-studio` Purpose: Experiment composition workflow. -Key files: `experiment-studio.component.*`, `variables-panel/*`, `data-handling/*`, `algorithm-panel/*`, `execution-panel/*` (result view + docked setup summary), `stepper/*`, `shared/*`, `guide/*`. +Key files: `experiment-studio.component.*`, `variables-panel/*`, `data-handling/*` (panel + `summary-workspace/`, `pipeline-terminal/`, `how-to-strip/`, `kmeans-cluster-source/` sections, `data-handling.models.ts`, `data-handling.context.ts`, pure `utils/`), `algorithm-panel/*`, `execution-panel/*` (result view + docked setup summary), `stepper/*`, `shared/*`, `guide/*`. Used by: `/experiment-studio` route and dashboard edit flows. Rules: Keep page-specific UI here; use `ExperimentStudioService` for shared selection/run state. Tests: Component specs live near components. From c621b04d97151186a19f4501d3605ccd90cc7b32 Mon Sep 17 00:00:00 2001 From: "K.Filippopolitis" <56073635+KFilippopolitis@users.noreply.github.com> Date: Fri, 23 Oct 2026 23:28:05 +0300 Subject: [PATCH 09/31] refactor(data-handling): share panel/table/search styles through CSS partials The summary workspace repeated 22 rules of the panel's stylesheets. They now live once in data-handling/styles/ and are @imported by both: - statistics-panel.shared.css: the statistics panel frame and code chip - search-and-empty-states.shared.css: inline search fields and empty states - tables.shared.css: data tables, the statistics table frame, row errors, disabled buttons (and their 768px overrides) The panel imports each partial at the top of the sheet the rules came from, so they move only within that file; selector lists that mixed shared and panel-only selectors (.preprocessing-search/.table-input/.table-select, .empty-state, button:disabled, .data-table td:first-child) were split so only the shared selector moved. The workspace imports all three ahead of its own rules (no rule of its own targets the same element at equal specificity before them). summary-workspace.component.css: 16.4 kB -> 12.5 kB. Harness: identical DOM, computed styles and screenshots against master. --- .../data-handling/data-handling.component.css | 16 +- .../data-handling.preprocessing.css | 47 +---- .../data-handling/data-handling.results.css | 83 +------- .../styles/search-and-empty-states.shared.css | 84 ++++++++ .../styles/statistics-panel.shared.css | 17 ++ .../data-handling/styles/tables.shared.css | 91 +++++++++ .../summary-workspace.component.css | 189 +----------------- 7 files changed, 208 insertions(+), 319 deletions(-) create mode 100644 src/app/pages/experiment-studio/data-handling/styles/search-and-empty-states.shared.css create mode 100644 src/app/pages/experiment-studio/data-handling/styles/statistics-panel.shared.css create mode 100644 src/app/pages/experiment-studio/data-handling/styles/tables.shared.css diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.css b/src/app/pages/experiment-studio/data-handling/data-handling.component.css index d76e12df5..191a3920a 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.css +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.css @@ -1,3 +1,5 @@ +@import './styles/statistics-panel.shared.css'; + .workflow-section, .pipeline-node { scroll-margin-top: calc( @@ -29,20 +31,6 @@ transform: translateY(1px); } -.statistics-panel-code { - padding: 1px 6px; - border-radius: var(--radius-sm); - background: rgba(43, 51, 233, 0.06); - color: var(--primary-dark); - font-family: var(--font-mono); - font-size: 0.72rem; - font-weight: 500; -} - -.statistics-panel { - padding: 16px 20px 20px; -} - /* ========================================================================= VERTICAL NODE-PIPELINE CANVAS ========================================================================= */ diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.preprocessing.css b/src/app/pages/experiment-studio/data-handling/data-handling.preprocessing.css index 25d9a17e5..877e97318 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.preprocessing.css +++ b/src/app/pages/experiment-studio/data-handling/data-handling.preprocessing.css @@ -11,6 +11,8 @@ Shared/one-off overrides that must win over styles.css live in the file that owns the rule's section, never duplicated. */ +@import './styles/search-and-empty-states.shared.css'; + .workflow-section-body.open { display: block; } @@ -76,8 +78,7 @@ outline-offset: -2px; } -.empty-state, -.empty-state-block { +.empty-state { margin: 12px 0; text-align: center; font-size: 0.9rem; @@ -95,22 +96,6 @@ justify-content: center; } -/* The icon is the only cue that a station is idle, so it must stay above 3:1. */ -.empty-state-block i { - font-size: 2.25rem; - color: var(--primary-color); - background: color-mix(in srgb, var(--primary-light) 18%, white 82%); - padding: 18px; - border-radius: 50%; - border: 1px solid var(--studio-card-border, rgba(43, 51, 233, 0.1)); - box-shadow: none; -} - -/* One notch below .station-card-title so the panel heading still leads. */ -.empty-state-block p { margin: 0; font-weight: 600; color: var(--text-main); } - -.empty-state-block span { font-size: 0.85rem; color: var(--text-muted); } - /* Glassmorphic feedback cards */ .glass-feedback { margin: 0 0 12px; @@ -135,15 +120,6 @@ to { opacity: 1; transform: translateY(0); } } -/* The preview card reads as the informational feedback variant the same way its - siblings do - through its own fill and edge, not a side stripe. */ -.search-inline { - position: relative; - max-width: 280px; - width: 100%; -} - -.preprocessing-search, .table-input, .table-select { width: 100%; @@ -161,7 +137,6 @@ background 0.2s ease; } -.preprocessing-search:focus-visible, .table-input:focus-visible, .table-select:focus-visible { outline: none; @@ -170,22 +145,6 @@ background: var(--card-bg); } -/* Rail search: pill with a leading glyph, matching the tray/button vocabulary. */ -.search-inline .preprocessing-search { - border-radius: 999px; - padding-left: 30px; -} - -.search-inline .search-inline-icon { - position: absolute; - left: 11px; - top: 50%; - transform: translateY(-50%); - color: var(--text-muted); - font-size: 0.72rem; - pointer-events: none; -} - /* Documentation: the algorithm configuration panel's disclosure, copied. A plain label with a CSS chevron that turns, and the text carried by a left rule instead of a card. The prefix stays because these styles are component-scoped, so this file diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.results.css b/src/app/pages/experiment-studio/data-handling/data-handling.results.css index 4b3b9315f..052609daf 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.results.css +++ b/src/app/pages/experiment-studio/data-handling/data-handling.results.css @@ -7,6 +7,8 @@ order). This file is LAST in the styleUrl array, which is why the responsive overrides live here. */ +@import './styles/tables.shared.css'; + .toggle-switch { position: relative; display: inline-block; @@ -67,17 +69,8 @@ transform: translateX(20px); } -.row-error { - margin-top: 5px; - color: #b91c1c; - font-size: 0.75rem; - font-weight: 600; - text-align: left; -} - .table-input:disabled, -.table-select:disabled, -button:disabled { +.table-select:disabled { opacity: 0.55; cursor: not-allowed; pointer-events: none; @@ -90,70 +83,10 @@ button:disabled { color: color-mix(in srgb, var(--text-muted) 74%, white 26%); } -.table-wrapper { - padding: 10px 12px 12px; -} - -.data-table { - width: 100%; - border-collapse: separate; - border-spacing: 0; - background: var(--card-bg); - border: 1px solid var(--studio-card-border, rgba(43, 51, 233, 0.1)); - border-radius: 8px; - overflow: hidden; -} - -.data-table th { - position: sticky; - top: 0; - z-index: 3; - background: var(--studio-card-header-bg); - color: var(--text-muted); - text-align: center; - padding: 9px 14px; - font-size: 0.72rem; - font-weight: 700; - letter-spacing: var(--letter-spacing-config); - text-transform: uppercase; - border-bottom: 1px solid var(--studio-card-header-border); - box-shadow: none; -} - -.data-table td { - text-align: center; - padding: 9px 14px; - font-size: 0.85rem; - color: var(--text-main); - border-bottom: 1px solid color-mix(in srgb, var(--primary-color) 6%, transparent); - font-variant-numeric: tabular-nums; -} - -.data-table tbody tr:last-child td { - border-bottom: 0; -} - -.data-table tbody tr:hover { - background-color: rgba(43, 51, 233, 0.03); - transition: background-color 0.2s ease; -} - -.data-table td:first-child { - font-weight: 600; - color: var(--text-muted); - text-align: left; - padding-left: 16px; -} - /* Rules below are prefixed with .statistics-table-wrapper so they outrank the --numeric overrides above: emulated encapsulation adds one attribute per compound selector, so a shorter selector loses regardless of order. */ -/* The wrapper owns the card edge; the .data-table frame drew a second box inside it. */ -.statistics-table-wrapper { - padding: 0; -} - .hidden-charts-for-export { position: fixed; top: -9999px; @@ -194,10 +127,6 @@ button:disabled { padding: 12px; } - .search-inline { - max-width: none; - } - .longitudinal-controls { flex-direction: column; align-items: stretch; @@ -208,12 +137,6 @@ button:disabled { gap: 8px; } - .data-table th, - .data-table td { - padding: 6px 8px; - font-size: 12px; - } - } /* ===== TRANSFORMATION: CREATE CATEGORICAL COLUMN ===== diff --git a/src/app/pages/experiment-studio/data-handling/styles/search-and-empty-states.shared.css b/src/app/pages/experiment-studio/data-handling/styles/search-and-empty-states.shared.css new file mode 100644 index 000000000..30810447c --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/styles/search-and-empty-states.shared.css @@ -0,0 +1,84 @@ +/* Shared by the Data Handling panel and its section components: Inline search fields and empty-state blocks. + Imported at the top of each user's stylesheet (the panel imports it from the sheet + these rules came from), so every component renders them identically. */ + +.empty-state-block { + margin: 12px 0; + text-align: center; + font-size: 0.9rem; + color: var(--text-muted); + padding: 24px; + background: var(--studio-card-header-bg); + border: 1px solid var(--studio-card-border, rgba(43, 51, 233, 0.1)); + box-shadow: none; + border-radius: var(--radius-lg, 12px); + display: flex; + flex-direction: column; + align-items: center; + gap: 12px; + min-height: 160px; + justify-content: center; +} + +/* The icon is the only cue that a station is idle, so it must stay above 3:1. */ +.empty-state-block i { + font-size: 2.25rem; + color: var(--primary-color); + background: color-mix(in srgb, var(--primary-light) 18%, white 82%); + padding: 18px; + border-radius: 50%; + border: 1px solid var(--studio-card-border, rgba(43, 51, 233, 0.1)); + box-shadow: none; +} + +/* One notch below .station-card-title so the panel heading still leads. */ +.empty-state-block p { margin: 0; font-weight: 600; color: var(--text-main); } + +.empty-state-block span { font-size: 0.85rem; color: var(--text-muted); } + +/* The preview card reads as the informational feedback variant the same way its + siblings do - through its own fill and edge, not a side stripe. */ +.search-inline { + position: relative; + max-width: 280px; + width: 100%; +} + +.preprocessing-search { + width: 100%; + box-sizing: border-box; + min-height: 34px; + border: 1px solid rgba(43, 51, 233, 0.15); + background: rgba(255, 255, 255, 0.9); + color: var(--text-main); + border-radius: 8px; + padding: 6px 9px; + font-size: 0.8rem; + transition: + border-color 0.2s ease, + box-shadow 0.2s ease, + background 0.2s ease; +} + +.preprocessing-search:focus-visible { + outline: none; + border-color: var(--primary-color); + box-shadow: 0 0 0 3px color-mix(in srgb, var(--primary-color) 18%, transparent); + background: var(--card-bg); +} + +/* Rail search: pill with a leading glyph, matching the tray/button vocabulary. */ +.search-inline .preprocessing-search { + border-radius: 999px; + padding-left: 30px; +} + +.search-inline .search-inline-icon { + position: absolute; + left: 11px; + top: 50%; + transform: translateY(-50%); + color: var(--text-muted); + font-size: 0.72rem; + pointer-events: none; +} diff --git a/src/app/pages/experiment-studio/data-handling/styles/statistics-panel.shared.css b/src/app/pages/experiment-studio/data-handling/styles/statistics-panel.shared.css new file mode 100644 index 000000000..0dbe13296 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/styles/statistics-panel.shared.css @@ -0,0 +1,17 @@ +/* Shared by the Data Handling panel and its section components: The statistics panel frame and its code chip. + Imported at the top of each user's stylesheet (the panel imports it from the sheet + these rules came from), so every component renders them identically. */ + +.statistics-panel-code { + padding: 1px 6px; + border-radius: var(--radius-sm); + background: rgba(43, 51, 233, 0.06); + color: var(--primary-dark); + font-family: var(--font-mono); + font-size: 0.72rem; + font-weight: 500; +} + +.statistics-panel { + padding: 16px 20px 20px; +} diff --git a/src/app/pages/experiment-studio/data-handling/styles/tables.shared.css b/src/app/pages/experiment-studio/data-handling/styles/tables.shared.css new file mode 100644 index 000000000..9c7804808 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/styles/tables.shared.css @@ -0,0 +1,91 @@ +/* Shared by the Data Handling panel and its section components: Data tables, the statistics table frame, inline row errors and disabled buttons. + Imported at the top of each user's stylesheet (the panel imports it from the sheet + these rules came from), so every component renders them identically. */ + +.row-error { + margin-top: 5px; + color: #b91c1c; + font-size: 0.75rem; + font-weight: 600; + text-align: left; +} + +button:disabled { + opacity: 0.55; + cursor: not-allowed; + pointer-events: none; +} + +.table-wrapper { + padding: 10px 12px 12px; +} + +.data-table { + width: 100%; + border-collapse: separate; + border-spacing: 0; + background: var(--card-bg); + border: 1px solid var(--studio-card-border, rgba(43, 51, 233, 0.1)); + border-radius: 8px; + overflow: hidden; +} + +.data-table th { + position: sticky; + top: 0; + z-index: 3; + background: var(--studio-card-header-bg); + color: var(--text-muted); + text-align: center; + padding: 9px 14px; + font-size: 0.72rem; + font-weight: 700; + letter-spacing: var(--letter-spacing-config); + text-transform: uppercase; + border-bottom: 1px solid var(--studio-card-header-border); + box-shadow: none; +} + +.data-table td { + text-align: center; + padding: 9px 14px; + font-size: 0.85rem; + color: var(--text-main); + border-bottom: 1px solid color-mix(in srgb, var(--primary-color) 6%, transparent); + font-variant-numeric: tabular-nums; +} + +.data-table tbody tr:last-child td { + border-bottom: 0; +} + +.data-table tbody tr:hover { + background-color: rgba(43, 51, 233, 0.03); + transition: background-color 0.2s ease; +} + +.data-table td:first-child { + font-weight: 600; + color: var(--text-muted); + text-align: left; + padding-left: 16px; +} + +/* The wrapper owns the card edge; the .data-table frame drew a second box inside it. */ +.statistics-table-wrapper { + padding: 0; +} + +@media (max-width: 768px) { + .search-inline { + max-width: none; + } +} + +@media (max-width: 768px) { + .data-table th, + .data-table td { + padding: 6px 8px; + font-size: 12px; + } +} diff --git a/src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.css b/src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.css index a5287c9aa..c591f2a7a 100644 --- a/src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.css +++ b/src/app/pages/experiment-studio/data-handling/summary-workspace/summary-workspace.component.css @@ -1,7 +1,11 @@ -/* The summary workspace's view of the Data Handling stylesheets. Rules for elements - that only live here moved from the panel; shared ones (tables, empty states, - search) are repeated, in the panel's cascade order: component, preprocessing, - results. */ +/* The summary workspace's slice of the Data Handling stylesheets, in the panel's + cascade order (component, preprocessing, results). Rules both render (panel + heading, search and empty states, tables) come from the shared partials in + ../styles/, which the panel imports too; the rest moved here from the panel. */ + +@import '../styles/statistics-panel.shared.css'; +@import '../styles/search-and-empty-states.shared.css'; +@import '../styles/tables.shared.css'; /* The panel nudges every icon 1px down (`.stats-panel-shell i`); the rule is scoped to the panel's own template, so it is repeated here for this one. */ @@ -95,16 +99,6 @@ font-size: 1rem; } -.statistics-panel-code { - padding: 1px 6px; - border-radius: var(--radius-sm); - background: rgba(43, 51, 233, 0.06); - color: var(--primary-dark); - font-family: var(--font-mono); - font-size: 0.72rem; - font-weight: 500; -} - .summary-export-group { position: relative; display: inline-flex; @@ -119,10 +113,6 @@ transform: rotate(180deg); } -.statistics-panel { - padding: 16px 20px 20px; -} - /* Every workspace sits in a pipeline node's section and inherits the node's frame; strip the inner card chrome so only one border is visible. */ :host-context(.pipeline-node) .statistics-workspace { @@ -237,101 +227,6 @@ } } -.empty-state-block { - margin: 12px 0; - text-align: center; - font-size: 0.9rem; - color: var(--text-muted); - padding: 24px; - background: var(--studio-card-header-bg); - border: 1px solid var(--studio-card-border, rgba(43, 51, 233, 0.1)); - box-shadow: none; - border-radius: var(--radius-lg, 12px); - display: flex; - flex-direction: column; - align-items: center; - gap: 12px; - min-height: 160px; - justify-content: center; -} - -/* The icon is the only cue that a station is idle, so it must stay above 3:1. */ -.empty-state-block i { - font-size: 2.25rem; - color: var(--primary-color); - background: color-mix(in srgb, var(--primary-light) 18%, white 82%); - padding: 18px; - border-radius: 50%; - border: 1px solid var(--studio-card-border, rgba(43, 51, 233, 0.1)); - box-shadow: none; -} - -/* One notch below .station-card-title so the panel heading still leads. */ -.empty-state-block p { margin: 0; font-weight: 600; color: var(--text-main); } - -.empty-state-block span { font-size: 0.85rem; color: var(--text-muted); } - -/* The preview card reads as the informational feedback variant the same way its - siblings do - through its own fill and edge, not a side stripe. */ -.search-inline { - position: relative; - max-width: 280px; - width: 100%; -} - -.preprocessing-search { - width: 100%; - box-sizing: border-box; - min-height: 34px; - border: 1px solid rgba(43, 51, 233, 0.15); - background: rgba(255, 255, 255, 0.9); - color: var(--text-main); - border-radius: 8px; - padding: 6px 9px; - font-size: 0.8rem; - transition: - border-color 0.2s ease, - box-shadow 0.2s ease, - background 0.2s ease; -} - -.preprocessing-search:focus-visible { - outline: none; - border-color: var(--primary-color); - box-shadow: 0 0 0 3px color-mix(in srgb, var(--primary-color) 18%, transparent); - background: var(--card-bg); -} - -/* Rail search: pill with a leading glyph, matching the tray/button vocabulary. */ -.search-inline .preprocessing-search { - border-radius: 999px; - padding-left: 30px; -} - -.search-inline .search-inline-icon { - position: absolute; - left: 11px; - top: 50%; - transform: translateY(-50%); - color: var(--text-muted); - font-size: 0.72rem; - pointer-events: none; -} - -.row-error { - margin-top: 5px; - color: #b91c1c; - font-size: 0.75rem; - font-weight: 600; - text-align: left; -} - -button:disabled { - opacity: 0.55; - cursor: not-allowed; - pointer-events: none; -} - /* A quiet callout: secondary text with an accent edge, the same edge the statistics table uses for its section bands. */ .federated-info-note { @@ -363,55 +258,6 @@ button:disabled { line-height: 1.45; } -.table-wrapper { - padding: 10px 12px 12px; -} - -.data-table { - width: 100%; - border-collapse: separate; - border-spacing: 0; - background: var(--card-bg); - border: 1px solid var(--studio-card-border, rgba(43, 51, 233, 0.1)); - border-radius: 8px; - overflow: hidden; -} - -.data-table th { - position: sticky; - top: 0; - z-index: 3; - background: var(--studio-card-header-bg); - color: var(--text-muted); - text-align: center; - padding: 9px 14px; - font-size: 0.72rem; - font-weight: 700; - letter-spacing: var(--letter-spacing-config); - text-transform: uppercase; - border-bottom: 1px solid var(--studio-card-header-border); - box-shadow: none; -} - -.data-table td { - text-align: center; - padding: 9px 14px; - font-size: 0.85rem; - color: var(--text-main); - border-bottom: 1px solid color-mix(in srgb, var(--primary-color) 6%, transparent); - font-variant-numeric: tabular-nums; -} - -.data-table tbody tr:last-child td { - border-bottom: 0; -} - -.data-table tbody tr:hover { - background-color: rgba(43, 51, 233, 0.03); - transition: background-color 0.2s ease; -} - -.data-table td:first-child, .metric-cell { font-weight: 600; color: var(--text-muted); @@ -477,11 +323,6 @@ button:disabled { padding-left: 11px; } -/* The wrapper owns the card edge; the .data-table frame drew a second box inside it. */ -.statistics-table-wrapper { - padding: 0; -} - .statistics-table-wrapper .statistics-table { border: 0; border-radius: 0; @@ -578,17 +419,3 @@ button:disabled { padding: 10px 12px; box-shadow: 0 4px 24px rgba(15, 23, 42, 0.04); } - -@media (max-width: 768px) { - .search-inline { - max-width: none; - } -} - -@media (max-width: 768px) { - .data-table th, - .data-table td { - padding: 6px 8px; - font-size: 12px; - } -} From bf617669382d55f3fbdaaa9d9b21a9b2a60db922 Mon Sep 17 00:00:00 2001 From: "K.Filippopolitis" <56073635+KFilippopolitis@users.noreply.github.com> Date: Fri, 23 Oct 2026 23:32:19 +0300 Subject: [PATCH 10/31] refactor(data-handling): find category rule builders by rule, not by position The panel located each derived-column category's filter builder through @ViewChildren('ruleModal') and a running index over drafts x rules, which only works while every builder is rendered by the panel's own template, in card order. data-handling/transformation/rule-builder-registry.ts adds the seam that lets the categorical card move into its own component: - RuleBuilderRegistry (provided by the panel): rule -> live builder; - RuleBuilderForDirective (app-filter-config-modal[ruleBuilderFor]): the builder registers itself for the rule it edits, follows a row's rule changes and unregisters on destroy (ngOnChanges, so it is findable right after detectChanges, which addTransformationRule relies on). commitTransformationRuleFilters, finishTransformationRule and addTransformationRule look the builder up by rule (ruleModalAt -> ruleBuilderAt); QueryList/ViewChildren are gone from the panel. Specs: the 19 sites that assigned fake QueryLists now call a local useRuleBuilders(component, [...]) helper (same positional fakes, registered by rule); the two that read real builders use component.ruleBuilders.get(rule). New rule-builder-registry spec (3). Harness: identical against master. --- .../data-handling.component.html | 6 +- .../data-handling.component.spec.ts | 76 ++++++++++--------- .../data-handling/data-handling.component.ts | 57 ++++++-------- .../data-handling.pipeline.spec.ts | 29 +++++-- .../rule-builder-registry.spec.ts | 74 ++++++++++++++++++ .../transformation/rule-builder-registry.ts | 54 +++++++++++++ 6 files changed, 220 insertions(+), 76 deletions(-) create mode 100644 src/app/pages/experiment-studio/data-handling/transformation/rule-builder-registry.spec.ts create mode 100644 src/app/pages/experiment-studio/data-handling/transformation/rule-builder-registry.ts diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.html b/src/app/pages/experiment-studio/data-handling/data-handling.component.html index 2d3c51545..e641cb30d 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.html +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.html @@ -1200,7 +1200,7 @@

    -
    +
    {{ variableLabel(variable) }}

    DataHandlingComponent) }, RuleBuilderRegistry, + SectionAnchors, ], templateUrl: './data-handling.component.html', // Order is the cascade: shell + pipeline canvas, then preprocessing stations, @@ -144,18 +144,8 @@ export class DataHandlingComponent implements OnDestroy { readonly variables = input([]); readonly filters = input([]); readonly progressStateChange = output(); - @ViewChild('sourceSection') - sourceSection?: ElementRef; - @ViewChild('rawSection') - rawSection?: ElementRef; - @ViewChild('setupSection') - setupSection?: ElementRef; - @ViewChild('filtersSection') - filtersSection?: ElementRef; - @ViewChild('processedSection') - processedSection?: ElementRef; - @ViewChild('transformationSection') - transformationSection?: ElementRef; + /** Scroll targets of goToSection, registered by the elements themselves. */ + private readonly sectionAnchors = inject(SectionAnchors); /** Each category's live filter builder, by rule (see RuleBuilderRegistry). */ readonly ruleBuilders = inject(RuleBuilderRegistry); @@ -2906,29 +2896,8 @@ export class DataHandlingComponent implements OnDestroy { return this.processedSummary; } - private sectionElement(section: SectionKey): ElementRef | undefined { - switch (section) { - case 'source': - return this.sourceSection; - case 'raw': - return this.rawSection; - case 'setup': - return this.setupSection; - case 'filters': - return this.filtersSection; - case 'processed': - return this.processedSection; - case 'transformation': - return this.transformationSection; - default: { - const _exhaustive: never = section; - return _exhaustive; - } - } - } - private scrollSectionIntoView(section: SectionKey): void { - const element = this.sectionElement(section)?.nativeElement; + const element = this.sectionAnchors.get(section)?.nativeElement; if (!element) return; requestAnimationFrame(() => { diff --git a/src/app/pages/experiment-studio/data-handling/section-anchors.spec.ts b/src/app/pages/experiment-studio/data-handling/section-anchors.spec.ts new file mode 100644 index 000000000..21ffb33de --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/section-anchors.spec.ts @@ -0,0 +1,34 @@ +import { Component, signal } from '@angular/core'; +import { TestBed } from '@angular/core/testing'; +import { SectionKey } from './data-handling.models'; +import { SectionAnchorDirective, SectionAnchors } from './section-anchors'; + +@Component({ + imports: [SectionAnchorDirective], + providers: [SectionAnchors], + template: `@if (shown()) {
    }`, +}) +class HostComponent { + readonly shown = signal(true); + readonly section = signal('raw'); +} + +describe('SectionAnchorDirective', () => { + it('registers its element for the section, follows changes and leaves no attribute', () => { + const fixture = TestBed.createComponent(HostComponent); + fixture.detectChanges(); + const anchors = fixture.debugElement.injector.get(SectionAnchors); + const el = (fixture.nativeElement as HTMLElement).querySelector('#target')!; + expect(anchors.get('raw')?.nativeElement).toBe(el); + expect(el.getAttributeNames().some((n) => n.toLowerCase().includes('anchor'))).toBe(false); + + fixture.componentInstance.section.set('processed'); + fixture.detectChanges(); + expect(anchors.get('raw')).toBeUndefined(); + expect(anchors.get('processed')?.nativeElement).toBe(el); + + fixture.componentInstance.shown.set(false); + fixture.detectChanges(); + expect(anchors.get('processed')).toBeUndefined(); + }); +}); diff --git a/src/app/pages/experiment-studio/data-handling/section-anchors.ts b/src/app/pages/experiment-studio/data-handling/section-anchors.ts new file mode 100644 index 000000000..093f534a6 --- /dev/null +++ b/src/app/pages/experiment-studio/data-handling/section-anchors.ts @@ -0,0 +1,45 @@ +import { Directive, ElementRef, Injectable, OnChanges, OnDestroy, inject, input } from '@angular/core'; +import { SectionKey } from './data-handling.models'; + +/** + * The element `goToSection` scrolls to for each pipeline section. Provided by the Data + * Handling panel; elements register themselves through SectionAnchorDirective wherever + * they are rendered (the panel or one of its station components). + */ +@Injectable() +export class SectionAnchors { + private readonly anchors = new Map>(); + + set(section: SectionKey, element: ElementRef): void { + this.anchors.set(section, element); + } + + delete(section: SectionKey, element: ElementRef): void { + if (this.anchors.get(section) === element) this.anchors.delete(section); + } + + get(section: SectionKey): ElementRef | undefined { + return this.anchors.get(section); + } +} + +/** Marks its element as the scroll target of a pipeline section. Bind it as a property + * (`[dhSectionAnchor]="'raw'"`) so no attribute is added to the DOM. */ +@Directive({ selector: '[dhSectionAnchor]' }) +export class SectionAnchorDirective implements OnChanges, OnDestroy { + readonly dhSectionAnchor = input.required(); + + private readonly anchors = inject(SectionAnchors); + private readonly element = inject>(ElementRef); + private section: SectionKey | null = null; + + ngOnChanges(): void { + if (this.section) this.anchors.delete(this.section, this.element); + this.section = this.dhSectionAnchor(); + this.anchors.set(this.section, this.element); + } + + ngOnDestroy(): void { + if (this.section) this.anchors.delete(this.section, this.element); + } +} From 4d55294d902ecc3c61eefaae4c0676218e5cce84 Mon Sep 17 00:00:00 2001 From: "K.Filippopolitis" <56073635+KFilippopolitis@users.noreply.github.com> Date: Fri, 23 Oct 2026 23:39:44 +0300 Subject: [PATCH 12/31] refactor(data-handling): extract the Raw data step into SourceDataStationComponent Step 1 (read-only snapshot: header, Records out, the source summary workspace) moves to data-handling/source-data-station/, attached to its pipeline-node section (section[app-source-data-station]); the panel keeps the section element and its state classes. Its scroll target registers through [dhSectionAnchor]. The step chrome every station shares (header, icon, title, badge, records out, body, section frame) moves once into styles/pipeline-node.shared.css, imported by the panel at the top of its first sheet (checked: no equal-specificity rule it used to override sits above it) and by the station. The two rules about the snapshot itself move with it as :host(.pipeline-node.is-readonly ...) rules; .pipeline-node .workflow-section-body is repeated for the station as :host(.pipeline-node) .workflow-section-body. Spec: SourceDataStationComponent (3). Harness: identical against master. --- .../data-handling/data-handling.component.css | 192 +--------------- .../data-handling.component.html | 37 +--- .../data-handling/data-handling.component.ts | 3 +- .../data-handling.preprocessing.css | 4 - .../data-handling/data-handling.results.css | 4 - .../source-data-station.component.css | 22 ++ .../source-data-station.component.html | 33 +++ .../source-data-station.component.spec.ts | 56 +++++ .../source-data-station.component.ts | 21 ++ .../styles/pipeline-node.shared.css | 207 ++++++++++++++++++ 10 files changed, 344 insertions(+), 235 deletions(-) create mode 100644 src/app/pages/experiment-studio/data-handling/source-data-station/source-data-station.component.css create mode 100644 src/app/pages/experiment-studio/data-handling/source-data-station/source-data-station.component.html create mode 100644 src/app/pages/experiment-studio/data-handling/source-data-station/source-data-station.component.spec.ts create mode 100644 src/app/pages/experiment-studio/data-handling/source-data-station/source-data-station.component.ts create mode 100644 src/app/pages/experiment-studio/data-handling/styles/pipeline-node.shared.css diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.css b/src/app/pages/experiment-studio/data-handling/data-handling.component.css index 191a3920a..37cef6998 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.css +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.css @@ -1,6 +1,6 @@ @import './styles/statistics-panel.shared.css'; +@import './styles/pipeline-node.shared.css'; -.workflow-section, .pipeline-node { scroll-margin-top: calc( var(--header-height, 64px) + var(--studio-sub-header-height, 44px) + var(--warning-banner-space, 0px) + 16px @@ -124,11 +124,6 @@ max-width: 100%; } -/* No body to divide from while the snapshot is hidden. */ -.pipeline-node.is-readonly:not(.is-active) .pipeline-node-header { - border-bottom: none; -} - .btn-add-step { display: inline-flex; align-items: center; @@ -184,102 +179,6 @@ background: color-mix(in srgb, var(--danger) 8%, white); } -.pipeline-node-header { - display: flex; - align-items: center; - justify-content: space-between; - padding: 14px 18px; - background: var(--studio-card-header-bg); - border-bottom: 1px solid rgba(43, 51, 233, 0.08); - gap: 12px; - cursor: pointer; -} - -.pipeline-node-header-left { - display: flex; - align-items: center; - gap: 12px; - min-width: 0; -} - -/* The step title is the header's keyboard and screen-reader toggle; it keeps the h3 look. */ -.pipeline-node-toggle { - appearance: none; - padding: 0; - border: 0; - background: none; - font: inherit; - color: inherit; - text-align: left; - cursor: pointer; - border-radius: 4px; -} - -/* Fold cue: the whole header toggles its step. */ -.pipeline-node-chevron { - color: var(--text-muted); - font-size: 0.8rem; - transition: transform 0.2s ease, color 0.2s ease; -} - -/* Outranks `.stats-panel-shell i`, whose 1px nudge it keeps. */ -.pipeline-node-header .pipeline-node-chevron.is-open { - transform: translateY(1px) rotate(180deg); -} - -.pipeline-node-header:hover .pipeline-node-chevron { - color: var(--primary-color); -} - -.pipeline-node-icon { - width: 38px; - height: 38px; - border-radius: var(--radius-md, 8px); - background: var(--card-bg); - color: var(--primary-color); - display: flex; - align-items: center; - justify-content: center; - font-size: 0.95rem; - flex-shrink: 0; - border: 0; - box-shadow: inset 0 0 0 1px var(--studio-card-border); -} - -.pipeline-node-title-group h3 { - font-size: 0.96rem; - font-weight: 700; - color: var(--text-main); - margin: 0; - line-height: 1.3; -} - -.pipeline-node-subtitle { - font-size: 0.78rem; - color: var(--text-muted); - line-height: 1.35; -} - -.pipeline-node-header-right { - display: flex; - align-items: center; - gap: 10px; - flex-shrink: 0; -} - -.pipeline-status-badge { - display: inline-flex; - align-items: center; - gap: 5px; - font-size: 0.72rem; - font-weight: 600; - padding: 2px 9px; - border: 0; - border-radius: 999px; - letter-spacing: var(--letter-spacing-config); - white-space: nowrap; -} - /* Not in the request: neutral outline. */ .pipeline-status-badge.optional { background: transparent; @@ -304,21 +203,6 @@ color: #78350f; } -/* Read only: plain label, no pill. */ -.pipeline-status-badge.readonly { - padding: 0; - background: transparent; - color: var(--text-muted); -} - -.pipeline-status-badge.readonly i { - font-size: 0.66rem; -} - -.pipeline-node-body { - padding: 0; -} - /* Sub-node rail: compact connected tree of active sub-steps, shown while the stage header is collapsed. Rail line + milestone dot echo the pipeline spine. */ /* Collapsed stage contents are inset (~64px a side on the 860px compact card, which @@ -582,29 +466,6 @@ .pipeline-canvas-header { flex-wrap: wrap; } - .pipeline-node-header { - flex-direction: column; - align-items: flex-start; - } - .pipeline-node-header-right { - width: 100%; - justify-content: space-between; - } -} - -.workflow-section { - background: transparent; - border: none; - border-radius: 0; - box-shadow: none; - overflow: visible; - transition: background-color 0.25s ease; - margin-bottom: 0; -} - -.workflow-section-body { - display: none; - padding: 14px 0 16px; } /* Inside a pipeline node the station card draws its own frame; without an inset @@ -613,51 +474,6 @@ padding-inline: 16px; } -.pipeline-node.is-readonly .workflow-section-body { - padding: 0; -} - -/* Stage header extras: the step kicker above the title and the "Records out" figure - before the status badge. */ -.pipeline-node-kicker { - display: block; - font-size: 0.66rem; - font-weight: 700; - letter-spacing: 0.06em; - text-transform: uppercase; - color: var(--text-muted); - line-height: 1.4; -} - -.pipeline-records-out { - display: flex; - flex-direction: column; - align-items: flex-end; - line-height: 1.25; - padding-right: 4px; - font-variant-numeric: tabular-nums; -} - -.pipeline-records-out-label { - font-size: 0.64rem; - font-weight: 700; - letter-spacing: 0.06em; - text-transform: uppercase; - color: var(--text-muted); -} - -.pipeline-records-out-value { - font-size: 0.9rem; - font-weight: 700; - color: var(--text-main); -} - -.pipeline-records-out-value span { - margin-left: 0.25em; - font-weight: 500; - color: var(--text-muted); -} - .pipeline-subnodes-caption { margin: 0 0 8px; font-size: 0.74rem; @@ -703,12 +519,6 @@ background: color-mix(in srgb, var(--success) 12%, white); } -@media (max-width: 720px) { - .pipeline-records-out { - display: none; - } -} - @media (max-width: 720px) { .pipeline-subnodes-tree { max-width: none; diff --git a/src/app/pages/experiment-studio/data-handling/data-handling.component.html b/src/app/pages/experiment-studio/data-handling/data-handling.component.html index e2a00d5e5..057e724a5 100644 --- a/src/app/pages/experiment-studio/data-handling/data-handling.component.html +++ b/src/app/pages/experiment-studio/data-handling/data-handling.component.html @@ -115,44 +115,11 @@

    {{ block.code }}

    -
    -
    -
    -
    -Step 1 -

    - Look at your selected variables exactly as stored. Nothing is changed here. -
    -
    -
    - @if (recordsOut.source !== null) { - Records out{{ recordsOut.source!.toLocaleString() }} - } - Read only - -
    -
    -
    - -
    - -
    -
    -
    -
    + >