[stephaniehobson]

One-line summary

Add unit, routing and end-to-end tests that check behavior through the DOM, fetch, URLs and the rendered charts, so the coming React, router and react-refetch upgrades can be verified without rewriting the tests. Fix the bugs they found.

Significant changes and points to review

• The Playwright suite runs twice.
  • The required "End-to-end tests" check runs every spec on recorded data in tests/playwright/fixtures/ (292 KB, last 12 weeks). Any page or console error fails the test, and so does any data request with no recording.
  • A new "Live data tests" check runs the same specs against data.firefox.com. The main ruleset doesn't require it, so a weekly upstream data change shows as a red check but doesn't block merge.
  • New npm scripts: test:playwright:recorded, test:playwright:live and record:fixtures. CONTRIBUTING.md has step-by-step instructions for re-recording and for running against live data.
• Unit tests move from Enzyme to React Testing Library 12, the last version that supports React 16. Enzyme has no React 17+ adapter, so it would have blocked the React upgrade.
  • Unit tests fail on any unexpected console.error, so React warnings show up as failures.
  • Dependabot skips major updates of @testing-library/react and @testing-library/dom, which belong to the React upgrade.
• Vitest grows from 2 files / 4 tests to 14 / 80. New coverage: lib/utils.js, container data shaping, metrics-graphics configuration, summary-bar proportions, labels and arrows, date selection, the dashboard and metric fetch states, every route in config.json, and LazyBoundary.
• New recorded-data browser specs check what the charts draw.
  • Every population drawn once, with the matching legend name and color.
  • A region or date change changes the plotted values.
  • Summary-bar hover and click.
  • The full table ranking.
  • A one-point population drawn as a dot.
  • An HTTP 500, and description sanitization.
• Fixes, each with a test that failed before it:
  • Summary-bar hover and click hid every label on main. d3-selection 3 passes the event as the handler's first argument. This is a regression from the dependency update, not yet deployed.
  • A failed metric fetch now shows #metric-overview-fetch-error. Before, the metric stayed blank or kept the previous region's data.
  • Chart.jsx read suggestedMax instead of suggestedYMax. No current metric sets a suggested maximum, so no chart changes today.
  • Summary-bar arrows are now hidden on segments under 4%, as the code intended.
• vite.config.mjs: the dev server's dependency pre-bundling works again. The dependency scan assumed React's automatic JSX runtime and failed on every start. That caused the "first visit 504s (Outdated Optimize Dep)" behavior. CI starts a cold server on every run.
• The live dashboards/ specs keep their assertions, with two changes:
  • "Charts render" now checks for any visible line. A population with a single data point draws a zero-size line path, shown as a dot instead. Live hardware data had one that week.
  • "Metric titles and order" counts .metric-overview instead of the lazily loaded .metric, which made it flaky.
• Documented, not changed: a metric whose populations are all null makes react-metrics-graphics throw. The chart's error boundary then shows a generic "Load error", and the rest of the page renders.
• Data issue: the live hardware data has a 48% "NULL" CPU-speed bucket for the week of 2026-10-12. That looks like an ETL problem.
• Adds docs/execplans/2026-10-21-improve-tests.md.
• Docs: AGENTS.md, CONTRIBUTING.md and docs/architecture/frontend.md are updated for the new test setup.

Issue / Bugzilla link

n/a

Testing

Run npm ci and npm run build:css first. If something else is already on port 3000, use npx vite --port 3001 --strictPort and port 3001 in the URLs below.

• Run npx vite --force. Check the terminal shows no "Failed to run dependency scan", and that http://localhost:3000/dashboard/hardware loads without a reload.
• On http://localhost:3000/dashboard/hardware, under Summary → GPU Vendor, hover the second segment. Check its name and percentage replace the first label.
• Click the narrow "Other" segment in the same bar. Check its label shows without a colored circle under it.
• In DevTools, block requests matching */Worldwide/MAU/*, then load http://localhost:3000/dashboard/user-activity. Check the first metric shows "Error fetching metric" instead of an empty space.
• Run npm run record:fixtures, then git diff --stat tests/playwright/fixtures. Check the recorder works for you, and that the diff is empty or shows only upstream changes.
• Look closely at tests/playwright/fixtureData.js, please. Every recorded and live spec goes through it.