[saqibmanan]

Covers CivicDataLab/DataSpaceFrontend#470 (merge commit 24149ee), merged 2026-10-14 and confirmed live on dev (X-Git-Sha = 84f4b13af57e6f83c5657945665312faa5f86e8e, which is a descendant of the merge commit — DataSpaceFrontend#503 merged on top of it the same day).

What changed

#470 ("Provider Flow Update") is a large provider-dashboard overhaul (168 files — dataset-edit stepper, publications, collaboratives, AI model editing). Almost all of it sits behind provider/org login, which this environment has no credentials for (see "Gaps" below). One piece is public and unauthenticated: a new Next.js route, app/api/website-title/route.ts — GET /api/website-title?url=<url> fetches the given URL server-side and returns its <title>, after checking the param is present and its protocol is http/https.

Category added: api (+ smoke so it runs)

New API-visible behaviour, no UI. Per this repo's own gotcha list, a test needs smoke to be selected on a plain push/PR event — api alone wouldn't run.

Where it landed and why

New file tests/api/smoke/test_api_028_website_title.py, next free number on CI. Uses the existing dev_frontend_client fixture (same one test_api_027_dev_noindex.py uses) — no new fixtures needed.

Proof (red → green)

Collection, under CI's real filter (pytest tests/api/smoke -m "smoke" --collect-only): all 3 new tests collected, not deselected.

Red — parent commit, local serve (§6 "stronger proof for frontend PRs"): built a worktree at 6e62549 (dev's tip just before #470), ran next dev --webpack locally, hit the route:

GET /api/website-title?url=https://example.com -> 404 (route doesn't exist yet)

Green — merge commit, local serve (same worktree, checked out to 24149ee):

GET /api/website-title?url=https://example.com      -> 200 {"title":null}   (outbound fetch blocked in this sandbox, see below)
GET /api/website-title                              -> 400 {"title":null}
GET /api/website-title?url=javascript:alert(1)      -> 400 {"title":null}

Green — against dev (HOME_URL_DEV=https://dev.civicdataspace.in), where outbound fetch isn't sandboxed:

tests/api/smoke/test_api_028_website_title.py::test_website_title_rejects_missing_url PASSED
tests/api/smoke/test_api_028_website_title.py::test_website_title_rejects_non_http_protocol PASSED
tests/api/smoke/test_api_028_website_title.py::test_website_title_fetches_a_real_page_title PASSED
3 passed in 4.05s

The third test asserts the actual fetched title ("Example Domain"), confirmed directly against dev first (curl https://dev.civicdataspace.in/api/website-title?url=https://example.com → {"title":"Example Domain"}) before being written into the test.

Gate correctness — against prod (not yet on main, verified via git merge-base --is-ancestor):

SKIPPED deployed_pr DataSpaceFrontend#470: https://civicdataspace.in runs 92cd5219, which doesn't include 24149ee8 yet

All three self-skip with the expected reason, confirming the deployed_pr gate is correctly wired — safe to merge now, turns on by itself once #470 reaches prod.

Gaps — not covered

• The rest of #470 (dataset-edit stepper, publications, collaboratives, AI model editing — the bulk of the 168-file diff) is entirely behind provider/org login. This is the scheduled cloud routine: no test credentials exist in this environment (TEST_EMAIL_1/TEST_PASSWORD_1 are local/CI-only secrets), so none of it can be exercised or proven here, not even red/green. Existing Page Objects under pages/provider/ (e.g. create_dataset_page.py) predate this restructure (the dataset-edit flow moved to dataset/[id]/edit/{access,metadata,publish,resources,review}) and are likely stale against the new wizard steps. This needs a human/local run with real credentials to assess Page Object drift and write authenticated coverage — flagging rather than guessing blind at a login-gated 168-file surface.
• SSRF shape, noted but not turned into a test: the route fetches an arbitrary caller-supplied URL (checked http://169.254.169.254/latest/meta-data/ and an unreachable localhost port against dev — both return 200 {"title":null}, identical to a normal non-HTML response, so there's no behavioural signal here to assert on from outside, and CivicDataSpace-test has no security marker to carry it anyway per this repo's category table). Worth a look from inside the repo (egress allow-list / IP-range check) rather than from this test suite.
• DataSpaceFrontend#503 (merged in the same 24h window, same author/branch) is pure CSS/layout (border radius, sticky-header offsets, a sidebar height constraint) — no user-visible behaviour change, no test.

Also assessed this run, no new PRs

• DataSpaceBackend#245, #244 (merged in the last 72h) are already covered by #198 (open draft from the previous run) — duplicate check per §8 stopped here.
• IDS-DRR-Frontend#548 — pure GitHub Action version bump (actions/setup-node), no application code — no test.
• ParakhAI-frontend, ParakhAI-Backend, IDS-DRR-Data-Management — no merges into dev in the last 72h.
• Pending sweep (§9): no new open dev PRs crossed the 72h threshold since test(api): metadata export error messages stop echoing caller input (DataSpaceBackend#245) #198's run yesterday; the same set (DataSpaceFrontend#420, DataSpaceBackend#52/#224, ParakhAI-frontend#441-444 already covered; ParakhAI-frontend#440, ParakhAI-Backend#55/#106, IDS-DRR-Frontend#531, IDS-DRR-Data-Management#103 already assessed-and-skipped) is still accurate — all remain open, none closed unmerged.

---

Run context: scheduled cloud routine, no test credentials available in this environment. Everything above is unauthenticated, read-only, and was actually executed (not reasoned about) — nothing in this PR is UNVERIFIED.

[saqibmanan]

Update: this is a real regression, not a flake, and it's bigger than just consumer-smoke. Re-ran both failed jobs once (job-level rerun, not just pytest's own --reruns) — both failed again at the exact same assertions.

Not this PR's either way: the only file this PR touches is tests/api/smoke/test_api_028_website_title.py; neither failure is anywhere near it, and api-smoke (this PR's own job) is green.

Root cause — consumer-smoke (1 failed, same both runs):
test_TC_SEC_03_sort_dropdown looks for a div matching contains(@class,'sort'), a <select>, or a <button> containing the text "Sort" (locators/consumer/locators.py::SECTOR_SORT_DROPDOWN). Checked the live markup on /sectors:

<span class="...">Sort :</span>
<div class="Labelled-module_hidden__ketQN"><label class="Label-module_base__4jZYj">Sort</label></div>
<div class="Select-module_Select__YceVe"><button type="button" ...>...</button></div>

#470 bumped opub-ui to 0.5.6, which replaced the native <select>-based sort control with this component's custom button+listbox (CSS-module hashed classes, no "sort" substring; the "Sort" label is a visually-hidden sibling, not inside the button). None of the locator's three alternatives match the new markup. Not a locator a credential-less run can safely rewrite blind — the listbox's accessible name/role needs checking in a browser, which is also true of every other opub-ui-driven control this bump touched.

Root cause — provider-smoke (11 failed, same both runs): every failure is one of two timeouts: "Timed out waiting for organization dashboard to load (Drafts tab)" or "Timed out waiting for Metadata tab in AI model editor", plus one "Individual dashboard sidebar item not visible: UseCases". This lines up exactly with the gap already flagged in this PR's body — #470 restructured the provider dashboard (dataset-edit stepper, AI model editor, nav) behind a login this environment has no credentials for, so pages/provider/* Page Objects are stale against it. Can't log in here to confirm the exact locator drift, but the pattern (every provider-dashboard entry point now timing out) matches a dashboard-shell/nav change, not 11 unrelated test bugs.

No fix exists to port. Both need someone with credentials (consumer side could be fixed without them, actually — it's unauthenticated — but updating SECTOR_SORT_DROPDOWN correctly needs eyes on the rendered opub-ui markup in a browser, which this sandbox can't do reliably from a one-off curl) to re-walk the affected locators/Page Objects against the new dashboard and sort-control markup. Flagging rather than guessing a patch blind at either.

This PR's own change (api-smoke) is green and unaffected. Will keep watching for further activity but am not spending more reruns on these — both reproduced identically twice, so a third run would only burn CI minutes.