[joshavant]

What Problem This Solves

The repository's acquired local workerd verifier can accept the compromised emSign keys even while the invoking Node runtime is patched.

User Impact

Local Wrangler and applicable Workers tests use workerd 1.20261021.1, which rejects those keys. Cloudflare's managed production runtime remains supplier-owned; dependency installation alone does not establish deployed-fleet remediation.

Why This Change Was Made

Use the existing npm overrides owner and regenerate its lockfile with the declared package manager. Keep Wrangler, Miniflare, source inputs, installation policy and service lifecycle unchanged. Replace every affected nested workerd graph; preserve legitimate private-CA trust.

Evidence

Candidate 4c389d9d9eec85783bd5c763d177c12135e65327, base e4954d7b90c20808646c6f43e698d2359cdc1799; native artifact SHA256 e98c9ce9473a19405f80bfa8979d0b7aabae6807ec43093b8d3a660e319bbd83.

• Actual owning Wrangler baseline accepted 14/14 verified incident endpoints with matching chain/hostname controls. Candidate result: 0/14 accepted; 14/14 underlying CertDistrusted; legitimate public/private HTTPS controls: both pass on baseline and candidate.
• Owner checks: npm run check → 0; npm exec -- wrangler deploy --dry-run --outdir dist-proof → 0. Workers-pool compatibility and nested-engine identity: All installed workerd binaries match the official Linux ARM64 artifact; no old engine graph remains in the final lock. Existing real Worker suites are included above..
• Independent review: canonical independent P0–P3 review completed scoped-clean, no actionable findings; source/lock review is distinct from all-platform qualification. Current-head hosted checks: not yet observed for this new head; consult this PR Checks tab.
• Evidence/artifact manifest: patch SHA256 20f2453362a9f70a912d06d38f94d54f83d368059a0a02d2078ec4d51924eb54; chain/fixture identities and results below. Platform and trust-mode limits: Authenticated latest-public-release vocabulary resolution remains a hosted gate; local check verifies the current retained vocabulary only..

Existing automatic effects: PR: Deploy workflow check job only, npm check/latest-release vocabulary/dry-run; discover/generate/publish/deploy excluded. Main push: plan latest public release vocabulary, optionally generate/check and commit only data/public-vocabulary.json+src/public-vocabulary.ts back to main with GitHub token, then deploy exact tagged Worker revision and verify 100% traffic, recording telemetry/deploy status. The same workflow also schedules maintenance independently. No npm package publication is configured.

No merge, deployment, release, image/package publication or cloud operation was performed during local qualification.

Reproduction and evidence limits

Both revisions use the actual repository-installed Wrangler and its frozen dependency graph, patched Node 24.22.1, identical isolated Pages fetch fixture, historical extra-CA plus system roots, and matching requests. This exercises the acquired local native verifier, not each authenticated application route. No production test seam or trust bypass was added. Each of the 14 endpoint chains passed independent OpenSSL chain/hostname verification and matched verified incident SPKI identities;14chains cover six TLS issuer keys, not the two additional code-signing keys. Supplier/source comparison separately matches the full eight-key denylist. Generic internal-error envelopes are corroborated by 14 exact underlying CertDistrusted log entries.

Fixture SHA256 8cf12c8291a4d9e76b2f1e63dd93ea1a533f9fe174dca7078e9992579bc18f16; additional-CA SHA256 7ddc10d975d45dcb1906a13d984edf8e5fcf13e6877f5a0dbbfd503c4e841e98. No host trust, production service, Docker daemon or Cloudflare setting was changed. Mac/Windows/native x64, outbound WSS, every provider, production managed-runtime identity and fleet uptake remain unverified by this lane.

Matched incident endpoint results

Each row has valid independent chain and exact-hostname verification. Baseline TLS completes; candidate fails with underlying CertDistrusted.

Endpoint	Incident issuer	Baseline / candidate
dv-g3b-emudhra.we-have-a-problem.com	EM DV TLS CA - G3B	accepted / CertDistrusted
dv-g3c-emudhra.we-have-a-problem.com	EM DV TLS CA - G3C	accepted / CertDistrusted
ov-g3b-emudhra.we-have-a-problem.com	EM OV TLS CA - G3B	accepted / CertDistrusted
ov-g3c-emudhra.we-have-a-problem.com	EM OV TLS CA - G3C	accepted / CertDistrusted
ev-g3b-emudhra.we-have-a-problem.com	EM EV TLS CA - G3B	accepted / CertDistrusted
ev-g3c-emudhra.we-have-a-problem.com	EM EV TLS CA - G3C	accepted / CertDistrusted
dv-g3b-aaa-emudhra.we-have-a-problem.com	EM DV TLS CA - G3B	accepted / CertDistrusted
dv-g3c-aaa-emudhra.we-have-a-problem.com	EM DV TLS CA - G3C	accepted / CertDistrusted
ov-g3b-aaa-emudhra.we-have-a-problem.com	EM OV TLS CA - G3B	accepted / CertDistrusted
ov-g3c-aaa-emudhra.we-have-a-problem.com	EM OV TLS CA - G3C	accepted / CertDistrusted
ev-g3b-aaa-emudhra.we-have-a-problem.com	EM EV TLS CA - G3B	accepted / CertDistrusted
ev-g3c-aaa-emudhra.we-have-a-problem.com	EM EV TLS CA - G3C	accepted / CertDistrusted
dv-g3b-noct-emudhra.we-have-a-problem.com	EM DV TLS CA - G3B	accepted / CertDistrusted
dv-g3b-aaa-noct-emudhra.we-have-a-problem.com	EM DV TLS CA - G3B	accepted / CertDistrusted

Chain-controls JSON SHA256 337b73f7d54256bbbc0af661e648ac2db90b55e066a273baab6d1a63609d94ab; full candidate server log SHA256 8465bdb0a04543cfd863ad5e2ce2635db9b83cd7e00dfb209355963927d795ef. Both legitimate controls complete HTTPS under the same additional trust. These are TLS acceptance results, not model/application success at the incident sites.

Hosted qualification result

Hosted check succeeded against integration commit 924e9f68f3788f718d18b7499547a959e9718238 containing this exact head: 540 tests passed and the authenticated latest-public-release vocabulary check passed for release revision 157d8c2bc3ea2fa183afdc11088e69690bf8e64e. Publish/deploy jobs were skipped. This closes the named authenticated vocabulary CI gap, not delivery.

Exact completed job. Log SHA256: 2f2e8d9c5508a8b788d70d5aa5103cf2a71646199a2f0cc935206ed8907e37b0.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[clawsweeper]

Codex review: needs maintainer review before merge.

What this changes

This PR pins workerd and its platform binaries to 1.20261021.1 and updates the matching install-script allowance.

Example: Local Wrangler fetches https://dv-g3b-emudhra.we-have-a-problem.com using the documented additional trust configuration.

• Before: The supplied baseline trace reports successful TLS acceptance.
• After: The supplied candidate trace reports CertDistrusted, while legitimate public and private HTTPS controls still pass.

Review scores

Measure	Result	What it means
Overall readiness	🦞 diamond lobster (5/6)	A focused security repair with shipped-entry-point behavior evidence, exact-head hosted qualification, no persisted-data migration, and no actionable finding.
Proof confidence	🦞 diamond lobster (5/6)	Sufficient (live_output): The supplied Linux ARM64 Wrangler trace exercises the acquired native verifier with identical trust settings: all 14 incident chains accepted on the baseline are rejected by the candidate with CertDistrusted, while legitimate public/private HTTPS controls pass; this establishes local remediation, with other platforms and managed-fleet uptake expressly unverified.
Patch quality	🦞 diamond lobster (5/6)	No actionable review findings were identified.

Product

Kind: Bug fix · Worth it: Yes · Fix scope: Complete
User problem: Local Worker HTTPS requests can accept incident certificate keys despite running under a patched Node runtime.
Reason: The change addresses the demonstrated local verifier failure with a minimal dependency update, and the supplied maintainer-authored classification establishes direction.

Merge readiness

✅ Ready for maintainer review

Keep open: main still uses the affected local verifier, and this PR provides a focused dependency repair with relevant behavior evidence and passing hosted checks; no actionable patch defect was found.

Priority: P2
Reviewed head: 4c389d9d9eec85783bd5c763d177c12135e65327

Before merge

None.

Findings

None.

Agent review details

How this fits together

Wrangler and Miniflare execute the npm-installed native workerd for local Worker development and runtime tests; its TLS verifier decides whether outbound HTTPS connections proceed.

flowchart LR
  A[Npm override and lockfile] --> B[Platform workerd binary]
  C[Wrangler and Miniflare] --> B
  D[Local Worker fetch] --> B
  B --> E[TLS certificate verification]
  E --> F[Accepted HTTPS or rejected handshake]

Loading

Technical review

Best possible solution:

Land the focused workerd pin while retaining the explicit boundary between local verifier remediation and supplier-managed production remediation.

Do we have a high-confidence way to reproduce the issue?

The contributor supplies a matched reproduction against the exact current-main baseline and candidate through installed Wrangler, with independent chain/hostname controls and concrete TLS results; no target code was executed during this review.

Is this the best way to solve the issue?

The npm override is a narrow repair that replaces the shared native verifier for both consumers without adding trust bypasses or changing the application contract.

AGENTS.md: not found in the target repository.

Codex review notes: model internal, reasoning medium; reviewed against e4954d7b90c2.

Provenance checked

• Workerd resolutions and matching install-script allowance keeps the original intent (7388b9e: Refresh the existing Worker toolchain and keep installation allowance aligned with its native runtime; this PR explains the security reason for advancing that runtime.)

Testing

Proof path: shipped entry point.

Security

None.

Evidence

What I checked:

• Exact introduced change: The pinned diff changes only workerd package resolutions, five platform artifacts, the matching script allowance, and a global npm override; Wrangler and Miniflare remain unchanged. (package.json:28, 4c389d9d9eec)
• Current main remains affected: The default branch remains at the supplied base and resolves workerd 1.20260926.1; its Node security update does not replace this separate native verifier. (package-lock.json:2965, e4954d7b90c2)
• Supplied production-path verification: The complete PR body identifies the exact base and candidate and reports actual installed Wrangler on Linux ARM64 with Node 24.22.1: baseline accepts 14 verified incident chains, candidate rejects all 14 with underlying CertDistrusted, and public/private HTTPS controls pass on both revisions. (4c389d9d9eec)
• Hosted qualification: The linked job reports success for the exact PR head, including npm ci, repository checks, authenticated vocabulary validation, and Wrangler dry-run; its raw log download was blocked by the network allowlist, which does not invalidate the supplied behavior trace. (.github/workflows/deploy.yml:106, 4c389d9d9eec)
• Toolchain provenance: build(deps): refresh Worker tooling and test dependencies #26 deliberately refreshed the existing Worker toolchain and matched the install-script allowance; the omitted provenance hunks also trace to that refresh. (package-lock.json:2981, 7388b9e26064)
• Separate Node repair and ownership: ci: pin Node.js incident security releases #30 pinned the invoking Node runtime to incident security releases; its merged change establishes prior domain contribution by joshavant but leaves workerd separately owned. (.github/workflows/deploy.yml:38, fb3918ecb5c0)

Likely related people:

• Peter Steinberger: Raw commit 7388b9e adds package.json:24 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 7388b9e26064; files: package.json)

Labels

Label changes:

• add P2: This repairs a demonstrated certificate-trust failure in the local development and test runtime, with no established production-fleet exposure in this repository.
• add proof: sufficient: Contributor real behavior proof is sufficient.
• add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
• add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Label justifications:

• P2: This repairs a demonstrated certificate-trust failure in the local development and test runtime, with no established production-fleet exposure in this repository.
• rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.
• proof: sufficient: Contributor real behavior proof is sufficient.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed October 26, 2026, 5:36 AM ET / 09:36 UTC.