feat: add WebRTC-Direct support

[mickvandijke]

Summary

Adds a direct, gateway-free browser endpoint to ant-node.

Browser clients can cold-bootstrap from one self-contained WebRTC Direct multiaddress, authenticate the node's persistent ML-DSA identity, establish a fresh ML-KEM-derived application session, walk the DHT themselves, read immutable chunks, obtain signed storage quotes, and submit paid immutable writes directly to closest storage nodes.

The HTTP manifest used by the local demo carries bootstrap metadata only. It never proxies file bytes, performs lookup for the browser, or receives wallet secrets.

Companion browser client: ant-client#186

Architecture

Direct browser transport

• Runs a separate ICE-lite + DTLS + SCTP WebRTC Direct listener alongside native QUIC.
• Persists the DTLS certificate so the certificate-pinned multiaddress remains stable across restarts.
• Publishes canonical /webrtc-direct/certhash/.../p2p/... endpoints through Saorsa's capability-negotiated V2 address plane.
• Preserves the existing V1 QUIC-only publication for older peers.
• Uses one persistent reliable ordered DataChannel per pooled browser/node association.
• Applies bounded frame/message sizes, connection/channel/request limits, 16-KiB DataChannel fragmentation, backpressure, cancellation, and payload-scaled deadlines.

Shared post-quantum application session

Browser protocol v4 removes the former plaintext v3 RPC channel and standalone ML-DSA HELLO challenge.

Before reading any RPC, the listener:

1. receives a versioned ephemeral ML-KEM-768 encapsulation key;
2. encapsulates a fresh shared secret;
3. signs a domain-separated transcript containing the client hello, KEM ciphertext, and ANT peer ID with the node's persistent ML-DSA-65 key; and
4. derives direction-separated application keys from the KEM secret and transcript hash.

Every later request and response, including HELLO, FIND_NODE, GET_CHUNK, QUOTE_CHUNK, PUT_CHUNK, errors, headers, and raw chunk bytes, is protected with ordered ChaCha20-Poly1305 records. Per-direction 64-bit sequences reject replay and reordering. Malformed handshakes, wrong identities, modified ciphertext, unexpected sequences, and authentication failures close the association.

The client and node use the same ant-protocol handshake, key derivation, encrypted-record, framing, and bounds implementation. ant-node owns only the RPC adapter and request admission logic.

Security boundary

The multiaddress certificate fingerprint still authenticates the WebRTC DTLS connection. The application session independently authenticates the ANT peer ID and protects RPC plaintext with post-quantum-derived keys.

This means compromise of only the classical DTLS key is insufficient to impersonate the ANT node or decrypt captured application records. It does not make ICE, DTLS, SCTP, certificate handling, packet lengths, timing, availability, or the WebRTC stack itself post-quantum secure.

The session authenticates the node to the browser, not the browser user to the node. Client authority remains method-specific; paid storage still requires the normal wallet transaction and verifiable payment proof.

Browser RPC surface

The versioned browser protocol remains deliberately narrow:

• HELLO returns protocol, peer, endpoint, capability, chunk-size, and payment metadata inside the already authenticated/encrypted session.
• FIND_NODE returns one bounded local closest-node view and never performs network lookup on the browser's behalf.
• GET_CHUNK returns only locally stored immutable data.
• QUOTE_CHUNK uses the ordinary signed quote and commitment path.
• PUT_CHUNK accepts content-addressed bytes only after the ordinary payment-proof and storage admission checks.

The node reuses native quote, commitment, pricing, payment-proof, content-address verification, and PUT admission behavior. Browser sessions are not inserted into routing tables and cannot invoke arbitrary DHT operations.

Endpoint discovery and deployment

• Standard builds enable WebRTC Direct; minimal/native-only builds may omit it with --no-default-features.
• Configuration can disable the listener with webrtc_direct.enabled = false.
• Public nodes map the browser UDP listener into the existing externally allowed UDP range and advertise the native transport's discovered external IP.
• ant-devnet can start a five-node browser-enabled network, publish a public file through normal storage admission, run local Anvil, and serve bootstrap metadata.
• Browser protocol v4 requires a coordinated browser-client and node deployment. Plaintext v3 and encrypted v4 fail closed rather than silently downgrading.
• Native QUIC, V1 address publication, storage formats, and existing node-to-node post-quantum networking are unchanged.
• This branch pins ant-protocol commit 4dad14b6947b6264e0b5982c976a385f9fdac9e0.

Coordinated draft stack

• Transport: saorsa-transport#160
• Addresses/shared lookup: saorsa-core#158
• Portable/PQ protocol: ant-protocol#29
• Companion browser client: ant-client#186

Draft dependencies are pinned by immutable Git SHA so this branch builds outside the sibling-worktree development setup.

Risk tier

• T0 — docs / tooling / CI / pure UX-output. Repo CI only.
• T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
• T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
• T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Reason: this adds a public node transport/protocol and V2 DHT endpoint-discovery path. It intentionally reuses existing stored-data and payment formats.

Compatibility

• Browser wire: breaking relative to the experimental plaintext v3 protocol; v4 uses a mandatory PQ handshake and encrypted records.
• Native wire: unchanged QUIC and V1 address publication.
• Storage: unchanged content-addressed chunks and public DataMaps.
• Payments: unchanged quote, commitment, transaction, and proof formats.
• API/configuration: adds the webrtc-direct feature, listener settings, devnet support, endpoint artifact, and browser RPC module.

Test evidence

Current-head protocol-v4 validation:

• cargo clippy --lib --features webrtc-direct -- -D warnings
• cargo test --lib web_rtc --features webrtc-direct
  • 12 WebRTC listener, endpoint, request, discovery, payment-hash, and certificate tests passed.
• cargo test --test webrtc_direct_devnet --features webrtc-direct -- --ignored --nocapture
  • 1 five-node integration passed against the pushed ant-protocol revision.
  • Starts local Anvil and five nodes.
  • Completes ML-KEM/ML-DSA session establishment and encrypted HELLO.
  • Performs iterative lookup and public download.
  • Verifies a real signed quote and payment proof.
  • Accepts a paid PUT through the ordinary verifier and reads the record back.
• Shared ant-protocol cryptographic tests: 5 passed.
• ant-core WASM check and clippy pass.
• Browser SDK release build, typecheck, and 8 tests pass.
• ADR governance and git diff --check pass.

Earlier headless-Chromium and independently deployed public-testnet tests validated WebRTC connectivity, multi-node lookup, range streaming, and paid uploads under protocol v3. They remain useful transport evidence but do not validate v4. A matching v4 testnet must be deployed for the next real-browser run.

ADR

ADR-0009: Direct browser clients over WebRTC Direct now records:

• the exact ML-KEM-768, ML-DSA-65, ChaCha20-Poly1305 design;
• independent DTLS-certificate and ANT-identity bindings;
• shared ant-protocol ownership;
• framing, replay protection, security limits, and operational costs;
• coordinated v4 rollout requirements; and
• current local-v4 evidence separately from historical public-v3 evidence.

The ADR remains Proposed pending human review.

New dependencies

• saorsa-transport 0.35.3 with its optional webrtc-direct feature.
• self_encryption 0.36 for the devnet's published browser fixture.
• The coordinated ant-protocol Git revision above; replace draft pins with reviewed releases before merge.

Mitigation / rollback

Disable webrtc_direct.enabled or build without the default WebRTC feature. Native QUIC, existing ant-core/ant-cli clients, payment behavior, and stored data remain usable independently.

Remaining draft work

• Attach the required Linear issue and confirm the proposed T3 tier.
• Rebase onto current main.
• Replace draft dependency pins with reviewed releases before merge.
• Deploy a matching v4 testnet and run real Chrome, Firefox, and Safari interoperability tests.
• Review production rate limits and deployment defaults.
• Relayed WebRTC for NATed storage nodes remains follow-up work; this PR directly serves publicly reachable nodes.

[grumbach]

Independent review of this PR at head f7127540, read as one stack together with saorsa-transport #160, saorsa-core #158 and ant-client #186. Everything below was verified against the source.

The server-side work is solid where it counts. The PQ handshake is genuinely unbypassable: handle_webrtc_channel establishes the session before any request is read (src/web_rtc.rs:947-951), and non-HELLO requests are refused until HELLO succeeds (src/web_rtc.rs:990-1024), with no plaintext fallback and no pre-handshake reachable handler. Browser stores and quotes go through the same ant_protocol.try_handle_request handler as native traffic (src/web_rtc.rs:1711), so there is no weaker parallel payment path, and payment_quote_from_browser_quote (src/web_rtc.rs:1763) rebuilds the native PaymentQuote, re-checks quote.hash() against the artifact and binds the content to the requested address, so the JSON projection cannot smuggle a modified quote past the signature. validate_webrtc_config is thorough about limit consistency and each invariant has a matching test. The two non-ignored devnet tests are real: a live listener, a real WebRTC dial, a real PQ session, and a check that the operator's RPC URL and credentials never reach a browser.

I also classified every src/replication/ and src/upgrade/ edit as behaviour-preserving, with one item I am less sure about: src/replication/mod.rs:3934 turns a closure returning an async block into an async closure, which looks equivalent but I could not type-check it against the pinned graph.

The listener is enabled by default, and its failure stops the node

Cargo.toml:186 has default = ["logging", "webrtc-direct"] and src/config.rs sets enabled: cfg!(feature = "webrtc-direct"), so every stock binary opens a second inbound UDP port, runs ICE/DTLS/SCTP and serves anonymous RPC. saorsa-transport's ADR-015 describes an opt-in listener, which is true of the cargo feature and not of the shipped node, and this repo's own ADR-0009 says production promotion is still gated on browser interop that has not happened.

Four things make that hard to back out of:

1. A listener failure aborts startup. In RunningNode::run (src/node.rs:549-582), any error from web_rtc::spawn calls p2p_node.shutdown() and returns Err. A UDP bind failure on the derived port, a truncated or unreadable certificate PEM, or on custom EVM networks a failed eth_chainId call now stops a healthy storage node from starting. Before this it would simply run.
2. No CLI kill switch. All three new flags set enabled = true (src/bin/ant-node/cli.rs:254-263). docs/WEBRTC_DIRECT_TESTNET.md:163 documents webrtc_direct.enabled = false in a config file and that does work, but a fleet driven by flags would need a config file introduced to turn this off.
3. --no-default-features does not isolate. It omits src/web_rtc.rs, but saorsa-webrtc stays an unconditional dependency (Cargo.toml:38-43), src/browser.rs:8 imports it unconditionally, and the [patch.crates-io] block (Cargo.toml:202-204) redirects saorsa-core and saorsa-transport to the unreleased PR revisions even in a disabled build. So rebuilding to roll back browser risk still ships the unreleased native core and transport lineage, which is not what docs/WEBRTC_DIRECT_TESTNET.md:162-164 promises.
4. The upgrader can leave a node offline. stop_on_upgrade defaults to false (src/config.rs:514). In that mode src/upgrade/apply.rs:628-674 spawns the replacement before graceful shutdown and reports success with no readiness handshake, and the parent then cancels (src/node.rs:656). If the child reaches the same derived UDP port before the old listener releases it, the child hits the fail-closed path above and dies while the parent exits anyway. Rollback only covers binary replacement, not child startup.

Certificate persistence is also not atomic: load_or_generate_certificate (src/web_rtc.rs:668-690) writes the PEM to its final path and chmods 0600 in a later await, so a crash in between leaves the private key at the process umask, and a partial write leaves a corrupt PEM which, given (1), then prevents the node from booting. Existing files never have their permissions repaired. Writing a mode-0600 temp file and renaming would fix both halves.

Worth knowing on the dependency side: Cargo.lock grows from 653 to 718 packages in every default build, including turn, webrtc-mdns, webrtc-media, webrtc-srtp, rtp, rtcp and sdp. ADR-015 says the design does not introduce TURN or STUN servers, yet a TURN implementation and an mDNS responder are now linked in. It is worth confirming webrtc-mdns does not emit multicast traffic in this configuration, because an unexpected mDNS responder on production hosts would surprise operators.

Two CI checks are red

The ADR governance job fails because docs/adr/ADR-0009-direct-browser-clients-over-webrtc-direct.md collides with ADR-0009-audit-proof-shape-and-protocol-families.md already on main, and docs/adr/README.md is not updated so the new ADR is absent from the index. Clippy fails with five denied lints, all in files this PR touched only mechanically: src/replication/config.rs:246, 255, 1663, src/replication/slice.rs:345 and src/replication/protocol.rs:1677.

Relatedly, the MSRV move from 1.75 to 1.91 exists to allow Duration::from_mins and from_hours, and 16 unrelated files across replication/ and upgrade/ were rewritten to use them. That enlarges the diff, couples a browser feature to a toolchain bump, and constrains every build environment. It would read better as its own PR.

Worth fixing before this ships

The derived port is not stable by default. NodeConfig::default() sets the native port to 0 (src/config.rs:438), and the browser port is derived from whatever ephemeral port the OS assigns (src/node.rs:539, src/web_rtc.rs:82-120). A node started without an explicit --port publishes a different WebRTC address after every restart, invalidating cached endpoints even though it reuses its certificate. ADR-0009:483-488 and docs/WEBRTC_DIRECT_TESTNET.md:132-137 both claim the complete address stays stable, which holds only for nodes given a fixed native port, which is what the testnet harness does.

A valid max_request_bytes makes the browser pay and then be rejected. The shared contract fixes the header bound at 64 KiB, but this node lets an operator configure max_request_bytes anywhere in 1..=65536 (src/web_rtc.rs:647) and enforces the smaller value at parse time (src/web_rtc.rs:959), before any request processing, and HELLO never advertises the node's real limit. A PUT header always exceeds 10 KiB, because the quote carries a 1,952-byte ML-DSA public key and a 3,309-byte signature hex-encoded, before the optional commitment sidecar. So with the limit set below about 11 KiB, HELLO and quote_chunk succeed, the browser pays on-chain, and put_chunk is rejected before storage. The 64 KiB default is safe with roughly 27 KiB of margin. Pinning the limit to the shared contract, or advertising it in HELLO, would close it.

A partial config section inverts the default. WebRtcDirectConfig has no container-level #[serde(default)], and enabled carries a plain #[serde(default)] (src/config.rs:161-163) resolving to false, while every other field has an explicit default function. So a TOML file with no [webrtc_direct] section enables the listener, and one with the section but without enabled disables it:

[webrtc_direct]
max_connections = 48

default_webrtc_listener_tracks_compile_time_feature (src/config.rs:776) only exercises WebRtcDirectConfig::default(), never the deserialised path, so it cannot catch this.

Storage errors reach anonymous browsers. src/web_rtc.rs:1503 returns "chunk read failed: {error}" and :1598, :1684 return {other:?}. src/browser.rs:76-84 goes to real trouble to avoid leaking RPC provider detail; this path does not, and with the file-per-chunk store those strings can carry filesystem paths.

Per-IP limits do not aggregate IPv6. canonical_source_ip (src/web_rtc.rs:474) only unmaps IPv4-mapped v6, so a routine /64 gives an attacker a practically unlimited number of distinct sources, each with its own connection slot, rate bucket and byte budget.

cache_insert_browser_devnet_seed is behind the wrong feature. src/payment/verifier.rs:1244 marks content as prepaid and is gated on webrtc-direct, which is on by default, so it ships in production binaries. It is pub(crate) and only reachable through Devnet::publish_public_file, which operates on its own verifier, so I do not believe it is exploitable, but a payment bypass belongs behind test-utils or a dedicated devnet feature.

resolve_automatic_config falls back to loopback. src/web_rtc.rs:99-108 uses 127.0.0.1 when the route probe fails, and the node then publishes that as its browser endpoint. saorsa-core drops loopback on ingest, so the publish is silently wasted rather than an error.

Test fidelity and validation gaps

The five-node devnet test is valuable native interoperability coverage, but it is not browser coverage and does not test the browser client. BrowserRpcClient (tests/webrtc_direct_devnet.rs:426) is a native Rust client built on saorsa_transport::webrtc_direct::WebRtcDirectClient, with hand-written framing and PQ logic using the same saorsa-webrtc crate and constants as the server. If node framing and this adapter drift together while the deployed WASM client does not, CI stays green and every real browser fails.

Three more specific gaps:

• Nothing moves a large payload. The whole suite transfers 31 bytes (tests/webrtc_direct_devnet.rs:206) and 32 bytes (:349); MAX_CHUNK_SIZE appears only as a read bound. So the 16 KiB DataChannel chunking loop, multi-fragment reassembly, byte reservations at scale, the size-scaled write deadlines and SCTP backpressure are never exercised, even though ant-devnet's own default public file is 5 MiB (src/bin/ant-devnet/main.rs:201) and the runbook implies automated verification publishes it.
• The admission-timeout test is a mock. first_data_channel_timeout_releases_connection_admission (src/web_rtc.rs:1874-1915) manually takes an admission guard, hands pending() to the timeout helper and lets the guard drop out of local scope. It never drives handle_connection, a real association or accept_data_channel, so it would still pass if production stopped wrapping the first channel in that helper. That is the invariant the listener's availability depends on.
• The five-node test bypasses discovery. It takes its initial endpoints straight from Devnet::browser_endpoints() (:203-220) and makes two manually scripted one-hop requests, with responses augmented by the in-process endpoint catalog, so it cannot catch a failure of V2 DHT publication, mixed-fleet propagation, alpha/K convergence, retained routing state, failure cooldowns or production seed bootstrapping.

ADR-0009 is admirably candid that there is no automated real-browser v5 flow and that Chrome, Firefox and Safari interop remain unmet acceptance criteria rather than claimed results. That is the right way to write it up. The tension is that an explicitly unmet acceptance criterion currently ships enabled by default.

For operations, what exists is startup logging, an endpoint file, static resource knobs, fail-fast startup and shutdown ordering. There is no listener health or readiness metric, no DHT reachability probe, no saturation alert, no runtime disable flag, no certificate backup or rotation procedure, no child-readiness handshake in the upgrader, no browser-aware rollback test and no firewall or NAT verification workflow.

Smaller things

• ADR-0009 contradicts itself on the connection profile: :507-517 says the implemented profile is v2 with no v1 fallback, :742-746 says the implementation currently uses v1. The code uses v2, so :742 is stale.
• docs/WEBRTC_DIRECT_TESTNET.md:8-10 says Rust 1.88 or newer while the package now requires 1.91, so following the runbook fails before you reach any test. The ADR also names ant-protocol 2.3.1 while Cargo uses 2.3.5, and says saorsa-transport owns persisted certificates while this repo loads and writes them.
• --webrtc-direct-advertised-addr has requires = "webrtc_direct_bind" (src/bin/ant-node/cli.rs:44), so overriding only the advertised address means redundantly repeating the bind.
• saorsa-webrtc is a git dependency with no version key in [dependencies], which cargo publish refuses, and the crate name is already taken on crates.io at 0.1.0 through 0.1.2 by a different crate. That blocks releasing this repo.

What I could not check

Nothing was built or run: the pinned cross-repo revisions are not in a local cargo cache here. No real browser, no NAT or firewall path, no mixed-version fleet, and no measurement of the resource envelope under load.