Skip to content

fix(net): health-check unmanaged-Site gateways through the tunnel - #931

Closed
Semen Kolesnikov (Lotarcc) wants to merge 5 commits into
Azure:mainfrom
Lotarcc:fix/net-unmanaged-gateway-healthcheck
Closed

Semen Kolesnikov (Lotarcc) wants to merge 5 commits into
Azure:mainfrom
Lotarcc:fix/net-unmanaged-gateway-healthcheck

Conversation

@Lotarcc

Copy link
Copy Markdown
Contributor

Depends on #926.

Gateways in a Site with manageCniPlugin: false have no pod CIDR, so they have no overlay address
and registerPeersWithHealthCheck skipped them. Their BPF entries were never health-checked or
failed over. Probing the gateway's internal IP over the underlay is not an option: firewalls
between Sites typically allow only the encapsulation (GENEVE), not the health-check port, so the
gateway would be marked down.

These gateways are now probed at their internal IPv4 address out of unbounded0:

  • healthcheck.Manager.AddPeerVia sets the outgoing interface per packet (IP_PKTINFO), so the
    probe leaves through unbounded0 without a kernel route and replies still reach the shared
    socket. The interface index is cached and refreshed after a failed send.
  • The remote node pins the gateway's IP in the BPF map (no route) when localCidrs keep it out, as
    a single nexthop, so probes keep flowing while the gateway is down and it can recover.
  • Only when this node has an IPv4 overlay address to probe from; IPv6 has no on-link fallback
    without a route, so IPv6-only gateways are still not probed.

Lab (kind, unmanaged Calico Site with an Internal gateway pool, remote managed Site over GENEVE,
UDP 9997 dropped on every node's underlay):

Step Gateway health on remote BPF entries
health-check port blocked on the underlay Up healthy
GENEVE from the remote dropped at the gateway Down unhealthy
GENEVE restored Up healthy

Not changed: Reconcile resets Healthy=true on every reconcile, so a Down state only drains
until the next reconcile. That applies to all gateways today.

Testing: TestManagerProbesViaInterface, TestSessionWriteViaUnknownInterface,
TestAddPeerViaKeepsSessionWhenUnchanged, TestAddPeerViaReplacesSessionWhenInterfaceChanges,
TestGatewayHealthTarget, TestGatewayProbePins, TestRegisterUnmanagedGatewayHealthCheck;
go test ./internal/net/healthcheck/ ./cmd/unbounded-net-node/ ./internal/net/controller/ on
Linux; make lint; lab scenario above.

@Lotarcc
Semen Kolesnikov (Lotarcc) marked this pull request as ready for review October 15, 2026 19:21
@Lotarcc
Semen Kolesnikov (Lotarcc) requested a review from a team October 15, 2026 19:21
@Lotarcc

Copy link
Copy Markdown
Contributor Author

Closing this for now to keep the review queue small. It builds on #926 (now node-side only) and #943, so I'll resend it once those land. The branch stays up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant