feat: Add firewall support to prevent local network access

.github/workflows/pr.yaml

@@ -14,6 +14,24 @@ jobs:
       - name: Validate hoprd.cfg.yaml.tpl
         run: ./test/validate-hoprd-config.sh
 
+      - name: Check that VPN clients can't reach local resources
+        run: ./test/firewall.sh
+
+      - name: Check the blocklist and per-peer shaping
+        run: ./test/traffic.sh
+
+      # Same checks inside the image that ships, so a missing bash/iproute2/iptables/ipset/curl
+      # fails here rather than on a DAppNode. Pulls the same base image the package build below
+      # does.
+      - name: Build the gnosisvpn-server image for the image checks
+        run: docker build -t gnosisvpn-server:checks gnosisvpn-server
+
+      - name: Check the firewall inside the built gnosisvpn-server image
+        run: FIREWALL_TEST_IMAGE=gnosisvpn-server:checks ./test/firewall.sh
+
+      - name: Check the blocklist and shaping inside the built gnosisvpn-server image
+        run: TRAFFIC_TEST_IMAGE=gnosisvpn-server:checks ./test/traffic.sh
+
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v3
 

Dockerfile

@@ -2,7 +2,7 @@ ARG UPSTREAM_VERSION
 # hoprd-pix-test, not hoprd: the plain image enables no PIX pool feature, so it inherits
 # hopr-lib's default `pix-bjj` and announces BabyJubJub as its PIX curve suite.
 # Change back to `hoprd` once Curvy pool is ready
-FROM europe-west3-docker.pkg.dev/hoprassociation/docker-images/hoprd-pix-test:${UPSTREAM_VERSION}
+FROM europe-west3-docker.pkg.dev/hoprassociation/docker-images/hoprd:${UPSTREAM_VERSION}
 
 # not used at the moment, but might be useful in the future
 ENV DAPPNODE=true

README.md

@@ -22,6 +22,114 @@ Runs a [GnosisVPN](https://vpn.gnosis.eth.limo) exit node on DAppNode: a HOPR no
   across installs, and isn't guaranteed stable across a `gnosisvpn-server` recreate - see
   "Known limitations" below.
 
+## Firewall
+
+`gnosisvpn-server` has its own network namespace, and
+[`gnosisvpn-server/wg-hooks.sh`](gnosisvpn-server/wg-hooks.sh) sets its forwarding policy.
+`wg-quick` runs the script from `PostUp`/`PreDown` in `wggvpn.conf`. It is a port of the
+`wg-hooks.sh` + `iptables.rules` pair from the exit-node appliance image (`gnosis_vpn-exit_os`).
+The appliance version can't be used as-is, because a DAppNode package never sees the host's
+network namespace (see "Known limitations" below).
+
+VPN clients are untrusted. Without the firewall, their traffic can reach the DAppNode host, the
+dappmanager, every package on `dncore_network` and the LAN behind the host. With it, only
+public destinations are reachable:
+
+- FORWARD policy is DROP. Client traffic goes through `GVPN_PRIVATE` first, which drops:
+  - RFC1918, CGNAT, link-local, loopback, `0.0.0.0/8` and multicast/reserved
+  - `dncore_network` (`172.33.0.0/16`), which is not in RFC1918
+  - the subnet this container is attached to
+- Only `wggvpn -> <uplink>` is accepted. Return traffic is `RELATED,ESTABLISHED` only, so the
+  exit node can't open connections to clients.
+- INPUT from `wggvpn` accepts only the ICMP echo that the GnosisVPN client sends to check the
+  tunnel (`connection.ping`, default `10.128.0.1`). The control API on port 8000 and all other
+  local services are not reachable from the tunnel.
+- `up` removes its old rules first, so a restart doesn't add duplicates. `down` leaves FORWARD
+  at DROP.
+- `entrypoint.sh` sets FORWARD to DROP before it starts `gnosis_vpn-server`. `wg-quick` brings
+  `wggvpn` up before `PostUp` runs the hook, so this closes that gap.
+
+Not covered:
+
+- IPv6. `gnosis_vpn-server` adds peers with IPv4 `allowed-ips` only and `wggvpn` has no IPv6
+  address, so WireGuard drops IPv6 client packets. The appliance also drops IPv6 with
+  `ip6tables`; this package does not. Re-check this when `gnosis_vpn-server` gets IPv6 support
+  ([gnosis/gnosis_vpn-client#330](https://github.com/gnosis/gnosis_vpn-client/issues/330)).
+- The DAppNode's own public IPv4 address. It is a public destination, so it passes. (The
+  appliance finds and blocks it with an external HTTPS lookup; this package doesn't.) What a
+  client reaches there depends on where the address lives:
+  - On the host's NIC (a VPS, or a public IP from the ISP): every service the host listens on
+    and every published port, e.g. SSH and the hoprd API on `3001`. Don't run this package on
+    such a host unless the host firewall restricts those services by source.
+  - On a router in front of the host: whatever the router hairpins back, e.g. its admin UI if
+    it listens on WAN, and every forwarded port. Assume NAT loopback is on.
+`just test-firewall` and `just test-firewall-image` check these rules (see
+[`test/README.md`](test/README.md)).
+
+## Blocklist
+
+[`gnosisvpn-server/gvpn-blocklist-update.sh`](gnosisvpn-server/gvpn-blocklist-update.sh) fills
+the `gvpn-blocklist` ipset, and `wg-hooks.sh` drops traffic to it, and traffic from it back to
+clients, before the client ACCEPT. Ported from the appliance, which uses the same FireHOL level1
+feed:
+
+- `entrypoint.sh` loads the cached list before it starts the server, so the last known-good list
+  filters the first forwarded packet. The cache lives in the `gnosisvpn-data` volume.
+- It then runs the refresher as a child process. A container has no timer, so the appliance's
+  twice-daily systemd timer becomes "refresh now, then every 12 h plus up to 1 h of jitter".
+- A refresh swaps the whole set in atomically (`ipset swap`), so forwarding never sees a
+  half-built list.
+- It fails open. An unreachable feed, or one that parses to fewer than 1000 or more than 131072
+  entries, leaves the previous list in place and logs why. A poisoned `0.0.0.0/1` can't
+  blackhole every session, because prefixes shorter than /8 are only accepted for the 224/3
+  bogon range.
+- IPv4 only, like the isolation rules - see "Not covered" above for why client IPv6 never
+  reaches the tunnel.
+
+Feeds are configured from the DAppNode UI, see "Settings in the DAppNode UI" below.
+
+## Per-peer shaping
+
+[`gnosisvpn-server/gvpn-peer-shaping.sh`](gnosisvpn-server/gvpn-peer-shaping.sh) builds an HTB
+tree on `wggvpn` when `wg-hooks.sh` runs, with one class per peer slot:
+
+- Download (server to client) is classified by destination address on `wggvpn` egress.
+- Upload (client to internet) is mirrored from `wggvpn` ingress onto an `ifb-gvpn` device and
+  classified there by source address. It has to happen on the `wggvpn` side: the uplink only
+  ever sees post-MASQUERADE addresses.
+- Each peer is capped at `GVPN_PEER_RATE_KBIT` (default 10000) and all peers together at
+  `GVPN_PEER_SHAPING_AGGREGATE_KBIT` (default 25000). It queues rather than drops, so TCP
+  settles at the cap instead of sawtoothing, and fq_codel signals with delay before loss
+  because a dropped packet costs an end-to-end retransmit across the mixnet.
+- `entrypoint.sh` derives the peer range from `WIREGUARD_CIDR` and `MAX_SLOTS`, the same way it
+  derives the pool for `config.toml`, and exports it for the `wg-quick` process that runs the
+  hook.
+- Unlike the appliance, a shaping failure does not stop the exit node. `tc` and the `ifb`/`sch_*`
+  modules come from the DAppNode host's kernel, and this container can't load modules into it. A
+  host without them serves unshaped and logs `per-peer shaping failed` on every interface sync.
+
+`just test-traffic` and `just test-traffic-image` check both of these (see
+[`test/README.md`](test/README.md)).
+
+## Settings in the DAppNode UI
+
+The exit node's limits and blocklist are optional fields in the package's setup wizard, so they
+are set in the DAppNode UI (the package's **Config**, or the wizard on install) rather than by
+editing `docker-compose.yml`. Leaving a field empty keeps the default:
+
+| UI field | Variable | Default |
+|---|---|---|
+| Per-client bandwidth limit | `GVPN_PEER_RATE_KBIT` | `10000` kbit/s |
+| Total bandwidth limit for all clients | `GVPN_PEER_SHAPING_AGGREGATE_KBIT` | `25000` kbit/s |
+| Blocklist feeds | `GVPN_BLOCKLIST_FEEDS` | FireHOL level1 |
+| Blocklist refresh interval | `GVPN_BLOCKLIST_REFRESH_INTERVAL_S` | `43200` s |
+| WireGuard client subnet | `WIREGUARD_CIDR` | `10.128.0.0/24` |
+| Maximum connected peers | `MAX_SLOTS` | `4` |
+
+Feed URLs are space-separated; one per line works too. The rest of the `GVPN_BLOCKLIST_*`
+variables (minimum entries, download timeout, jitter) stay environment-only, with the
+appliance's defaults.
+
 ## Known limitations vs. production
 
 A regular (non-core) DAppNode package can't get `network_mode: host` or a custom docker

docker-compose.yml

@@ -6,7 +6,7 @@ services:
       context: .
       args:
         # see https://console.cloud.google.com/artifacts/docker/hoprassociation/europe-west3/docker-images/hoprd
-        UPSTREAM_VERSION: 5.0.0-rc.1-commit.7d90171
+        UPSTREAM_VERSION: 5.0.2-pr.232
     ports:
       - "1422:1422/udp"
       - "1422:1422/tcp"
@@ -65,6 +65,13 @@ services:
       WIREGUARD_CIDR: "10.128.0.0/24"
       WIREGUARD_MTU: "1420"
       MAX_SLOTS: "4"
+      # Per-peer traffic shaping, in kbit/s: each client is capped at the per-peer rate and
+      # all clients together at the aggregate. See README's "Per-peer shaping".
+      GVPN_PEER_RATE_KBIT: "10000"
+      GVPN_PEER_SHAPING_AGGREGATE_KBIT: "25000"
+      # Space-separated threat feeds for the blocklist. The last known-good list is cached
+      # in the gnosisvpn-data volume. See README's "Blocklist".
+      GVPN_BLOCKLIST_FEEDS: "https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level1.netset"
     restart: unless-stopped
     deploy:
       resources:

gnosisvpn-server/Dockerfile

@@ -1,17 +1,20 @@
 FROM europe-west3-docker.pkg.dev/gnosisvpn-production/docker-images/gnosis_vpn-server:0.7.0@sha256:443c4264bb41a82526ea7d227676f5060633e489569b4fb7fe6299f6a28ff78d
 
-# The base image (Alpine) ships wg/wg-quick/ip but not iptables/ipset, which the PostUp/PreDown
-# rules in wggvpn.conf.tpl need (ported from the gnosisvpn-server Ansible role).
-RUN apk add --no-cache iptables ip6tables ipset bash
+# The scripts need bash, iproute2 (ip, tc), iptables and ipset, and the blocklist
+# refresher needs curl. Install them explicitly instead of relying on the base image
+# (Alpine); apk skips any that are already installed.
+RUN apk add --no-cache bash iproute2 iptables ipset curl
 
 WORKDIR /app
 
 # Replace the image's own config.toml/wggvpn.conf with our templated versions. entrypoint.sh
 # renders these and execs gnosis_vpn-server directly (not via the image's own wrapper.sh,
 # which would leave it as a child process instead of PID 1 - see entrypoint.sh for why that
-# matters for shutdown).
+# matters for shutdown). PostUp/PreDown in wggvpn.conf run wg-hooks.sh, the forwarding
+# policy - see README's "Firewall" section. wg-hooks.sh runs gvpn-peer-shaping.sh, and
+# entrypoint.sh runs gvpn-blocklist-update.sh.
 COPY config.toml.tpl wggvpn.conf.tpl ./
-COPY entrypoint.sh ./entrypoint.sh
-RUN chmod +x ./entrypoint.sh
+COPY entrypoint.sh wg-hooks.sh gvpn-blocklist-update.sh gvpn-peer-shaping.sh ./
+RUN chmod +x ./entrypoint.sh ./wg-hooks.sh ./gvpn-blocklist-update.sh ./gvpn-peer-shaping.sh
 
 ENTRYPOINT ["/app/entrypoint.sh"]

gnosisvpn-server/entrypoint.sh

@@ -15,6 +15,19 @@ wg_address="${o1}.${o2}.${o3}.$((o4 + 1))"
 wg_start="${o1}.${o2}.${o3}.$((o4 + 2))"
 wg_end="${o1}.${o2}.${o3}.$((o4 + 1 + max_slots))"
 
+# The pool has to stay inside the CIDR's last /24: per-peer shaping steers packets by
+# the last octet only, and .255 is the broadcast address.
+if (( o4 + 1 + max_slots > 254 )); then
+  echo "gnosisvpn-server: FATAL: WIREGUARD_CIDR ${WIREGUARD_CIDR} with MAX_SLOTS=${max_slots} needs last octets .$((o4 + 1))-.$((o4 + 1 + max_slots)), past .254" >&2
+  exit 1
+fi
+
+# Consumed by wg-hooks.sh through gvpn-peer-shaping.sh, which wg-quick runs with this
+# process's environment (gnosis_vpn-server spawns wg-quick as a plain child process).
+export GVPN_PEER_SUBNET_PREFIX="${o1}.${o2}.${o3}"
+export GVPN_PEER_MIN_OCTET=$((o4 + 2))
+export GVPN_PEER_MAX_OCTET=$((o4 + 1 + max_slots))
+
 # gnosis_vpn-server's own wrapper.sh generates a WireGuard key when PRIVATE_KEY isn't set,
 # but doesn't persist it - a fresh key on every restart would break peer registration.
 # Persist it in the /data volume instead, generating it once here.
@@ -53,6 +66,18 @@ if [ "${current_ip_forward}" != "1" ]; then
   echo "gnosisvpn-server: WARNING: net.ipv4.ip_forward is '${current_ip_forward}', not '1', and this container cannot set it itself. VPN client traffic will not reach the internet until it's enabled on the DAppNode host." >&2
 fi
 
+# wg-quick brings wggvpn up (with its peers) before PostUp runs wg-hooks.sh, so close
+# FORWARD now; wg-hooks.sh then opens only the public path. If this fails, the container
+# fails: the exit node must not start without it.
+iptables -P FORWARD DROP
+
+# The blocklist set lives in this container's namespace, so its cache lives in the /data
+# volume and the refresh runs as a child process: a container has no timer. Loading the
+# cache first (no network) filters the first forwarded packet with the last known-good
+# list; the refresher then replaces it from the feeds in the background, failing open.
+/app/gvpn-blocklist-update.sh --from-cache
+/app/gvpn-blocklist-update.sh --loop &
+
 # Run the binary directly as PID 1 (not via the image's wrapper.sh, which launches it as a
 # child process) so Docker's SIGTERM on `stop`/`down` reaches gnosis_vpn-server itself. It
 # has its own graceful-shutdown handling, which is what actually runs wg-quick down/PreDown.