From 06ad4199c61d29a6817c659de2811e8cf7996a61 Mon Sep 17 00:00:00 2001 From: Fadi George Date: Tue, 13 Oct 2026 13:12:24 -0700 Subject: [PATCH 01/10] test: [SDK-5481] add demo fault-injection scripts for retry testing Co-Authored-By: Claude Opus 5.5 --- .gitignore | 3 +- examples/demo/scripts/retry/README.md | 107 ++++++++ examples/demo/scripts/retry/fault_proxy.py | 110 +++++++++ examples/demo/scripts/retry/report.py | 121 +++++++++ examples/demo/scripts/retry/retry-test.sh | 271 +++++++++++++++++++++ 5 files changed, 611 insertions(+), 1 deletion(-) create mode 100644 examples/demo/scripts/retry/README.md create mode 100644 examples/demo/scripts/retry/fault_proxy.py create mode 100644 examples/demo/scripts/retry/report.py create mode 100755 examples/demo/scripts/retry/retry-test.sh diff --git a/.gitignore b/.gitignore index 0e7dffa5d..7bcb1bd91 100644 --- a/.gitignore +++ b/.gitignore @@ -21,4 +21,5 @@ iOS_SDK/Carthage/Build .build/ examples/demo/App/Secrets.plist -examples/demo/Local.xcconfig \ No newline at end of file +examples/demo/Local.xcconfig +examples/demo/scripts/retry/.state/ diff --git a/examples/demo/scripts/retry/README.md b/examples/demo/scripts/retry/README.md new file mode 100644 index 000000000..ca5da8e3b --- /dev/null +++ b/examples/demo/scripts/retry/README.md @@ -0,0 +1,107 @@ +# Retry test harness + +Injects failures into the SDK's requests from the demo app, so executor retries and backoff can be +watched and checked. A [mitmproxy](https://mitmproxy.org) addon fails only requests that carry the +SDK's `SDK-Version` header. The demo's own REST calls and all other traffic pass through. Only +`onesignal.com` traffic is decrypted. + +```bash +brew install mitmproxy +cd examples/demo/scripts/retry +./retry-test.sh help +``` + +## Setup + +1. Build and install the demo app on a simulator (see `examples/demo/README.md`), launch it once, + and allow notifications. The prompt holds the app inactive, so foreground retries never fire + while it is showing. Use `fresh` instead of reinstalling, so the permission is kept. +2. `./retry-test.sh start` starts mitmproxy on port 8899, points the Mac's Wi-Fi/Ethernet HTTP(S) + proxy at it (the simulator uses the Mac's proxy), and trusts its CA on the booted simulator. +3. `./retry-test.sh fresh` wipes the demo's data and relaunches it as a new install. +4. In another terminal, `./retry-test.sh watch` follows each SDK request. +5. When done, **`./retry-test.sh stop`** restores the Mac's proxy settings. + +Leave the Mac's proxy alone with `RETRY_TEST_PROXY_MODE=local` (mitmproxy captures the `App` +process, which needs its network extension approved once). For a physical device, use +`RETRY_TEST_PROXY_MODE=manual` and set the device's Wi-Fi proxy to the Mac, as `start` explains. + +## Commands + +| Command | Does | +| --- | --- | +| `fail [--times N] [--retry-after S] [--add]` | Fail matching requests. Status `0` drops the connection, like being offline. Without `--times` it fails until `clear`. `--add` keeps earlier rules. Applies right away. | +| `clear` / `rules` | Remove or show the rules | +| `report [regex]` | Per route: each attempt, the gap since the previous one, and the gap expected | +| `reset-log` | Forget recorded attempts, for example between scenarios | +| `fresh` / `relaunch` | New install without reinstalling / kill and relaunch | +| `background` / `foreground` | Background the demo by opening Settings / bring it back | +| `network-flap [s]` | Turn the Mac's Wi-Fi off for `s` seconds (default 10). Drops your Mac's connection too. | +| `logs` | Stream the demo's SDK request and retry logs | + +Routes: `create-user`, `fetch-user`, `update-user` (tags, language, session data), `identity` (add +alias and login's Identify User share it), `remove-alias`, `create-sub`, `update-sub`, +`delete-sub`, `identity-by-sub`, `custom-events`, `all`. + +## Reading `report` + +``` +POST apps//users (User executor) + # time status body gap expected note + 1 0.0s 408* edc1f203 + 2 2.8s 408* 8f1a8f24 2.8s 2.5-5.0s executor backoff #1 + 3 11.4s 408* d85e5a15 8.6s 5.0-10.0s executor backoff #2 + 4 31.0s 201 3a910c5a 19.6s 10.0-20.0s executor backoff #3 +``` + +What the SDK should do: + +- **Executor backoff**: 5s, doubling to 300s, times a jitter of 0.5 to 1. `Retry-After` is the + minimum wait (capped at 1h), and a 429 without one waits 60s. +- **Client reattempts first**: `OneSignalClient` itself re-sends a 5xx or status 0 four times + (5, 15, 45, 135s) before the executor sees the failure. That count is kept per request for the + run, so later failures of the same request go straight to the executor backoff. Use a + retryable status the client does not reattempt, like **408**, to see the executor backoff right + away. +- **Repo executors** (everything but the User executor) only resend on the 5s poll, so their window + allows up to 5s more. The User executor wakes itself on time. +- **Foreground, and network return while in the foreground, end the executor backoff early**, but + not `Retry-After`. `EARLY` right after those is expected. One case happens without you doing + anything: at cold start, the first failure often lands just before the app goes active, so the + second attempt follows within a second. +- `body` is a hash of the request body. A new hash on each attempt is normal, since bodies hold + timestamps. + +## Scenarios + +Run `reset-log` before each one. Unless noted, the request should succeed after the last failure, +and nothing should need a relaunch. + +1. **Exponential backoff, User executor**: `fail create-user 408 --times 4`, then `fresh`. Gaps + should fall in the windows above, then 201. +2. **Repo executor**: `fail update-user 408 --times 3`, then add a tag in the demo. The PATCH + retries on the next poll after each backoff. +3. **Retry-After**: `fail update-user 429 --times 2 --retry-after 20`, then add a tag. Gaps of at + least 20s. `background` then `foreground` during the wait must not cut it short. +4. **429 without Retry-After**: `fail update-user 429 --times 1`, then add a tag. About 60s. +5. **5xx**: `fail update-user 503 --times 6`, then add a tag. Four client reattempts (5, 15, 45, + 135s), then executor backoff. Slow, about 3.5 minutes. +6. **Offline**: `fail all 0`, change tags, aliases and email, wait for a few failures, then `clear`. + Everything goes out by the next backoff, or right away after `foreground`. +7. **Foreground ends the backoff**: `fail create-user 408 --times 6`, then `fresh`. After about + 25s, `background`, wait, `foreground`. An attempt goes out right as the app becomes active. +8. **Network return**: as in 7, but `network-flap 5` with the app in the foreground. An attempt goes + out when Wi-Fi comes back. Repeat with the app in the background: no attempt then. +9. **Order is kept**: `fail create-user 408 --times 3`, then `fresh` and log in during the + failures. Identify User (PATCH `.../identity`) is not sent until Create User succeeds. +10. **Fetch User**: `fail fetch-user 408 --times 3`, then log in to an existing external ID. + Fetch User retries in the same run, and later user requests wait for it. +11. **Kill during backoff**: `fail update-user 408`, add a tag, wait for two failures, `clear`, + then `relaunch`. The cached request is sent right after launch. +12. **Non-retryable is not retried**: `fail update-user 400 --times 1`, then add a tag. One attempt, + no retries. +13. **Update Subscription no longer retries every 5s**: `fail update-sub 408 --times 4`, then + toggle push in the demo. Gaps grow instead of repeating every 5s. +14. **Login/logout while failing**: `fail all 408`, log in, log out, log in as someone else, then + `clear`. Requests for each user go out in order, with no errors about the wrong user. +15. **Identity Verification**: repeat 1, 2 and 9 with an app that requires JWT. diff --git a/examples/demo/scripts/retry/fault_proxy.py b/examples/demo/scripts/retry/fault_proxy.py new file mode 100644 index 000000000..5477300ea --- /dev/null +++ b/examples/demo/scripts/retry/fault_proxy.py @@ -0,0 +1,110 @@ +""" +mitmproxy addon that injects failures into OneSignal SDK requests so executor retries can be +observed in the demo app. Driven by `retry-test.sh`; see README.md. + +Only requests carrying the SDK's `SDK-Version` header are touched, so the demo's own REST calls +pass through. Every SDK request is appended to attempts.jsonl for `retry-test.sh report`. + +Rules are re-read whenever rules.json changes. Each rule: + {"route": "update-user", "method": "PATCH", "path": "", "status": 503, + "retry_after": "30", "times": 3} +`status` 0 drops the connection (the SDK sees status 0, like being offline). `times` is how many +matching requests fail before the rule passes them through; null fails forever. +""" + +import hashlib +import json +import os +import re +import time + +from mitmproxy import http + +STATE_DIR = os.environ["RETRY_TEST_STATE_DIR"] +RULES_PATH = os.path.join(STATE_DIR, "rules.json") +ATTEMPTS_PATH = os.path.join(STATE_DIR, "attempts.jsonl") + +API_PATH_PREFIX = re.compile(r"^/(api/v1/)?") + + +class FaultProxy: + def __init__(self): + self.rules_mtime = None + self.rules = [] + self.remaining = [] + + def load_rules(self): + try: + mtime = os.stat(RULES_PATH).st_mtime_ns + except FileNotFoundError: + mtime = None + if mtime == self.rules_mtime: + return + self.rules_mtime = mtime + self.rules = [] + if mtime is not None: + with open(RULES_PATH) as f: + self.rules = json.load(f) + self.remaining = [rule.get("times") for rule in self.rules] + print(f"[fault] rules: {json.dumps(self.rules) if self.rules else 'none, passing everything'}") + + def match(self, flow): + path = API_PATH_PREFIX.sub("", flow.request.path.split("?")[0]) + for i, rule in enumerate(self.rules): + if rule.get("method") and rule["method"] != flow.request.method: + continue + if not re.search(rule["path"], path): + continue + if self.remaining[i] is not None: + if self.remaining[i] <= 0: + continue + self.remaining[i] -= 1 + return rule + return None + + def request(self, flow: http.HTTPFlow): + if "onesignal.com" not in flow.request.pretty_host or "SDK-Version" not in flow.request.headers: + return + self.load_rules() + rule = self.match(flow) + flow.metadata["fault_rule"] = rule + if rule is None: + return + if rule["status"] == 0: + self.record(flow, 0, rule) + flow.kill() + return + headers = {"Content-Type": "application/json"} + if rule.get("retry_after") is not None: + headers["Retry-After"] = str(rule["retry_after"]) + body = json.dumps({"errors": [{"title": f"Injected by fault_proxy ({rule['route']})"}]}) + flow.response = http.Response.make(rule["status"], body, headers) + + def response(self, flow: http.HTTPFlow): + if "fault_rule" not in flow.metadata: + return + self.record(flow, flow.response.status_code, flow.metadata["fault_rule"]) + + def error(self, flow: http.HTTPFlow): + if "fault_rule" in flow.metadata and flow.metadata["fault_rule"] is None: + self.record(flow, 0, None) + + def record(self, flow, status, rule): + path = API_PATH_PREFIX.sub("", flow.request.path.split("?")[0]) + entry = { + "t": time.time(), + "method": flow.request.method, + "path": path, + "status": status, + "injected": rule is not None, + "retry_after": rule.get("retry_after") if rule else None, + "body": hashlib.sha1(flow.request.raw_content or b"").hexdigest()[:8], + } + with open(ATTEMPTS_PATH, "a") as f: + f.write(json.dumps(entry) + "\n") + tag = "INJECTED" if rule else "passed" + stamp = time.strftime("%H:%M:%S", time.localtime(entry["t"])) + print(f"[fault] {stamp} {status:>3} {tag:<8} {flow.request.method:<6} {path} body={entry['body']}", flush=True) + + +addons = [FaultProxy()] diff --git a/examples/demo/scripts/retry/report.py b/examples/demo/scripts/retry/report.py new file mode 100644 index 000000000..504d9c9fc --- /dev/null +++ b/examples/demo/scripts/retry/report.py @@ -0,0 +1,121 @@ +""" +Prints each SDK route's attempts from attempts.jsonl with the gap since the previous attempt, and +the gap the SDK should have waited. Usage: report.py [route regex] + +Expected gaps model the SDK's two retry layers: + - OneSignalClient re-sends a 5xx or status 0 up to 4 times itself (5, 15, 45, 135s). The count + lives on the request object, so a request only gets those once per run. + - Then the executor backs off: 5s doubling to 300s, times jitter in [0.5, 1]. Retry-After, or + 60s for a 429 without one, is the minimum. Repo executors (everything except the User + executor) only resend on their 5s poll, so up to 5s more is allowed. +A foreground or network return ends an executor backoff early, so short gaps after those are fine. +""" + +import json +import re +import sys +from collections import OrderedDict + +CLIENT_DELAYS = [5, 15, 45, 135] +POLL_SLACK = 5.5 +CLOCK_SLACK = 1.0 +# Login's Identify User shares PATCH .../identity with Add Aliases, so it is judged with the looser +# repo executor window. +USER_EXECUTOR = re.compile(r"^POST apps/[^/]+/users$|^GET apps/[^/]+/users/by/|^GET apps/[^/]+/subscriptions/[^/]+/user/identity$") +EXECUTOR_ROUTE = re.compile(r"^apps/[^/]+/(users(/by/.*)?|subscriptions/[^/]+|custom_events)$") +ID_SEGMENT = re.compile(r"(by/[^/]+/|subscriptions/)[^/]+") + + +def route_key(entry): + path = ID_SEGMENT.sub(r"\1:id", entry["path"]) + return f"{entry['method']} {path}" + + +def is_user_executor(entry): + return bool(USER_EXECUTOR.search(f"{entry['method']} {entry['path']}")) + + +def executor_window(failures, status, retry_after): + exponential = min(5 * 2 ** (failures - 1), 300) + low, high = exponential * 0.5, exponential + floor = None + if retry_after is not None: + try: + floor = min(max(float(retry_after), 0), 3600) + except ValueError: + floor = 60 + elif status == 429: + floor = 60 + if floor is not None: + low, high = max(low, floor), max(high, floor) + return low, high, floor is not None + + +def main(): + path = sys.argv[1] + route_filter = re.compile(sys.argv[2]) if len(sys.argv) > 2 else None + groups = OrderedDict() + with open(path) as f: + for line in f: + if line.strip(): + entry = json.loads(line) + groups.setdefault(route_key(entry), []).append(entry) + + if not groups: + print("No SDK requests recorded yet.") + return + + for key, entries in groups.items(): + if route_filter and not route_filter.search(key): + continue + if not EXECUTOR_ROUTE.search(entries[0]["path"]): + continue + repo = not is_user_executor(entries[0]) + print(f"\n{key} ({'repo executor, 5s poll' if repo else 'User executor'})") + print(f" {'#':>3} {'time':>8} {'status':>6} {'body':>8} {'gap':>8} {'expected':>16} note") + client_retries = 0 + executor_failures = 0 + expected = None + start = entries[0]["t"] + for i, entry in enumerate(entries): + gap = entry["t"] - entries[i - 1]["t"] if i else None + note = "" + if expected and gap is not None: + low, high, kind = expected + if gap < low - CLOCK_SLACK: + note = f"EARLY ({kind})" + if kind.startswith("executor") and not kind.endswith("Retry-After"): + note += "; fine if right after a foreground or network return" + elif gap > high + CLOCK_SLACK: + note = f"LATE ({kind})" + else: + note = kind + elif gap is not None: + note = "new attempt after success" + expected_text = f"{expected[0]:.1f}-{expected[1]:.1f}s" if expected else "" + gap_text = f"{gap:.1f}s" if gap is not None else "" + mark = "*" if entry["injected"] else " " + print(f" {i + 1:>3} {entry['t'] - start:>7.1f}s {entry['status']:>5}{mark} {entry.get('body', ''):>8} {gap_text:>8} {expected_text:>16} {note}") + + status = entry["status"] + if 200 <= status < 300: + client_retries = executor_failures = 0 + expected = None + elif status in (400, 401, 402, 403, 404, 409, 410): + expected = None + elif (status >= 500 or status == 0) and client_retries < len(CLIENT_DELAYS): + delay = CLIENT_DELAYS[client_retries] + client_retries += 1 + expected = (delay, delay, "client reattempt") + else: + executor_failures += 1 + low, high, has_floor = executor_window(executor_failures, status, entry.get("retry_after")) + if repo: + high += POLL_SLACK + kind = f"executor backoff #{executor_failures}" + expected = (low, high, kind + (", Retry-After" if has_floor else "")) + print("\n* = failure injected by the proxy") + + +if __name__ == "__main__": + main() diff --git a/examples/demo/scripts/retry/retry-test.sh b/examples/demo/scripts/retry/retry-test.sh new file mode 100755 index 000000000..e3341abd7 --- /dev/null +++ b/examples/demo/scripts/retry/retry-test.sh @@ -0,0 +1,271 @@ +#!/usr/bin/env bash +# Fault-injection harness for manually testing SDK request retries in the demo app. +# Run `./retry-test.sh help` or see README.md. +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +STATE_DIR="${RETRY_TEST_STATE_DIR:-$SCRIPT_DIR/.state}" +RULES="$STATE_DIR/rules.json" +ATTEMPTS="$STATE_DIR/attempts.jsonl" +PROXY_LOG="$STATE_DIR/proxy.log" +PID_FILE="$STATE_DIR/proxy.pid" +SAVED_PROXY="$STATE_DIR/saved-proxy" +PORT="${RETRY_TEST_PORT:-8899}" +# `system` points the Mac's HTTP(S) proxy at mitmproxy (the simulator uses the Mac's settings). +# `local` uses mitmproxy's local capture of the `App` process instead, which needs its macOS +# network extension approved once, but leaves the Mac's proxy alone. +# `manual` only listens; point a physical device's Wi-Fi proxy at this Mac yourself. +PROXY_MODE="${RETRY_TEST_PROXY_MODE:-system}" +BUNDLE_ID="${RETRY_TEST_BUNDLE_ID:-com.onesignal.example}" +CA_CERT="$HOME/.mitmproxy/mitmproxy-ca-cert.pem" + +export RETRY_TEST_STATE_DIR="$STATE_DIR" +mkdir -p "$STATE_DIR" + +die() { echo "error: $*" >&2; exit 1; } + +route_rule() { + # Prints " " for a route name. Paths are relative to https://api.onesignal.com/. + case "$1" in + create-user) echo "POST ^apps/[^/]+/users$" ;; + fetch-user) echo "GET ^apps/[^/]+/users/by/[^/]+/[^/]+$" ;; + update-user) echo "PATCH ^apps/[^/]+/users/by/[^/]+/[^/]+$" ;; + identity) echo "PATCH /identity$" ;; + remove-alias) echo "DELETE /identity/[^/]+$" ;; + create-sub) echo "POST /subscriptions$" ;; + update-sub) echo "PATCH ^apps/[^/]+/subscriptions/[^/]+$" ;; + delete-sub) echo "DELETE ^apps/[^/]+/subscriptions/[^/]+$" ;; + identity-by-sub) echo "GET /user/identity$" ;; + custom-events) echo "POST /custom_events$" ;; + all) echo "- ." ;; + *) die "unknown route '$1'. Routes: create-user fetch-user update-user identity remove-alias create-sub update-sub delete-sub identity-by-sub custom-events all" ;; + esac +} + +proxy_running() { [[ -f "$PID_FILE" ]] && kill -0 "$(cat "$PID_FILE")" 2>/dev/null; } + +network_service() { + local iface + iface="$(route -n get default 2>/dev/null | awk '/interface:/ {print $2}')" + [[ -n "$iface" ]] || die "no default network route" + networksetup -listnetworkserviceorder | awk -v dev="$iface" ' + /^\([0-9]+\)/ { sub(/^\([0-9]+\) /, ""); name = $0 } + $0 ~ "Device: " dev "\\)" { print name; exit }' +} + +set_system_proxy() { + local service + service="$(network_service)" + [[ -n "$service" ]] || die "could not map the default interface to a network service" + { + echo "$service" + networksetup -getwebproxy "$service" | awk -F': ' '/^(Enabled|Server|Port)/ {print $2}' | paste -sd' ' - + networksetup -getsecurewebproxy "$service" | awk -F': ' '/^(Enabled|Server|Port)/ {print $2}' | paste -sd' ' - + } > "$SAVED_PROXY" + networksetup -setwebproxy "$service" 127.0.0.1 "$PORT" + networksetup -setsecurewebproxy "$service" 127.0.0.1 "$PORT" + echo "Pointed '$service' HTTP(S) proxy at 127.0.0.1:$PORT (restored by 'stop')." +} + +restore_system_proxy() { + [[ -f "$SAVED_PROXY" ]] || return 0 + local service web secure + { read -r service; read -r web; read -r secure; } < "$SAVED_PROXY" + restore_one() { + local kind="$1" enabled server port + read -r enabled server port <<< "$2" + if [[ "$enabled" == "Yes" && -n "$server" ]]; then + networksetup "-set${kind}" "$service" "$server" "$port" + else + networksetup "-set${kind}state" "$service" off + fi + } + restore_one webproxy "$web" + restore_one securewebproxy "$secure" + rm -f "$SAVED_PROXY" + echo "Restored '$service' proxy settings." +} + +install_ca() { + for _ in $(seq 1 50); do [[ -f "$CA_CERT" ]] && break; sleep 0.1; done + [[ -f "$CA_CERT" ]] || die "mitmproxy CA not generated at $CA_CERT" + if xcrun simctl list devices booted | grep -q Booted; then + xcrun simctl keychain booted add-root-cert "$CA_CERT" + echo "Trusted the mitmproxy CA on the booted simulator(s)." + else + echo "No booted simulator. Boot one, then run '$0 trust-ca'." + fi +} + +cmd_start() { + command -v mitmdump >/dev/null || die "mitmdump not found. Install with: brew install mitmproxy" + proxy_running && die "proxy already running (pid $(cat "$PID_FILE")). Run '$0 stop' first." + : > "$ATTEMPTS" + [[ -f "$RULES" ]] || echo "[]" > "$RULES" + local mode_args=(--listen-port "$PORT") + [[ "$PROXY_MODE" == "local" ]] && mode_args=(--mode "local:App") + # Only onesignal.com is decrypted; everything else on the Mac is tunnelled untouched. + mitmdump "${mode_args[@]}" \ + --allow-hosts '^(.+\.)?onesignal\.com(:443)?$' \ + --set connection_strategy=lazy \ + --set termlog_verbosity=warn \ + -s "$SCRIPT_DIR/fault_proxy.py" > "$PROXY_LOG" 2>&1 & + echo $! > "$PID_FILE" + sleep 1 + proxy_running || { cat "$PROXY_LOG"; die "mitmdump exited"; } + echo "mitmdump running (pid $(cat "$PID_FILE")), log: $PROXY_LOG" + case "$PROXY_MODE" in + system) set_system_proxy; install_ca ;; + local) install_ca ;; + manual) + echo "Set the device's Wi-Fi HTTP proxy to $(ipconfig getifaddr en0 2>/dev/null || echo ''):$PORT," + echo "then open http://mitm.it on the device to install and trust the CA." ;; + *) die "unknown RETRY_TEST_PROXY_MODE '$PROXY_MODE'" ;; + esac + echo "Relaunch the demo app so it picks up the proxy: $0 relaunch" +} + +cmd_stop() { + if proxy_running; then + kill "$(cat "$PID_FILE")" && echo "Stopped mitmdump." + fi + rm -f "$PID_FILE" + restore_system_proxy +} + +write_rules() { + # write_rules ; written atomically so the addon never reads a partial file. + echo "$1" > "$RULES.tmp" && mv "$RULES.tmp" "$RULES" +} + +cmd_fail() { + local route="${1:-}" status="${2:-}" + [[ -n "$route" && -n "$status" ]] || die "usage: $0 fail [--times N] [--retry-after S] [--add]" + shift 2 + local times="null" retry_after="null" add=false + while [[ $# -gt 0 ]]; do + case "$1" in + --times) times="$2"; shift 2 ;; + --retry-after) retry_after="\"$2\""; shift 2 ;; + --add) add=true; shift ;; + *) die "unknown option $1" ;; + esac + done + local method path + read -r method path <<< "$(route_rule "$route")" + [[ "$method" == "-" ]] && method="" + python3 - "$RULES" "$add" "$route" "$method" "$path" "$status" "$times" "$retry_after" > "$RULES.tmp" <<'PY' +import json, sys +rules_path, add, route, method, path, status, times, retry_after = sys.argv[1:] +rules = json.load(open(rules_path)) if add == "true" else [] +rules.append({"route": route, "method": method or None, "path": path, "status": int(status), + "times": json.loads(times), "retry_after": json.loads(retry_after)}) +print(json.dumps(rules)) +PY + mv "$RULES.tmp" "$RULES" + echo "Rules now: $(cat "$RULES")" +} + +cmd_clear() { + write_rules "[]" + echo "Cleared all rules; requests pass through." +} + +cmd_reset_log() { : > "$ATTEMPTS"; echo "Cleared recorded attempts."; } + +cmd_network_flap() { + # NWPathMonitor in the simulator follows the Mac's interfaces, so turning Wi-Fi off and on + # exercises the "network returned while foregrounded" trigger. + local seconds="${1:-10}" device + device="$(networksetup -listallhardwareports | awk '/Hardware Port: Wi-Fi/ {getline; print $2}')" + [[ -n "$device" ]] || die "no Wi-Fi interface found" + echo "Wi-Fi ($device) off for ${seconds}s..." + networksetup -setairportpower "$device" off + sleep "$seconds" + networksetup -setairportpower "$device" on + echo "Wi-Fi back on." +} + +cmd_fresh() { + # Wipes the demo's data and app-group containers so the SDK starts as a new install (Create + # User and so on), without uninstalling, so an already granted notification permission is kept + # and the permission prompt does not hold the app inactive. + local data group plist + data="$(xcrun simctl get_app_container booted "$BUNDLE_ID" data)" || die "demo app not installed on the booted simulator" + xcrun simctl terminate booted "$BUNDLE_ID" 2>/dev/null || true + # Preferences go through the simulator's cfprefsd, which would write its cached copy back. + for plist in "$data"/Library/Preferences/*.plist; do + [[ -f "$plist" ]] && xcrun simctl spawn booted defaults delete "${plist%.plist}" 2>/dev/null || true + done + rm -rf "${data:?}/Documents/"* "${data:?}/Library/Caches/"* "${data:?}/Library/Preferences/"* "${data:?}/tmp/"* + while read -r _ group; do + [[ -d "$group" ]] || continue + for plist in "$group"/Library/Preferences/*.plist; do + [[ -f "$plist" ]] && xcrun simctl spawn booted defaults delete "${plist%.plist}" 2>/dev/null || true + done + rm -rf "${group:?}/"* + done < <(xcrun simctl get_app_container booted "$BUNDLE_ID" groups 2>/dev/null) + echo "Cleared demo app data." + xcrun simctl launch booted "$BUNDLE_ID" +} + +cmd_logs() { + xcrun simctl spawn booted log stream --level debug --style compact \ + --predicate 'process == "App" AND (eventMessage CONTAINS[c] "request" OR eventMessage CONTAINS[c] "retry" OR eventMessage CONTAINS[c] "re-attempt")' +} + +usage() { + cat < + +Proxy + start Start mitmproxy, point the proxy at it, trust its CA on the booted simulator + stop Stop mitmproxy and restore proxy settings + trust-ca Trust the mitmproxy CA on the booted simulator(s) + +Faults (take effect immediately, no relaunch needed) + fail [--times N] [--retry-after S] [--add] + Fail matching SDK requests. Status 0 drops the connection. + --times N: fail N times, then pass. Default: until cleared. + --add: keep existing rules instead of replacing them. + clear Remove all rules + rules Show current rules + +Observe + watch Follow every SDK request as it passes the proxy + report [route regex] Gaps between attempts per route vs. the expected backoff + reset-log Forget recorded attempts + logs Stream the demo app's SDK request/retry logs + +App / environment + relaunch Kill and relaunch the demo app on the booted simulator + fresh Wipe the demo app's data (new SDK user) and relaunch, keeping the + notification permission. Allow the prompt once after installing. + background | foreground Send the demo app to the background, or bring it back + network-flap [seconds] Turn the Mac's Wi-Fi off and back on (default 10s) + +Routes: create-user fetch-user update-user identity remove-alias create-sub update-sub + delete-sub identity-by-sub custom-events all +Env: RETRY_TEST_PORT ($PORT), RETRY_TEST_PROXY_MODE ($PROXY_MODE), RETRY_TEST_BUNDLE_ID ($BUNDLE_ID) +EOF +} + +case "${1:-help}" in + start) cmd_start ;; + stop) cmd_stop ;; + trust-ca) install_ca ;; + fail) shift; cmd_fail "$@" ;; + clear) cmd_clear ;; + rules) cat "$RULES" 2>/dev/null || echo "[]" ;; + watch) tail -n +1 -F "$PROXY_LOG" | grep --line-buffered '^\[fault\]' ;; + report) shift; python3 "$SCRIPT_DIR/report.py" "$ATTEMPTS" "$@" ;; + reset-log) cmd_reset_log ;; + logs) cmd_logs ;; + fresh) cmd_fresh ;; + relaunch) xcrun simctl terminate booted "$BUNDLE_ID" 2>/dev/null || true; xcrun simctl launch booted "$BUNDLE_ID" ;; + background) xcrun simctl launch booted com.apple.Preferences >/dev/null && echo "Demo app backgrounded." ;; + foreground) xcrun simctl launch booted "$BUNDLE_ID" >/dev/null && echo "Demo app foregrounded." ;; + network-flap) shift; cmd_network_flap "$@" ;; + help|-h|--help) usage ;; + *) usage; exit 1 ;; +esac From 6f139cc9f73ed678f2191aadbebf2781edd95775 Mon Sep 17 00:00:00 2001 From: Fadi George Date: Tue, 13 Oct 2026 13:44:21 -0700 Subject: [PATCH 02/10] test: [SDK-5481] drive the demo UI from the retry harness Co-Authored-By: Claude Opus 5.5 --- examples/demo/scripts/retry/README.md | 51 +++++----- .../demo/scripts/retry/flows/_scroll-to.yaml | 11 +++ .../demo/scripts/retry/flows/add-alias.yaml | 16 ++++ .../demo/scripts/retry/flows/add-email.yaml | 13 +++ .../demo/scripts/retry/flows/add-tag.yaml | 16 ++++ .../retry/flows/allow-notifications.yaml | 6 ++ examples/demo/scripts/retry/flows/login.yaml | 13 +++ examples/demo/scripts/retry/flows/logout.yaml | 8 ++ .../demo/scripts/retry/flows/toggle-push.yaml | 8 ++ examples/demo/scripts/retry/retry-test.sh | 92 +++++++++++++++---- 10 files changed, 192 insertions(+), 42 deletions(-) create mode 100644 examples/demo/scripts/retry/flows/_scroll-to.yaml create mode 100644 examples/demo/scripts/retry/flows/add-alias.yaml create mode 100644 examples/demo/scripts/retry/flows/add-email.yaml create mode 100644 examples/demo/scripts/retry/flows/add-tag.yaml create mode 100644 examples/demo/scripts/retry/flows/allow-notifications.yaml create mode 100644 examples/demo/scripts/retry/flows/login.yaml create mode 100644 examples/demo/scripts/retry/flows/logout.yaml create mode 100644 examples/demo/scripts/retry/flows/toggle-push.yaml diff --git a/examples/demo/scripts/retry/README.md b/examples/demo/scripts/retry/README.md index ca5da8e3b..7ff3bb734 100644 --- a/examples/demo/scripts/retry/README.md +++ b/examples/demo/scripts/retry/README.md @@ -6,21 +6,25 @@ SDK's `SDK-Version` header. The demo's own REST calls and all other traffic pass `onesignal.com` traffic is decrypted. ```bash -brew install mitmproxy +brew install mitmproxy mobile-dev-inc/tap/maestro cd examples/demo/scripts/retry ./retry-test.sh help ``` +[Maestro](https://maestro.mobile.dev) drives the demo's UI (`tap`) through the flows in `flows/`. + ## Setup -1. Build and install the demo app on a simulator (see `examples/demo/README.md`), launch it once, - and allow notifications. The prompt holds the app inactive, so foreground retries never fire - while it is showing. Use `fresh` instead of reinstalling, so the permission is kept. -2. `./retry-test.sh start` starts mitmproxy on port 8899, points the Mac's Wi-Fi/Ethernet HTTP(S) - proxy at it (the simulator uses the Mac's proxy), and trusts its CA on the booted simulator. -3. `./retry-test.sh fresh` wipes the demo's data and relaunches it as a new install. -4. In another terminal, `./retry-test.sh watch` follows each SDK request. -5. When done, **`./retry-test.sh stop`** restores the Mac's proxy settings. +1. `xcrun simctl create "OneSignal Retry Test" "iPhone 17"`. The harness uses its own simulator + (`RETRY_TEST_DEVICE`, by name or UDID) because the other OneSignal demos share the bundle ID. +2. `./retry-test.sh install` builds the demo and installs it there. +3. `./retry-test.sh start` starts mitmproxy on port 8899, points the Mac's Wi-Fi/Ethernet HTTP(S) + proxy at it (the simulator uses the Mac's proxy), and trusts its CA on the simulator. +4. `./retry-test.sh fresh`, then `./retry-test.sh tap allow-notifications`. The permission prompt + holds the app inactive, so foreground retries never fire while it is showing. `fresh` wipes the + demo's data without reinstalling, so the permission is kept from then on. +5. In another terminal, `./retry-test.sh watch` follows each SDK request. +6. When done, **`./retry-test.sh stop`** restores the Mac's proxy settings. Leave the Mac's proxy alone with `RETRY_TEST_PROXY_MODE=local` (mitmproxy captures the `App` process, which needs its network extension approved once). For a physical device, use @@ -34,6 +38,8 @@ process, which needs its network extension approved once). For a physical device | `clear` / `rules` | Remove or show the rules | | `report [regex]` | Per route: each attempt, the gap since the previous one, and the gap expected | | `reset-log` | Forget recorded attempts, for example between scenarios | +| `tap [args]` | Use the demo: `add-tag `, `add-alias