Repository navigation
test: [SDK-5535] add a fault-injection harness for operation retries - #2791
Conversation
16da0eb to
d5e8ecf
Compare
📊 Diff Coverage Report✓ Coverage check passed (no source files changed) |
21a8eaf to
5a731af
Compare
There was a problem hiding this comment.
Multi-model review (Opus 5.5, GPT 5.6 Sol, Grok 4.7)
Backoff math, route regexes, and debug-only CA trust look right for the single-operation happy path. A few harness bugs will make report lie or leave the device in a bad state.
Act on (3/3)
report.pygroups by method + stripped path, so different records share one backoff counter. Scenario 11 (and any parallel ops on the same route) will print false EARLY/LATE. Key on the full path or body hash.tappipes Maestro throughgrep … || true, so a failed flow still exits 0. Scenarios that assert “not sent” become untrustworthy.network-flaphas no trap and always enables both radios. Ctrl-C, a failedsvc, or wireless adb leaves the device offline.
Consider (2/3)
--addreloadsremainingfrom originaltimes, so an in-flight--times Nrule starts over.- 404 is treated as terminal, but user executors retry 404 inside the post-create window.
Noted
- Bind mitmdump to
127.0.0.1(it currently listens on all interfaces). fail … 0 --retry-afterrecords a floor the SDK never sees.float()Retry-After vs SDKtoIntOrNull()(e.g.1.5→ report 1.5s, SDK 60s).
Dismissed
- Release CA leak —
debug-overridesplus the debug source set only.
Sent by Cursor Automation: PR Reviews
abdulraqeeb33
left a comment
There was a problem hiding this comment.
Add a trap in retry-test.sh that turns the radios back on and clears the proxy on exit. Drop || true on Maestro. Update the body, since #2790 now removes the global delay.
5a731af to
e54c53b
Compare
b70b016 to
17692ea
Compare
Port of the iOS retry harness to the Android demo: mitmproxy fails SDK requests, Maestro drives the demo, and report.py checks retry gaps against OperationRepo's per-op backoff. Debug demo builds trust the mitmproxy CA when built with -PRETRY_TEST_CA. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… operation Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… review issues Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
17692ea to
8dcf953
Compare
|
@abdulraqeeb33 rebased onto main, so this is just the harness now. Redid the fixes in 8dcf953:
From the Cursor review: I left the device proxy alone on exit since it's meant to stay set between commands until Reran backoff, other records, foreground, and network return on an emulator. Details are in the body. |
…er check Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abdulraqeeb33
left a comment
There was a problem hiding this comment.
Thanks, all the earlier points are fixed and the docs match what merged. Two small ones in retry-test.sh:
- A typo'd route installs a match-all rule.
dieinside$(route_rule ...)only exits the subshell, soreadgets empty strings and the empty path matches every request.spec="$(route_rule "$route")" || exit $?before thereadfixes it. network-flaponly refuses serials with:. Android 11+ wireless debugging serials look likeadb-XXXX._adb-tls-connect._tcp, so Wi-Fi goes off, adb drops, and the radios stay off. Also refuse*._adb-tls-connect._tcp, and don't print "Network restored." whensvc enablefails.
Nits: restore stylus_handwriting_enabled on stop, and skip a truncated last line in report.py instead of crashing.
… in the retry harness Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@abdulraqeeb33 thanks, both were real. Fixed in 2c8c68d:
I'd leave |
…erification apps Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>


Description
One Line Summary
Adds
examples/demo/scripts/retry, a harness that injects failures into the demo's SDK requests and checks retry timing. It is a port of the iOS harness from OneSignal/OneSignal-iOS-SDK#1775.Details
Motivation
Unit tests for #2790 mock the executors, so they don't show real retry gaps, or whether foreground and network return wake backed-off operations on a device. This lets anyone reproduce those cases against the demo app.
Scope
fault_proxy.py: mitmproxy addon, based on iOS. It fails only requests with theSDK-Versionheader and logs every attempt, timed at the request. It listens on loopback only.report.py: per record (or per body for id-less routes), the gap between attempts vs the expected window: 5s doubling to 300s times 0.5-1 jitter,Retry-Afteras the minimum, and no client-side re-sends. It also lists any SDK request sent whileHttpClient's Retry-After gate should have held it.retry-test.sh: rewritten for adb. It sets the device's global HTTP proxy (10.0.2.2on emulators,adb reverseon devices), clears data withpm clear, presses Home to background, and turns Wi-Fi and data off for a network flap, restoring only what was on even after Ctrl-C. A failed Maestro flow failstap.installbuilds the demo against local SDK source.network_security_configwithdebug-overridesonly. When built with-PRETRY_TEST_CA=<pem>, the debug build also trusts that CA, so no certificate has to be installed on the device. Release builds are unaffected.Testing
Manual testing
Android 16 emulator (API 36), demo built from main after #2790:
Affected code checklist
Checklist
Overview
Testing
Final pass
🤖 Generated with Claude Code