feat(sponsorblock): experimental on-device sponsor detection with a skip button (GitHub flavor)

[seshuthota]

Summary

Experimental, opt-in, GitHub-flavor-only. When SponsorBlock has no segments for a video, a small on-device model reads the captions and, if it finds a sponsor, shows the existing skip button for it. It never skips on its own.

• Off by default. Settings > Integrations > SponsorBlock > "On-device detection (experimental)". Enabling it starts a one-time model download (~21.5 MB: 17.97 MB model + 3.6 MB tokenizer), with progress, retry, delete, and a link to the model page.
• Only runs when the SponsorBlock lookup returns nothing, and not for live streams. Results are cached per video/transcript.
• Predictions are marked with a flow-ml- uuid prefix. SponsorBlockHandler never auto-skips them and SponsorBlockSkipPolicy always shows the manual skip button for them, even if the category is set to auto-skip.
• Fixes the skip button's position in the two-pane layout (it was drawn over the related list because the overlay used the full pane width).
• GitHub flavor only, same pattern as Discord: implementation in app/src/github, no-op stubs in app/src/foss, githubImplementation for ONNX Runtime. The FOSS APK has no ONNX Runtime (arm64 release: 19.5 MB) and the setting is hidden there.
• ONNX Runtime is a reduced-operator build (only what this model needs) built by tools/onnxruntime/build-onnxruntime-android.sh; no AAR is committed. The arm64 .so is 2.9 MB vs 32.1 MB for the Maven artifact (AAR 4.7 MB). assembleGithubRelease/Nightly fail with a clear message if app/libs/onnxruntime-android-1.29.0.aar is missing; GitHub debug builds fall back to the Maven artifact so contributors aren't blocked. CI builds the AAR (cached by hash of tools/onnxruntime/**) before lint/release. -keep class ai.onnxruntime.** { *; } is added to ProGuard rules because the custom AAR ships no consumer rules.

Model

• Repo: https://huggingface.co/CuriousDragon/ettin-17m-sponsor-combined-android-int8-mm
• The app downloads a pinned revision (0b8399f4f8bc226b21d8c8f39d4611abd536890c) and verifies SHA-256 for both files before installing, so the download can't change under an installed build. Changing the model means bumping the revision and constants in SponsorModelStore.kt together.
• Base: jhu-clsp/ettin-encoder-17m (MIT), fine-tuned as a BILOU token classifier (768-token windows, 128 overlap, threshold 0.7).
• Size: 28.97 MB -> 17.97 MB (-38%) via selective dynamic INT8 (embeddings, attention and FFN MatMuls in middle layers; first-layer FFN-out and the classifier head stay FP32). Held-out span F1 (FP32 -> this model): 0.7355 -> 0.7308 combined, 0.5700 -> 0.5625 ScriptSmith, 0.8582 -> 0.8515 Xenova. About 17% faster per window on a Xiaomi Pad 6 (196 vs 237 ms/window, pinned cores).

License and training data

• Model weights: CC BY-NC-SA 4.0, because the labels derive from the SponsorBlock database (CC BY-NC-SA 4.0, https://sponsor.ajay.app/). The base encoder is MIT. The app downloads the weights at runtime; they are not bundled in the APK or this repo.
• Training data:
  • SponsorBlock sponsor labels (CC BY-NC-SA 4.0).
  • Xenova/sponsorblock-768: no license is stated on its dataset page.
  • ScriptSmith/sponsorblock-youtube-metadata-2024: CC BY-4.0 (covers the compilation, not the underlying YouTube transcripts).
• The non-commercial/share-alike terms apply to the weights only, not to the app's code. Worth a look from you given the Xenova dataset has no stated license.

Related issue

None.

Change type

• Feature
• Build, packaging, or CI

Validation

• ./gradlew :app:assembleGithubDebug: passes
• ./gradlew :app:testGithubDebugUnitTest and :app:testFossDebugUnitTest: pass (the sponsor tests that need the model files ran with -PsponsorModelAssets=<dir>, otherwise they skip)
• ./gradlew ktlintCheck, :app:compileFossDebugKotlin, :app:assembleGithubRelease :app:assembleFossRelease (with the AAR present): pass
• assembleGithubRelease without the AAR fails with the message above; GitHub debug and FOSS compile without the AAR
• R8 mapping of the release build keeps the 71 ai.onnxruntime classes
• Not run: the CI AAR-build step (Docker on GitHub runners), the release/FOSS APKs on a device, the phone layouts
• Manually tested on a device (earlier revision of this branch, before the github/foss split): model download from the pinned revision, detection on a video with no SponsorBlock segments (22 windows, ~5.6 s, one span found), no auto-skip, skip button inside the video after the position fix. I have not re-run it on a device after the split and rebase.

Test device and Android version: Xiaomi Pad 6 (arm64, MIUI/HyperOS), debug build.

Risk and compatibility

• Experimental: it can miss a sponsor or mark the wrong part of a video (the 39-video benchmark had 19 positives; labels are noisy). The settings text says so.

• Network: one model download, only after the user enables the toggle. No telemetry. Nothing is sent about videos; captions are read locally.

• Background/battery: inference runs only when SponsorBlock has no segments and the toggle is on. Battery and thermal impact are not measured.

• Preferences: adds one DataStore key (sponsorOnDeviceEnabled, default false). No Room schema change.

• Size: GitHub APK includes the 2.9 MB (arm64) runtime; the model is downloaded, not bundled. FOSS is unaffected.

• The model is hosted on a personal Hugging Face account, as discussed.

• The change does not introduce secrets, private data, or unexpected telemetry.

• New user-facing text uses Android string resources (English only).

• Dependency changes are limited to this PR (ONNX Runtime, GitHub flavor only).

• This PR does not change the Room schema.

• No breaking changes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LeLeHxbB5VJpe961SJVMpd