Repository navigation
Conversation
Copies the channel's compressed audio at the extractor (progressive via AudioSyncExtractorsFactory, HLS via a chunk-extractor wrapper), decodes it on a side thread and cuts it into speech segments with Silero, logging segment media times, look-ahead and player period position under tag LiveAiSubs. Adds SubtitleLookaheadLoadControl, which holds playback until D seconds are buffered, only while required. Everything is inert unless switched on over adb in a debug build (debug.arvio.liveaisubs*). No UI and no AI calls yet. Also makes the tap's output wrappers internal and the segmenter's maximum segment length configurable.
… playback Live channels get AI subtitles in the viewer's language that appear on time: the channel's audio is copied as it downloads (extractor tap), cut into speech segments with Silero, and each segment is sent to Gemini, which transcribes and translates it in one call. Lines are timed from the segment's own media time and shown by the player position, while SubtitleLookaheadLoadControl holds playback back so answers arrive first. - Pipeline skips its backlog when analysis falls behind playback (busy boxes) - Segments end at speech pauses after 3.5 s, hard limit 5 s - Up to 2 requests in flight, applied in order; 429 backoff, invalid key, offline and unsupported-audio states with toasts - Delay Auto (default, from measured latency) / 6 / 10 / 15 s; "Preparing subtitles" while held; pause-to-build when turned on mid-channel - Settings (AI subtitles section): enable with key check, delay, activation per channel / always, target language, own Gemini key when the AI subtitles model is not Gemini; cloud synced - Subtitle picker gets an "AI subtitles" entry, remembered per channel - Overlay uses the profile's subtitle style; hidden debug panel via debug.arvio.liveaisubs.overlay - Strings in en, iw, de, es, pt, pt-BR; unit tests for scheduler, delay math, Gemini JSON parsing and the picker option
Picking "AI subtitles" turns them on for that channel even when the settings switch is off; the switch now only makes channels start with them (together with Always on).
…m the start - Settings are now just the feature switch and "auto-translate when a channel starts"; the delay is always measured, the language is the profile's subtitle language and the key is the AI subtitles Gemini key. - Switching to AI subtitles while a channel plays restarts the stream, so the audio that was already buffered is subtitled too; "Preparing subtitles" shows during the hold and until the first line arrives. - Audio is captured whenever AI subtitles are on for the channel (mini player included), so the lines for the start-up hold are not lost.
Once playback is held back, every coming line is translated in time, so the extra note after the hold only misled during quiet stretches.
…udio as fallback - Tap text tracks next to audio: CEA-608 closed captions and parsed text renditions (WebVTT etc.) become sentences that are translated as text; audio is only sent for stretches the captions do not cover. - Picture subtitles (DVB, PGS, VobSub) are never used, including ones Media3 parses into cues during extraction (original type in Format.codecs). - Auto-translate does not start on a channel that already has subtitles in the profile language.
The hidden look-ahead/latency panel (debug.arvio.liveaisubs.overlay) and its counters were for measuring on a device; the measured AI latency still drives the Auto delay.
github-actions
Bot
added
the
area: android
…t run, keep catch-up position - Tag audio and caption work with the channel source it was captured from and drop stale work before sending, so an earlier channel's lines never reach the next one. - When AI cannot run (no or rejected key, undecodable audio) skip the look-ahead hold and show the channel's own subtitles; during a quota or offline pause show them too. - Do not restart the stream when AI is switched on during catch-up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
AI subtitles for Live TV: translated subtitles for live channels in the profile's subtitle language, using the user's own Gemini key.
Live TV has no "ahead" to read from, so playback is held back a few seconds. The app hears the broadcast before the viewer does, so each line can be translated and shown on time.
How it works
LoadControlwrapper (SubtitleLookaheadLoadControl) keeps playback from starting until enough is buffered. The hold covers one speech segment plus the measured AI answer time plus a margin, clamped to 6–15 s ("Auto"). While it waits, a short "Preparing subtitles… N s" note shows.Format.codecs).gemini-3.5-flash-liteas a WAV in onegenerateContentcall, which transcribes and translates. JSON out, minimal thinking.Settings (Settings → AI subtitles)
Effect on existing code
The regular (VOD) player behaves as before. Shared code only gained optional parameters with the old defaults:
SpeechSegmenter: segment-length parameters.AudioSyncDecoder: thread priority.AudioTapExtractorOutput/AudioTapTrackOutput/ JSON helpers inSubtitleTranslationService: madeinternalso Live TV can reuse them, logic unchanged.Measured on a Mi Box 4 (English news channel → Hebrew)
Tests
LiveCueSchedulerTest,GeminiLiveProtocolTest,Cea608TextAssemblerTestandLiveAiSubtitlesOptionTestare new. All Live TV unit tests pass.