Skip to content

Add Stereo Call Recording Core For STS - #1596

Merged
swu-wise[bot] merged 9 commits into
mainfrom
claude/salvage-sts-recorder
Nov 6, 2026
Merged

swu-wise[bot] merged 9 commits into
mainfrom
claude/salvage-sts-recorder

Conversation

@jaywedgeworth22

@jaywedgeworth22 jaywedgeworth22 commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Refs AFC-618

Summary

Adds core STS call-recording primitives that produce lossless, uncompressed stereo WAV files for post-call debugging, with the person on the left channel and the bot on the right so speakers remain distinguishable. The source documents recording as off by default and retains lossless audio as debugging evidence.

  • src/lib/sts/recorder.ts records user and bot PCM16 audio independently, copies reused decode buffers, drops incomplete trailing bytes, and contains recording failures so they do not interrupt a call.
  • Tracks captured at different rates, such as 16 kHz microphone audio and 24 kHz provider audio, are resampled to 16 kHz before being interleaved. The shorter track is padded with silence rather than truncated.
  • The recorder reports duration and elapsed time, returns no recording when both tracks are silent, and ignores late writes after a non-empty recording has been finalized.
  • Helpers generate timestamped, path-sanitized recording names and identify recordings that have reached the 30-day retention window. The supplied patch does not show call-path enablement, file persistence, downloads, or automatic deletion.
  • src/lib/sts/recorder.test.ts adds coverage for WAV headers and sizes, channel placement, silence padding, empty and truncated input, reused buffers, finalized recordings, elapsed time, resampling, mixed sample rates, file naming, and retention boundaries.

No test execution results or live-call runtime verification are reported.

Unresolved Review Findings

The automated review reports eight findings: three high severity and five medium severity.

  • High: Independently captured tracks have no shared start timeline, so different capture start times can misalign the stereo channels.
  • High: Recording names can collide for the same bot within the same minute, potentially overwriting an earlier recording.
  • High: Omitting the optional sample rate can cause 24 kHz bot audio to be treated as 16 kHz.
  • Medium: The invalid-frame test uses null as never, bypassing TypeScript’s frame-type check.
  • Medium: Calling stop() before any audio is recorded does not freeze the recorder, allowing subsequent writes.
  • Medium: Finalization duplicates buffered audio and performs per-sample WAV writes, increasing memory use and processing cost for longer calls.
  • Medium: The recorder retains copied PCM chunks after creating the WAV.
  • Medium: The generated recording name contains underscores that conflict with the repository’s SAFE_NAME rule.

Summary

  • Adds a shared-clock stereo recorder for STS calls, producing a 16 kHz PCM16 WAV with the user on the left channel and the bot on the right.
  • Aligns delayed speech by padding gaps longer than 100 ms with silence while preserving small timing jitter and queueing early frames behind existing audio.
  • Requires an explicit sample rate for every frame, locks each track to its first accepted rate, and counts refused rate or duration-limit frames through droppedFrames.
  • Limits each track to one hour by default, supports a configurable cap, freezes empty and completed recorders, and releases raw track buffers after creating the WAV.
  • Updates recording filenames to include seconds and a sanitized call ID, preventing collisions while retaining sortable, attachment-compatible names.

Tests

  • Updates existing recorder tests to provide sample rates.
  • Adds coverage for shared-clock alignment, silence padding, jitter and early-frame handling, rate mismatches, duration caps, large clock jumps, stopped-recorder behavior, retained elapsed time, filename sanitization, and filename collisions.
  • These tests were added or updated in the patch; no test execution results were provided.

Open review findings

The supplied automated review reported 10 unresolved findings: 9 medium and 1 low.

  • The cap test uses the wall clock and can fail after a 100 ms pause.
  • Shared-clock padding can insert artificial silence into the microphone track.
  • An invalid maxTrackSeconds can bypass or defeat the recording cap.
  • Large silence gaps can cause large synchronous buffer allocations.
  • Raw track chunks remain resident while the WAV is allocated.
  • A WAV build failure causes stop() to permanently lose the recording.
  • droppedFrames combines rate errors with recording-cap rejections.
  • The default wall clock can corrupt a recording after a system time synchronization.
  • Recorder timing depends on a clock that can jump.
  • Recorder tests contain timing-dependent expectations.

Summary
Updates the STS stereo call recorder core that captures user and bot audio into one WAV with left=user and right=bot, aligned on a shared clock and emitted at the recording sample rate. The diff is limited to the recorder module and tests; no STS call-path integration is included.

Changes

  • Track append results now distinguish rate rejection from cap rejection; droppedFrames counts all refused frames and new cappedFrames counts only cap refusals.
  • Silence padding is applied only to the bot/provider track, which is intermittent; the continuous microphone track is never padded, so late microphone frames queue instead of splitting speech.
  • Provider-track silence gaps are stored as sample counts instead of allocated buffers, reducing memory until the WAV is built.
  • Invalid maxTrackSeconds values (NaN, zero, negative, infinite) fall back to MAX_TRACK_SECONDS, preventing an unbounded cap or a cap that discards every frame.
  • The default recorder clock uses performance.now() when available with Date.now() fallback, so wall-clock steps do not change recording placement or duration.
  • stop() releases raw track chunks before allocating the WAV to lower peak memory, and catches WAV build failures so teardown receives null instead of an exception; a failed build loses that recording and later stop() calls return null. stop() remains idempotent.
  • Adds an optional buildWav injection point for failure-path testing.

Tests

  • Existing recorder tests now pass a deterministic now function.
  • Adds tests for microphone no-padding behavior, invalid cap fallback, separate cap/rate refusal counts, long provider pauses and WAV offsets, WAV build failure containment, and the monotonic default clock. No test-run output is included in the provided context.

Review
Automated code review reported no issues in the reviewed changes.

The agreed scope from the TTS/STT/STS work, as a foundation rather than a
reachable feature: nothing wires this to a call yet.

Left is the person and right is the bot.  A mono mix is no use for the one
thing a call recording is for, which is working out afterwards who said what,
and the bot's own voice reaches the microphone through the room anyway, so a
mix cannot even separate them reliably.

The two sides are not the same rate, which is the thing most likely to ship
broken: the capture path resamples the microphone to 16 kHz while the
providers speak back at their own rate, and Gemini is 24 kHz.  One WAV has one
sample rate, so each track is brought to the file's rate before the
interleave.  Writing the 24 kHz track in beside 16 kHz audio plays the bot
fast and an octave high, which would make a debugging recording useless.

Uncompressed WAV on purpose: this is off by default and exists to find out
what a call actually said, so lossless means evidence rather than a lossy copy
of it, and it is the only part verifiable without a live microphone.

26 tests, no platform fakes needed.

Co-authored-by: Jay Wedgeworth <12656028+jaywedgeworth22@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Nov 4, 2026

Copy link
Copy Markdown

AFC-618

@kody-ai

This comment has been minimized.

Comment thread src/lib/sts/recorder.test.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
@swu-wise
swu-wise Bot marked this pull request as ready for review November 6, 2026 03:40
Comment thread src/lib/sts/recorder.ts Resolved
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor
Autofix run https://github.com/Simple-With-Us/BotFleet/actions/runs/42158134143 started for `sentry[bot]`'s tagged request. Owner/sentry request: exempt from the per-bot cap (owner rule 2026-10-20).

swu-wise Bot added 2 commits November 6, 2026 03:52
…ate, freeze on empty stop, release raw audio, typed-array WAV fill, unique call-id file names
…ount frames past it, no unbounded silence allocation
@swu-wise

swu-wise Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Follow-up on the review replies, head 3516afe: this adds a behavior the first push did not have, so it is recorded here.

Each track is now capped at 3600 s of audio (MAX_TRACK_SECONDS, overridable per recorder with maxTrackSeconds). That is about 115 MB at 16 kHz and 173 MB at 24 kHz per track. A frame that would push a track past the cap, including any silence padding it needs, is refused and counted in droppedFrames; what was already recorded is kept. This also stops a large clock jump from allocating a huge silence block, and it matches the module's own rule that write must not allocate unboundedly.

Limits: a call longer than an hour records only its first hour on that track, and the caller has to read droppedFrames to know. Peak memory inside stop() is still about two copies of one track (contiguous view plus resample). I have not measured it on a real hour-long call. Gaps under 100 ms are treated as clock jitter and not padded, so the two sides can be up to that far apart. Bot audio sent faster than real time queues behind earlier bot audio instead of landing at its arrival time.

Evidence: pnpm exec tsc -p tsconfig.json --noEmit clean; src/lib/sts/recorder.test.ts 35 pass. The two new cap tests fail when the cap check is removed. CI for this head was still running when posted.

@swu-wise

swu-wise Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

@kody start-review

Please review head 3516afe. All 9 earlier Kody threads are fixed or answered with evidence and resolved, and the added per-track memory cap is described in the PR comment above. CI is green on this head.

@kody-ai

This comment has been minimized.

Comment thread src/lib/sts/recorder.test.ts Resolved
Comment thread src/lib/sts/recorder.test.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
Comment thread src/lib/sts/recorder.ts Resolved
swu-wise Bot added 2 commits November 6, 2026 05:24
…inite cap validation, release before WAV build, build failures contained, cappedFrames, monotonic default clock, fake clocks in tests
@swu-wise

swu-wise Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Correction to two replies on the previous review round (head d007c5d): two things I said were untested now have tests.

  • The stop() build-failure catch: createStsCallRecorder takes an optional buildWav (defaults to buildStereoWav), and a test injects one that throws RangeError. stop() returns null and does not throw.
  • The default clock: a test mocks performance.now and steps Date.now ten minutes forward; the recording is unaffected. It fails when the default is switched back to Date.now.
    tsc is clean and the recorder tests pass (41). Peak memory at the cap is still unmeasured on a real call.

@swu-wise

swu-wise Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

@kody start-review

Please review head ec7dbb5. The 10 threads from your last review are fixed or answered with evidence and resolved. This head changes behavior (bot-only gap padding, silence stored as counts, finite cap validation, release before WAV build, monotonic default clock), so it needs a fresh look. CI is green.

@kody-ai

kody-ai Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Kody Review Complete

Great news! 🎉
No issues were found that match your current review configurations.

Keep up the excellent work! 🚀

Kody Guide: Usage and Configuration
Interacting with Kody
  • Request a Review: Ask Kody to review your PR manually by adding a comment with the @kody start-review command at the root of your PR.

  • Validate Business Logic: Ask Kody to validate your code against business rules by adding a comment with the @kody -v business-logic command.

  • Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Current Kody Configuration
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ✅

Access your configuration settings here.

​

@swu-wise
swu-wise Bot merged commit 48abfc8 into main Nov 6, 2026
16 checks passed
@swu-wise
swu-wise Bot deleted the claude/salvage-sts-recorder branch November 6, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants