Skip to content

Add transcription, recording and a saved history to the main window - #275

Open
alexkroman wants to merge 3 commits into
mainfrom
transcription-history
Open

alexkroman wants to merge 3 commits into
mainfrom
transcription-history

Conversation

@alexkroman

Copy link
Copy Markdown
Collaborator

Summary

  • Transcribe files: drop audio/video on the window or Browse…; uploaded to the async API (any length, speaker labels), titled by the LLM, opened in a transcript sheet.
  • Record: mic + Mac audio together (Core Audio process tap + aggregate device), streamed to the realtime API so words, partials included, appear live; pause/resume; diarized by an async pass after Stop.
  • One saved history (history.json): every dictation, read-aloud clip (with its audio), file and recording, in a searchable, filterable Recent list (All / Dictations / Read Aloud / Transcripts), with ⌘F, double-click/Return to open, ⌘C, Delete with confirmation.
  • Transcript sheet: speaker-coloured turns, Plain text, LLM tasks on gemma-4-31b (meeting notes, summary, key points, to-do list, entities, speaker summary/quotes, custom), playback (the original audio on Full, read-aloud for task results), Show in Finder.
  • Auto-record meetings (Settings › General, off by default): starts when a meeting app or browser call takes the mic, stops a minute after.
  • Main window: compact key rows with each key's own tap/hold mode, a drop well, native list selection.
  • Ask: a short gap between the end-ask and start-read cues.

Guardrails

Two carve-outs, recorded in AGENTS.md, the guardrails skill and check-invariants.sh (each exempting one file): the Record button's realtime socket (RealtimeTranscriber.swift) and the file-transcript LLM Gateway client (TranscriptTransform.swift). Dictation's request is unchanged; saved dictations are copied from the in-memory ring, so a secure-field dictation is never saved.

Test plan

  • scripts/check.sh green (engine coverage 90.8%)
  • New engine suites: history store, async client, realtime client (loopback WebSocket), live transcript, recorder mix, read clips, meeting auto-record, transforms/titles
  • New UI tests (TranscriptionUITests, TranscribeFileUITests) pass locally against a seeded history and a canned transport
  • Full UI suite on CI (locally, harness-dependent tests fail identically on main on this machine)
  • Manual: record a two-person call, check live turns, diarization after Stop, Play, Show in Finder

🤖 Generated with Claude Code

alexkroman-assembly and others added 3 commits October 30, 2026 18:27
Transcribe dropped or browsed audio files through the async API (any
length, speaker labels), record the mic and Mac audio together with a live
realtime transcript, and keep every dictation, read-aloud clip, file and
recording in one searchable, filterable Recent list. Each entry opens in a
sheet with Full, Plain text, LLM tasks (meeting notes, summary, key points,
to-do list, entities, speaker summary/quotes, custom), playback and Show in
Finder. Optional auto-record for meetings in Settings > General.

Guardrails: carve-outs for the Record button's realtime socket and the
file-transcript LLM Gateway client, recorded in AGENTS.md, the guardrails
skill and check-invariants.sh. Dictation's request is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Code review:
- A failed async pass no longer deletes a recording; the history isn't
  saved before it has loaded; a closed socket ends Stop's wait at once.
- The live-failure path drops its dead error check: the async pass after
  every recording re-transcribes the whole file anyway.
- Pausing keeps the realtime socket open, now documented as deliberate.

Simplify:
- LiveTranscript caches its speaker turns per final, and the live view
  draws partials beside an equatable SpeakerTurnsView.
- The open entry is stored and refreshed from history's didSet; playback
  is tracked by entry id, so no row resolves a bookmark to ask.
- The Recent list re-filters only when the history, filter or query
  changes.
- The realtime transcriber sends the recorder's hand-offs whole (split
  only past the 1000 ms ceiling); the mixer appends without a temporary.
- Shared helpers: SpeakerTurn.isMultiSpeaker, AudioRoute.read,
  Date.relativeLabel, RealtimeTranscriber.chunkBytes, GatedSpeaker.
- TranscriptsModel takes its API key provider, so UI-test runs never
  read the real Keychain.
- AppCoordinator drops its recentDictations copy; the UI-test echo reads
  the saved history.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Copy

- ios-typecheck.sh compiled BlurtDesign without App/Shared, which
  project.yml now adds for FlowLayout, so the Mac Catalyst stand-in
  failed on CI (it only runs where no iOS runtime is installed).
- testRecentRowCopyShowsConfirmation selects the row first: Edit › Copy,
  which `app.menus` can match, copies the selected row now, and a
  right-click doesn't select.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants