diff --git a/.claude/skills/project-guardrails/SKILL.md b/.claude/skills/project-guardrails/SKILL.md index 30b524eb..2db38377 100644 --- a/.claude/skills/project-guardrails/SKILL.md +++ b/.claude/skills/project-guardrails/SKILL.md @@ -63,14 +63,13 @@ genuinely correct, and reaching for it means it's time to stop and ask. that is about the **upload**: the service starts inferring as the audio arrives, and still answers with one final transcript. No deltas, no partial results, no WebSocket — don't read the route name as permission to add them. - The one socket in the tree is the experimental read-selection-aloud feature's - AssemblyAI streaming **TTS** client (`TTS/AssemblyAISpeechSynthesizer.swift`, - the only file `check-invariants.sh` exempts); it speaks text, never - transcribes, so it is not a door back to streaming STT. + There is no socket in the tree: read-aloud's TTS client + (`TTS/AssemblyAISpeechSynthesizer.swift`) is a streaming HTTP response, and + it speaks text, never transcribes, so it is not a door back to streaming STT. - **No separate LLM cleanup pass.** Cleanup rides in the same dictation request, as `config.llm_instruction` (`CleanupInstruction`). No LLM Gateway client for dictation, no `StylerProtocol`, no post-transcription styling - stage. The one gateway client in the tree is the experimental read-aloud + stage. The one gateway client in the tree is the read-aloud feature's (`TTS/ReadAloudLLM.swift`, the only file `check-invariants.sh` exempts from this rule): the skip-jargon listening rewrite of a selection, and the spoken answer to a request about one. It never cleans up a dictation, so diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index 9edda948..29c629af 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -79,7 +79,10 @@ jobs: needs: changes if: needs.changes.outputs.code == 'true' runs-on: macos-26 - timeout-minutes: 30 + # The UI suite alone is ~17 minutes (47 tests, driving every trigger key and + # read-aloud end to end) on top of the engine suite, the leak run and the + # lint passes; 30 cut a passing run off in its last lint step. + timeout-minutes: 45 steps: - name: Checkout blurt diff --git a/AGENTS.md b/AGENTS.md index d6063e7b..fcd7ce77 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -92,8 +92,8 @@ Sources/BlurtEngine/ the engine (dependency-free Swift package) TextShortcuts (the on-device phrase expansion) STT/ AssemblyAITranscriber, Conversation/TranscriptionContext, CleanupInstruction, SyncSTTLimits - TTS/ experimental read-aloud: SelectionSpeaker, AssemblyAISpeechSynthesizer - (the one WebSocket), SpeechSegmenter, StreamingPCMPlayer + + TTS/ read-aloud: SelectionSpeaker, AssemblyAISpeechSynthesizer + (streaming HTTP), ListeningChunks, SpeechSegmenter, StreamingPCMPlayer + PCMSchedule, SelectionCopy, ReadAloudLLM (the skip-jargon rewrite and the answer to a spoken ask, the one gateway client) App/Blurt/ @@ -423,10 +423,10 @@ rule along with the row. | Add an external SPM dependency to the engine | Dependency-free by rule (biggest supply-chain risk); a `check.sh` guard fails on `.package(` in `Package.swift` or a `url:`/`github:` package in `project.yml`. Extend `BlurtEngine` instead. | | Use `AVAudioEngine` / `installTap` for capture | A long-lived engine bound its input graph to one device and went stale on a mic↔built-in switch — `-10868` (`kAudioUnitErr_FormatNotSupported`) or all-zero buffers. `MicCapture` builds a fresh `AVCaptureSession` recorder per capture (owner-directed move from `AVAudioRecorder`, 2026-08-25). | | Pre-open the mic to shave bring-up latency (a warm/prepared recorder) | Measured against `kAudioDevicePropertyDeviceIsRunningSomewhere`: neither building an `AVCaptureSession` nor the retired `AVAudioRecorder.prepareToRecord()` opens the device, so neither pre-pays the 180–600 ms route activation `record()` costs. A warm-recorder lifecycle bought ~15 ms and cost a device-identity check, a pin check, an expiry and a bring-up flag. `MicCapture.warmUp()` is stateless: it absorbs the process's first-touch cost and holds nothing. | -| Add streaming STT | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. The one WebSocket allowed is the experimental read-aloud feature's streaming **TTS**, confined to `Sources/BlurtEngine/TTS/AssemblyAISpeechSynthesizer.swift` (the invariant gate exempts only that file). | +| Add streaming STT | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. There is no WebSocket anywhere: read-aloud's **TTS** is AssemblyAI's streaming HTTP API, a response body read as it arrives, and the invariant gate exempts no file. | | Call the dictation API in a way its docs don't describe | **The reference is the source of truth.** Every path, header, multipart part, `config` key, response key and error key Blurt sends or reads is one it documents. The route has been renamed repeatedly and still answers to names the docs have dropped — `prompt`, `keyterms`, `word_boost`, `language_code`, `llm: null` — so a 200 is not evidence a field is supported, only that it has not been removed yet; treat anything absent from the docs as deprecated. Undocumented behaviour carries no compatibility promise, so a feature resting on it can stop working silently: that is why `llm: null` (the only rewrite off switch there is) and the `message` error field were dropped on 2026-09-11, along with `warmUp()`'s GET at the bare host root in favour of the documented `GET /warm`. Running tighter than a documented cap is fine — `STTPrompt.characterCap` does. Measuring is still how a claim gets settled, and the docs have been wrong twice (see `DictationWireTypes` on unknown fields, `STTPrompt` on the 6000 cap), but a measurement licenses _distrusting_ a documented field, never _sending_ an undocumented one. | | Keep a copy of the recording for anything but the over-cap fallback | `MicCapture.stop()` answers a byte count, not the audio (owner-directed, 2026-09-08): accumulating the blob in the capture lock cost a second `memcpy` of every byte and held a duplicate until release, and nothing read it back. The one copy that exists is the transcriber's (`relay`, in `AssemblyAITranscriber+LiveCapFallback.swift`), kept on the upload's task to back the documented `/v1/transcribe` fallback when the streaming route refuses a recording past its 120 s cap (`413 audio_too_large`): the 2026-09-08 rule's "no retry-from-buffer path", reversed for that one case on 2026-10-29. Don't read it back for a retry, a replay, a cache or a second request of any other kind, and don't move it into `MicCapture`. `check-invariants.sh` keeps the buffered route's path inside that one file. | -| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by `config.llm_instruction` on the same `/v1/transcribe/live` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `STTPrompt`. The one gateway client allowed is the experimental read-aloud feature's, confined to `Sources/BlurtEngine/TTS/ReadAloudLLM.swift` (the invariant gate exempts only that file): the skip-jargon listening rewrite of a selection, and the spoken answer to a request about one, neither of which the TTS socket has a server-side equivalent for. It never cleans up a dictation; the only transcript it sees is a hand-back press's request (`.handedBack`), which is never pasted. | +| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by `config.llm_instruction` on the same `/v1/transcribe/live` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `STTPrompt`. The one gateway client allowed is the read-aloud feature's, confined to `Sources/BlurtEngine/TTS/ReadAloudLLM.swift` (the invariant gate exempts only that file): the skip-jargon listening rewrite of a selection, and the spoken answer to a request about one, neither of which the TTS API has a server-side equivalent for. It never cleans up a dictation; the only transcript it sees is a hand-back press's request (`.handedBack`), which is never pasted. | | Add local models or model downloads | Transcription is a remote AssemblyAI call: no on-device ASR/LLM, no model cache, no download UI. | | Pin transcription to English, or set a language at all | Hurt non-English transcription; language is left to the model's own detection. **No `config.language_codes`** either — note the plural, which is the documented spelling, though the undocumented singular `language_code` is accepted too (a 200, not the unknown-key 400 a bogus name earns), so both are live and both are pinned absent. Detection was measured to work with neither it nor a prompt set (es/fr/de/ja clips each transcribed in their own language against the live endpoint, rewrite included), and re-measured 2026-09-11: a Spanish clip came back as Spanish with no field, with `language_codes: ["es"]`, **and** with `language_codes: ["en"]` — asking for English did not force English, so the reference's `["en"]` default does not describe this route and omission is not a bet on a default. Setting one would only take working detection away. `KeytermsWireTests` asserts the absence of both spellings. | | Send `config.prompt` alongside `config.stt_prompt` | They are the same field under two names — the docs give `stt_prompt` as the field and note it is "Also accepted as `prompt`" — and a request carrying both is rejected before the audio is read (`provide only one of stt_prompt or prompt; they are the same field`). So this is never a compatible addition: it is a 400 on _every_ dictation. `stt_prompt` is the documented spelling and the only one to send; `prompt` is the legacy one, and the route's own inner validator still answers under it (see `STTPrompt.characterCap`), which is a reason to treat it as deprecated rather than interchangeable. | @@ -471,7 +471,7 @@ framework, or notarization rejects the build; roll-forward-only for a bad releas the one exception (XCTest, because XCUIAutomation requires it). - **A store setter with no production caller gets deleted.** The exception is a value that's _encoded_ on write (`TriggerKeyStore`'s keycode, `SoundPackStore`'s id, `StyleProfileStore`'s JSON - profile list): there, give the setter a production caller instead — see `HotkeyStepView`, where assigning the raw `@AppStorage` slot + profile list): there, give the setter a production caller instead — see `KeysSettingsTab`, where assigning the raw `@AppStorage` slot directly left the setter test-only, so a change to the encoding would have kept `swift test` green while the picker silently wrote the old form. For a plain `Bool` there is nothing to encode, so the Settings toggle writing the slot through `@AppStorage` is the whole story and the store stays @@ -882,20 +882,24 @@ lands), and `keyTap.refreshBinding()` after the user picks a different trigger k ## Hotkey -The dictation trigger is a **lone modifier key** — one by default, plus any other given a style (see -Per-key styles below) — tap-to-toggle or hold-to-talk, implemented in-house. Four pieces, three of them pure engine logic: +The triggers are **lone modifier keys**, each set to dictate, read the selection aloud, ask about it, +or nothing, and each tapped, held, or either (see Keys below) — implemented in-house. Four pieces, +three of them pure engine logic: - **`TriggerKey`** (`Hotkey/TriggerKey.swift`) — enum of the curated lone momentary modifiers usable - as the trigger (right ⌘, right ⌥, `fn`), `rawValue` = the macOS virtual keycode, plus `label` - ("right ⌘") and the device-modifier masks the event source needs. Right-side modifiers are chosen - because a solo press rarely collides with app shortcuts. + as triggers (right ⌘, right ⌥, `fn`, left ⌘, left ⌥, left ⌃), `rawValue` = the macOS virtual + keycode, plus `label` ("right ⌘") and the device-modifier masks the event source needs (one bit + per physical side). Dictation defaults to right ⌘ because a solo press rarely collides with app + shortcuts; left ⌥ defaults to reading aloud (not left ⌘, which is down for every shortcut and + ⌘-click), and a read waits for the press to prove itself (below). - **`TriggerKeyStore`** — persists the chosen keycode in `UserDefaults` (`BlurtTriggerKeyCode`), defaulting to **right ⌘**. - **`DictationKeyGate`** — pure, clock-free state machine (`idle`/`armed`/`latched`) turning modifier-down/up and other-key-down into `start`/`stop`/`cancel`/`none`. Recording starts the - instant the modifier goes down; what a key-up means is the persisted **`TriggerActivation`**'s - call (`TriggerActivationStore`, default tap-or-hold; the Shortcut section's Activation picker - writes it): under the default a release ≥ `holdThreshold` (default 1 s) is a **hold** + instant the modifier goes down; what a key-up means is that key's **`TriggerActivation`** + (`TriggerKeyBindings.activation(for:)`: the key's own mode in `TriggerKeyActivationStore`, else + tap-or-hold for a key that reads aloud, else `TriggerActivationStore`'s shared mode, default + tap-or-hold): under tap-or-hold a release ≥ `holdThreshold` (default 1 s) is a **hold** (push-to-talk → stop) while a shorter release **latches** (tap-to-toggle; next tap stops); tap-only latches on every release, hold-only stops on every release. A combo (modifier + another key, e.g. ⌘C) from idle cancels the fresh capture; over a latched recording it @@ -904,7 +908,8 @@ Per-key styles below) — tap-to-toggle or hold-to-talk, implemented in-house. F - **`DictationKeyRouter`** — the event-routing layer over the gate: only the bound keycodes' flag changes drive the modifier (the first to go down owns the gate, `activeKeyCode`, until it is idle), and only genuine down/up **edges** reach the gate (`flagsChanged` deliveries re-report the bit whether or not it changed, so a repeat must not double-fire). - `reset()`/`rebind(triggerKeyCodes:activation:)` report whether they discarded a live recording the host must + A press from idle rebuilds the gate under its own key's mode, so each key keeps its own. + `reset()`/`rebind(activations:)` report whether they discarded a live recording the host must cancel upstream, and so does **`recoverFromDroppedEvents(heldFlags:)`** — the disabled-tap rule: events may have been dropped, so the gate's state survives only while the active trigger is still physically held (its key-up is still coming); otherwise that key-up was @@ -932,38 +937,72 @@ any other key restores the value it replaced. That record lives in a suite both Blurt Dev restore each other to Do Nothing — and outside `DefaultsKey` on purpose, so a settings reset doesn't erase it: the relaunch after a reset is what restores the user's setting. `scripts/reset-install.sh` restores from it directly, since it wipes the host domains. The write -only takes effect at the next log-in (macOS reads it then), so the Shortcut footer warns that `fn` +only takes effect at the next log-in (macOS reads it then), so the Keys tab's footer warns that `fn` may also open the emoji picker or Apple Dictation until then, and links to Keyboard Settings. A keycode 63 saved before `fn` was removed (#146) is migrated to right ⌘ once (`TriggerKeyStore.migrateStaleFunctionBinding()`), so restoring `fn` doesn't silently rebind it. Don't fix this with an active tap that swallows `fn`: the tap is listen-only for latency. -The trigger is editable in the Shortcut section of the setup/settings UI (`HotkeyStepView`) — a -`Picker` over `TriggerKey.allCases` that writes `TriggerKeyStore`, after which -`AppCoordinator.dictationBindingChanged()` re-reads it into the tap. For display strings, use -`TriggerKeyStore().triggerKey.label` for one-shot reads; in views that must re-render live on a -Settings change, use **`@BoundTriggerKey`** — a `DynamicProperty` in `Wizard/BoundTriggerKey.swift` wrapping -the `@AppStorage(TriggerKeyStore.defaultsKey)` + `TriggerKey.fromPersisted` pair — rather than -restating that pairing per view. The unset default belongs to `fromPersisted` (an absent keycode maps -to right ⌘), so views must not re-declare `TriggerKey.rightCommand.rawValue` themselves. - -**Per-key styles.** A Key Styles row in its own section on the Styles pane (`KeyStylesSection`, `KeyStylesRow`) — a status naming the -keys set up, and an Edit… button — opens a sheet with one pop-up per `TriggerKey`: the style a press -of that key dictates with — Same as main window, Default, or a profile (disabled while enhanced -transcripts are off) — or Off for keys other than the bound one. On Styles, under the profiles it -picks from, rather than under the key picker, so General stays short enough for a small display; -its own section rather than a row of Custom Styles, because that section is disabled while -enhanced transcripts are off and turning a key on or off still works then; one row and a sheet, -not a row per key, to keep the pane short. `TriggerKeyStyleStore` (`BlurtTriggerKeyStyles`, keycode → style id) holds -it; an absent value is today's single key with the current style. Rebinding swaps each moved key's -entry with a stash (`BlurtTriggerKeyStyleStash`): the key moved away from goes back to what it was -as an extra before it was bound (off, unless the user had set it up as one), and gets its bound -style back if it is bound again. Turning another key on or off doesn't reset the router, so a -recording latched while the sheet is open survives it. Each key is still a lone modifier -from the same set, watched by the same listen-only tap. The press command carries its key +The keys are editable on Settings' Keys tab (`KeysSettingsTab`; below); after any change +`AppCoordinator.dictationBindingChanged()` re-reads the binding into the tap. Every reader of the +keys — the tap, the read-aloud router, `PressStyle`, every view — goes through one snapshot, +**`TriggerKeyBindings`** (`TriggerKeyBindings(defaults:)` for one-shot reads), which owns the +defaults and the rules; views that must re-render live on a Settings change use +**`@ObservedKeyBindings`** (`Wizard/ObservedKeyBindings.swift`), which observes the four raw slots +through `@AppStorage` and builds the snapshot from them. The unset defaults belong to the bindings +(an absent keycode maps to right ⌘), so views must not restate them. + +**Keys.** Settings' Keys tab, after General (`KeysSettingsTab`), is a table with two pop-ups per +`TriggerKey`: its action (`TriggerKeyChoice`) — Off, dictate (with the main window's style, no custom style, or a profile; the +styles disabled while enhanced transcripts are off), Read Selection Aloud, or Ask About Selection — and whether it is +tapped, held, or either. How read-aloud sounds (voice, speed, skipping jargon — +`ReadAloudStyleStore`) is its own Settings tab, Read Aloud, before Styles (`ReadAloudSettingsTab`); +picking a voice plays a sample of it, as picking a cue sound does. Its own tab, not rows on General, +because General has to stay short enough for a small display (three per-key rows pushed the +cue-sound picker off CI's screen). +`TriggerKeyStyleStore` (`BlurtTriggerKeyStyles`, keycode → choice, the stored form a style id had +before read and ask) and `TriggerKeyActivationStore` (`BlurtTriggerKeyActivations`, keycode → mode, +the bound key's included) are the writes; both encode through `TriggerKeyMap`. An absent choice is +the key's default — the bound key dictating with the current style, left ⌥ reading aloud, the rest +off — so Off is stored explicitly where it isn't the default; an absent mode follows +`TriggerActivationStore`'s shared one, so an install from before per-key modes reads unchanged. The +bound key (`TriggerKeyStore`, the one the ready screen and menu bar name) always dictates: +`TriggerKeyStyleStore.setChoice` refuses to take the only dictating key off dictation, and taking +the bound key off it moves the binding to the next dictating key first. A key's entries mean the +same in either role, so moving the binding moves nothing else. Changing another key doesn't reset +the router, so a recording latched while Settings is open survives it. The press command carries its key (`DictationSession.Command.press(key:)`), and the session resolves that key's style once, on the press's own turn, into a `PressStyle` value: its instructions ride the request's context, its name the Recent row. It never writes `BlurtActiveStyleProfile`, a later press can't restyle a dictation under way, and a mapped style since deleted falls back to the selection. +**Read aloud and ask** live in the engine: `SelectionSpeechRouting` (pure) decides, and +`SelectionSpeechRouter` (main-actor) carries it out — the selection read, the hold timer, the speech +task — reporting each read's start, end, loudness and failure to the app, which owns only the pill, +the cues and the beep. A dictating key's commands pass straight to the session, so dictation +never waits on an Accessibility read. A read or ask press reads the selection, and nothing starts at +the key-down — a modifier going down over a selection is as often the start of ⌘C — until the press +proves itself: a tap (the gate's `.latch`), or the key still down `holdDelay` (350 ms) later in a +mode that holds. A combo before then does nothing. A read then plays until the key's next stop; an +ask records like a dictation (`pressHandingBack`) and the transcript handed back is answered aloud. +With nothing selected, the press does nothing. A read or answer wears the dictation pill +(`OverlayUIState.speaking`: the orb and meter, the meter following the voice — `StreamingPCMPlayer` +keeps each chunk's 50 ms loudness windows on the media timeline, `PCMLevel`, and the meter reads the +one playing) between its own start and stop cues (read-aloud's `SoundPackStore` slot). While a read +plays, only a new capture takes the pill (`PipelinePhase.overlayState(whileSpeaking:)`); when it ends, +the pill goes back to a steady state still under way and never replays a dictation's old notice +(`overlayStateAfterSpeech`). Each cue pair reloads after an output-route change on its own schedule +(`CueReprimeSchedule`): dictation's at the next press or terminal phase, read-aloud's at once unless a +capture is running. `SelectionSpeechRouter` takes any `ReadAloudSpeaker` (`SelectionSpeaker` in +production), which is how the UI tests run read and ask offline; a stopped read's cancelled request +reads as a cancellation, never a failure. The TTS is AssemblyAI's documented Streaming TTS API +(`AssemblyAISpeechSynthesizer`: `POST https://tts.us.assemblyai.com/v1/speech`, `voice_id`, the +audio read as the response streams). A selection is read in pieces (`ListeningChunks`: a short first +piece, so reading starts after a paragraph, then about 1,500 characters each, at paragraph or +sentence ends), one request each, rewritten first when skip-jargon is on; the next piece is asked +for only once less than 30 s of audio is queued (`SelectionSpeaker.bufferTarget`), so a long +article (up to 100,000 characters) starts quickly and is synthesized only as far as it is heard; +the voices offered are the English ones on the docs' Voices page (`ReadAloudVoice.catalog`), since a +read sends no `language` and a catalog voice would otherwise speak its own. + ## Contextual prompt and key terms `Sources/BlurtEngine/STT/STTPrompt.swift` builds the string passed as the dictation @@ -1237,13 +1276,16 @@ loop re-reads the status each pass. `signOut()` ends the connection server-side only then forgets it here; a revoke the server couldn't confirm (503) leaves everything in place and returns an inline report so the user can retry (`SignInFailure.failureReport` makes the same inline-vs-alert split as `APIKeySubmission`). `CredentialRows.resolve` decides which of the account -and key rows show and which button is filled. `InstallReset` clears the session item with the key. +and key rows show and which button is filled. Pasting a key is the legacy path: where sign-in is +configured, a new install sees only Sign In, and the key row shows only for an install that already +holds a pasted key (to change or remove it) or a build without sign-in (the UI-test harness). `InstallReset` clears the session item with the key. The wire half (`OAuthAuthorization`, `OAuthClient`) is in the `AssemblyAI` library. The build turns sign-in on with four Info.plist values (`BLURT_OAUTH_*`, read by `OAuthConfiguration.blurt`): -release's are production's, committed in `App/Blurt/Config/SignIn.release.xcconfig`; the dev -configurations' come from the gitignored `SignIn.local.xcconfig` via `SignIn.xcconfig`, since they -name sandbox deployments. Any one empty hides sign-in, which is how a dev build without the local -file runs, and a session inherited by such a build can still be signed out (locally). A signed-in key also +production's, committed in `App/Blurt/Config/SignIn.release.xcconfig`, for every configuration — +dev builds too, through `SignIn.xcconfig`, which then includes the gitignored +`SignIn.local.xcconfig`: a developer's override pointing a dev build at a sandbox deployment, whose +values aren't committed. Any one empty hides sign-in, and a session inherited by a build without +it can still be signed out (locally). A signed-in key also dictates against the host its sign-in configures (`BLURT_OAUTH_DICTATION_ORIGIN`, optional; a sandbox key only works against its sandbox's host): the account keeps a `DictationOrigin` current as it signs in and out, and the app's `baseURL` closure for `AssemblyAITranscriber` reads it at every request and diff --git a/App/Blurt/Blurt.xcodeproj/project.pbxproj b/App/Blurt/Blurt.xcodeproj/project.pbxproj index af4be2b6..828937ac 100644 --- a/App/Blurt/Blurt.xcodeproj/project.pbxproj +++ b/App/Blurt/Blurt.xcodeproj/project.pbxproj @@ -35,7 +35,6 @@ 0F6D746DF48BB048827A358F /* rom1b-29-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 786672855B44AB80CAF4D782 /* rom1b-29-stop.m4a */; }; 1088FECB82D84AD0951CA1E2 /* juno-83-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 920F5A55960A00E2AECF1362 /* juno-83-stop.m4a */; }; 10EA68562CE70506895CB2FE /* rom1b-31-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 1D5C52A673686FB041595227 /* rom1b-31-stop.m4a */; }; - 121DC6CD82DEAC7CC0F5AC24 /* BoundTriggerKey.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8999B37D9418B7A6923BD358 /* BoundTriggerKey.swift */; }; 1275CFCDBF1D593B545C8A84 /* rom1b-15-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 6D89280137DAC07403FB60FC /* rom1b-15-start.m4a */; }; 15210F6BE968D3E2EF771DE0 /* rom1a-5-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 2CA6F4E6A10AE93F169DF19B /* rom1a-5-stop.m4a */; }; 152452FE264E453808327885 /* juno-122-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 9FE15281EA6DE5C2C1978C44 /* juno-122-stop.m4a */; }; @@ -60,6 +59,7 @@ 1BC2EB8B211378DD842647EC /* rom1a-6-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = BBF8A81BBD03B233A8C2B845 /* rom1a-6-stop.m4a */; }; 1BEC33EC0C95E9A52CC14F2B /* juno-65-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = E5F9E5E96FA7EC31380F6669 /* juno-65-stop.m4a */; }; 1C40B5DB92262E05808ECCEA /* DictationKeyTap.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3258A84E37C372993B1572EE /* DictationKeyTap.swift */; }; + 1C89CB94C7D10B67BD0573DF /* KeysSettingsTab.swift in Sources */ = {isa = PBXBuildFile; fileRef = AE6928CEB0A1A92241762DB6 /* KeysSettingsTab.swift */; }; 1C92A0295860131BCF95AF1C /* rom1b-3-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 5B5B261F4D195B5E6E3B5567 /* rom1b-3-stop.m4a */; }; 1CCDD063B29F52F4681EA371 /* rom1b-6-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 3996A11A62CD49D6CB89CB8E /* rom1b-6-start.m4a */; }; 1D1100AB3D47EFC80D47B04B /* SettingsWindowRoot.swift in Sources */ = {isa = PBXBuildFile; fileRef = 37CBB4580F251B182A8B1937 /* SettingsWindowRoot.swift */; }; @@ -72,9 +72,10 @@ 20EF88890F9D05CC4381F626 /* juno-15-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 71BEAB59A7A2280EB1EDC80E /* juno-15-stop.m4a */; }; 20FCCD07969B601231884018 /* juno-67-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 06A8F7161C5917AADFF6AC77 /* juno-67-stop.m4a */; }; 2103C3F415E9C102664CA13C /* rom1a-20-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = F7307E06A286147462426313 /* rom1a-20-stop.m4a */; }; + 22526AFA400A8E77982C3B3A /* KeyActionName.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3879273E32399C14F3533899 /* KeyActionName.swift */; }; 234AFC3335EE59A5CF61E79C /* juno-78-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 2A97148549AA99C8CDD7C316 /* juno-78-start.m4a */; }; + 24607B2B47E779840B7C225C /* ReadAloudSettingsTab.swift in Sources */ = {isa = PBXBuildFile; fileRef = 754D13A631535995AF41AA5B /* ReadAloudSettingsTab.swift */; }; 24CD4A998CA258D89140D418 /* juno-12-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = A0C78659208520BB93EAB372 /* juno-12-start.m4a */; }; - 2638A7B2FF8239CF3D861F0C /* KeyStylesRow.swift in Sources */ = {isa = PBXBuildFile; fileRef = 00D3C74B2EFEC7A63A7E27A8 /* KeyStylesRow.swift */; }; 2747AA6669B2406D4436E2A3 /* juno-53-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 56F69B42F0D495142B17FF83 /* juno-53-start.m4a */; }; 27CD78DC27ACE6A0E2EDFD15 /* juno-41-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 8E71DBB3C21D616DDFA77D0B /* juno-41-stop.m4a */; }; 282E843FAECCBCA0CDB04E48 /* juno-93-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = D85C603DADD81B58F7D390F7 /* juno-93-stop.m4a */; }; @@ -122,6 +123,7 @@ 3BEA1249CB0D5B8583C85C33 /* juno-6-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 49F7AD1193F18CCDF17B9EBA /* juno-6-stop.m4a */; }; 3C1C9F5BE822A6D177369042 /* rom1b-10-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 77C74C74945D0A990578D911 /* rom1b-10-start.m4a */; }; 3CEAFC28F861460AFC573257 /* juno-74-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = D3681457B0FB13FF3FD6BAF6 /* juno-74-stop.m4a */; }; + 3E5C59CDCE131B9C602814AC /* ReadAloudFlowsUITests.swift in Sources */ = {isa = PBXBuildFile; fileRef = B9A7E9E1922395ABBA81D943 /* ReadAloudFlowsUITests.swift */; }; 3F85F8F73725C2DE7C315D34 /* rom1a-3-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = F2D66E4734A5DC086904E421 /* rom1a-3-stop.m4a */; }; 3F954DB8B4BF0BFB867F6467 /* rom1a-0-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 555254497CA693DF00B3B924 /* rom1a-0-stop.m4a */; }; 3FB9E592DE0C331A7F7A5891 /* rom1a-3-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = F96D128F79D585621827FF6A /* rom1a-3-start.m4a */; }; @@ -136,7 +138,6 @@ 44A8D4A3DD256E9D02A79C27 /* rom1a-15-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 3BA1B8BBEEC6B5A2960688C3 /* rom1a-15-start.m4a */; }; 44ECD071FA8B324584F31F23 /* rom1b-28-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 562C5F709DE00A5F38F045A5 /* rom1b-28-start.m4a */; }; 454ADE9A9137F4E0A0AF4C24 /* juno-2-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 40A711B3442D9CBE0189BEF3 /* juno-2-stop.m4a */; }; - 4563B6FCC9AA36F95CFD9D88 /* SelectionSpeechRouter.swift in Sources */ = {isa = PBXBuildFile; fileRef = C10286F2D7AD2A279FBBEBFC /* SelectionSpeechRouter.swift */; }; 45BB291D75223AC958B137BF /* juno-36-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = E6A271DA085CFB136745066C /* juno-36-stop.m4a */; }; 468547FA48D3745D73230860 /* rom1b-26-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 6A9A4B58A6524D4F954DFE65 /* rom1b-26-stop.m4a */; }; 46D3EBAEB518E57ADDC45A40 /* juno-63-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 0DBB595A2AB7221F408B4775 /* juno-63-start.m4a */; }; @@ -148,6 +149,7 @@ 4B3BA1626DE3D22AA0CB3E45 /* juno-31-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 2C7B119035D271E56434FB51 /* juno-31-start.m4a */; }; 4C2F89013028E3D7829D0CBD /* juno-100-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 1C72B7BEF6CFEC52928C6CED /* juno-100-stop.m4a */; }; 4D33337B6794A265D46FCE6C /* juno-5-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 4DB6065E9D1F64F3AF0FD03B /* juno-5-start.m4a */; }; + 4D344136EED62ED843F3BE76 /* KeysAndReadAloudUITests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3041943C2A14446C97466CA4 /* KeysAndReadAloudUITests.swift */; }; 4D94F20832B9828A5642B1B8 /* juno-91-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 0C17FEE505740AA5BCBB0E21 /* juno-91-stop.m4a */; }; 4E8F1A9F7F78845ADF7A3F01 /* rom1a-22-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = F1340BFA7AF0D9D50EAA4A24 /* rom1a-22-stop.m4a */; }; 4EE19BF49EB60AB0B48B0575 /* juno-8-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 1CF1F098A8DC41DD8508480E /* juno-8-start.m4a */; }; @@ -156,12 +158,14 @@ 500A15D5374847D847555931 /* juno-55-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 2978FCBEE1BC015D9CFC8646 /* juno-55-stop.m4a */; }; 5025717A1723EEEFF385533B /* juno-12-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 2D51B930706AF22BF9BA0CEF /* juno-12-stop.m4a */; }; 5120B387A51EF21F65CEA342 /* juno-51-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = FFCE00A121F0A2EA669DE42B /* juno-51-stop.m4a */; }; + 526C973DCC1F588F91DEF1E5 /* SoundPackCatalog+ReadAloud.swift in Sources */ = {isa = PBXBuildFile; fileRef = A5A03BCF719B616FFF41455C /* SoundPackCatalog+ReadAloud.swift */; }; 53CD236A3C9985EF336637DA /* juno-115-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 35048A9866E106410CDCED81 /* juno-115-stop.m4a */; }; 54887F064F7C9D737A503EB3 /* juno-44-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = F5A287AB11C9314F4AF37A61 /* juno-44-start.m4a */; }; 550697861DEEF702CBBFA8FA /* juno-94-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 4FB6BF9F449D954A52952C6B /* juno-94-stop.m4a */; }; 550F30201148FAC5EACA2315 /* juno-70-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 92753DF24C8BB9D740CC7B6C /* juno-70-stop.m4a */; }; 55A02BEEBE538AE0048CC2B9 /* juno-73-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = F4512C06E3020F0F2D518A3E /* juno-73-stop.m4a */; }; 572211105316DB06C2AC9134 /* juno-30-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 41736827B0D4E7E1D024DCF1 /* juno-30-start.m4a */; }; + 583D87E4396443D2DD0E4ED9 /* KeyActionName.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3879273E32399C14F3533899 /* KeyActionName.swift */; }; 59C168C1D913BE352C83FFD5 /* juno-45-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = D733CCA5061A2B7B74A23A65 /* juno-45-stop.m4a */; }; 5B0EDCD855E9676BCE76E1A2 /* rom1a-7-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 4873421ACF290B88FA257D6A /* rom1a-7-start.m4a */; }; 5B117EB54F223A9960EC23B6 /* juno-88-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 19A708C2C2C7CDF52E47A8C4 /* juno-88-start.m4a */; }; @@ -310,6 +314,7 @@ ADE2784E2C702F3A126A2DF6 /* rom1b-30-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 8D369A708E176C12114384AD /* rom1b-30-start.m4a */; }; ADF5353D6E27FDBB61557DFE /* juno-99-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = B432346450961555004A2BDE /* juno-99-stop.m4a */; }; AE2ED5CDCD04D04FD69DE501 /* rom1a-28-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 1610DB028EBE814A3A163301 /* rom1a-28-start.m4a */; }; + AFA3982826E71CE491DD82E1 /* ObservedKeyBindings.swift in Sources */ = {isa = PBXBuildFile; fileRef = EB55696AF62110C04E63B601 /* ObservedKeyBindings.swift */; }; B13A61E6F639E2543D4B6804 /* juno-51-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 36CF30F1384D606BE74362C8 /* juno-51-start.m4a */; }; B13F272AB1FF6533B5636595 /* juno-40-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = CFA57E6B7B3A81E2531990B5 /* juno-40-start.m4a */; }; B2DD0930AA5C7250B6FB127C /* juno-117-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = CB487BE3C821FF18E414CF4B /* juno-117-stop.m4a */; }; @@ -321,6 +326,7 @@ B5ADC67B78CA3EA70D6BAEB3 /* RecentDictationsSection.swift in Sources */ = {isa = PBXBuildFile; fileRef = A0EC4E3D73862F8E4DB6F47D /* RecentDictationsSection.swift */; }; B6ACE56CCFCAE22FC67833AF /* juno-76-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 94546743017BF75E0DC24ACC /* juno-76-start.m4a */; }; B6B9598209C21737FC80D9FC /* MenuBarUITests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 63869ED0BF971F732668349B /* MenuBarUITests.swift */; }; + B79801F65857FFF5829AE13A /* HarnessDriving.swift in Sources */ = {isa = PBXBuildFile; fileRef = C23288EE0E6273C0D7603791 /* HarnessDriving.swift */; }; B844779F8427A1225C45A9FB /* WizardController.swift in Sources */ = {isa = PBXBuildFile; fileRef = 30E57A9C9B934A2120381005 /* WizardController.swift */; }; BA31A2D639A9EBEAFA22DF5D /* juno-63-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = E97DEAE3F8FD9ADD7DB41B02 /* juno-63-stop.m4a */; }; BAC7996520BFCAA7BE0F929F /* juno-125-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 72144FD6362B731D322BA316 /* juno-125-stop.m4a */; }; @@ -388,6 +394,7 @@ D8156DB52C2BDCE9875E7462 /* rom1b-16-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = B24D0EACA5DCC94CFCAD9753 /* rom1b-16-start.m4a */; }; D88B8E1DE2F71D0A3BD39516 /* juno-83-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 724B039E044BE4B2E3EAA3C5 /* juno-83-start.m4a */; }; D8B18E0C6E3D62D8A0031226 /* juno-71-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = D1D56A6899994805C9F0DBBD /* juno-71-stop.m4a */; }; + D928EB58EC2072A46170CC9D /* KeysLiveUpdateUITests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 898FC470C399A329D901DE30 /* KeysLiveUpdateUITests.swift */; }; DAC7039274C55389324A07B2 /* juno-82-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = E84C190EF80F6B36F8241BFC /* juno-82-stop.m4a */; }; DAD3C772A4DCE75359126F27 /* juno-37-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 1EA8B0FCDE5A026C5E8B5EFA /* juno-37-start.m4a */; }; DB035E176FF9589F745696B9 /* juno-60-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 2B964C67C7B0BA077F649FDA /* juno-60-stop.m4a */; }; @@ -436,12 +443,12 @@ F30578AA785CAE98E944F6B7 /* TextShortcutsSection.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3A50C85936C100F9E51CC2AD /* TextShortcutsSection.swift */; }; F3D87A72BED81C8532FCB503 /* juno-3-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = A2C5021E8F72182F331DA397 /* juno-3-start.m4a */; }; F3E79C6326A2CBF8CE0CF091 /* rom1a-31-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 1E6C29571118FA5B039C4FC1 /* rom1a-31-stop.m4a */; }; - F42E6D4E62345772C5A49A08 /* HotkeyStepView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 50FFCDBD1FB2CF0A81A68D18 /* HotkeyStepView.swift */; }; F4FC8BE4A0593197C1375EC1 /* juno-43-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 47F6D91FC89A0214F121991C /* juno-43-stop.m4a */; }; F515E1D5C8254ACE1BDADD63 /* juno-64-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 6A56F68E7D3228E23904A005 /* juno-64-start.m4a */; }; F5D3911E0D8207F1967C6B3A /* juno-107-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 35059AC7D847666F9D7FFF07 /* juno-107-start.m4a */; }; F648F912EEC5A44F5930AE37 /* juno-61-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 9C3B044CD20456A76EBB99FB /* juno-61-stop.m4a */; }; F681156BFD979F56CB5B407A /* juno-103-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 6AFFA1AFC4F88A35BA39CECF /* juno-103-stop.m4a */; }; + F74265E4664F68358FABC862 /* ReadAloudSettingsUITests.swift in Sources */ = {isa = PBXBuildFile; fileRef = A82DEA00FB07C1630A62871C /* ReadAloudSettingsUITests.swift */; }; F7BC4D95BD8D5BBE31C6F911 /* juno-21-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 51193F08C115C6AB13BECC07 /* juno-21-stop.m4a */; }; F86C329D9E43393542252891 /* juno-39-start.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 3B0E95E0D720A1D8EC05EA6F /* juno-39-start.m4a */; }; F884F1BBD3F59D7BB9EBA0FA /* juno-89-stop.m4a in Resources */ = {isa = PBXBuildFile; fileRef = 2D96219CD4991D4D13B56CC9 /* juno-89-stop.m4a */; }; @@ -467,7 +474,6 @@ 002FB9F5FD28A66C04993007 /* juno-61-start.m4a */ = {isa = PBXFileReference; path = "juno-61-start.m4a"; sourceTree = ""; }; 00768AE8171902B439F0602F /* juno-89-start.m4a */ = {isa = PBXFileReference; path = "juno-89-start.m4a"; sourceTree = ""; }; 0088DE3FD3980CA727EB02FE /* juno-111-start.m4a */ = {isa = PBXFileReference; path = "juno-111-start.m4a"; sourceTree = ""; }; - 00D3C74B2EFEC7A63A7E27A8 /* KeyStylesRow.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = KeyStylesRow.swift; sourceTree = ""; }; 0150E727307295CE46981426 /* rom1b-19-stop.m4a */ = {isa = PBXFileReference; path = "rom1b-19-stop.m4a"; sourceTree = ""; }; 01B663528C4E1AFEF24675E1 /* rom1b-21-start.m4a */ = {isa = PBXFileReference; path = "rom1b-21-start.m4a"; sourceTree = ""; }; 01F10B3261EA8F8616B8595A /* MainWindowRoot.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MainWindowRoot.swift; sourceTree = ""; }; @@ -554,6 +560,7 @@ 2FFA326076660C91492A6CE1 /* juno-8-stop.m4a */ = {isa = PBXFileReference; path = "juno-8-stop.m4a"; sourceTree = ""; }; 30117E33FF99C309A29FB903 /* AppIcon.icon */ = {isa = PBXFileReference; lastKnownFileType = wrapper.icon; path = AppIcon.icon; sourceTree = ""; }; 3012F22300681F3DD71AD51B /* juno-34-stop.m4a */ = {isa = PBXFileReference; path = "juno-34-stop.m4a"; sourceTree = ""; }; + 3041943C2A14446C97466CA4 /* KeysAndReadAloudUITests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = KeysAndReadAloudUITests.swift; sourceTree = ""; }; 30703F589CF213720C7E3B15 /* StyleRow.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = StyleRow.swift; sourceTree = ""; }; 30E57A9C9B934A2120381005 /* WizardController.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = WizardController.swift; sourceTree = ""; }; 31986D541F06DA91827C18A0 /* juno-58-start.m4a */ = {isa = PBXFileReference; path = "juno-58-start.m4a"; sourceTree = ""; }; @@ -572,6 +579,7 @@ 3751D3C923F3614E866B947C /* juno-106-start.m4a */ = {isa = PBXFileReference; path = "juno-106-start.m4a"; sourceTree = ""; }; 37CBB4580F251B182A8B1937 /* SettingsWindowRoot.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SettingsWindowRoot.swift; sourceTree = ""; }; 383E03AB60BA21AAD92655D5 /* ReadyView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReadyView.swift; sourceTree = ""; }; + 3879273E32399C14F3533899 /* KeyActionName.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = KeyActionName.swift; sourceTree = ""; }; 38CAFC7A39E679F1CA3A9692 /* blurt */ = {isa = PBXFileReference; lastKnownFileType = folder; name = blurt; path = ../..; sourceTree = SOURCE_ROOT; }; 3996A11A62CD49D6CB89CB8E /* rom1b-6-start.m4a */ = {isa = PBXFileReference; path = "rom1b-6-start.m4a"; sourceTree = ""; }; 3A0683C1BE5AD05E4C724956 /* BlurtUITestSupport.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = BlurtUITestSupport.swift; sourceTree = ""; }; @@ -627,7 +635,6 @@ 4FEFFC17F6198DB4F43A71F1 /* rom1b-27-stop.m4a */ = {isa = PBXFileReference; path = "rom1b-27-stop.m4a"; sourceTree = ""; }; 5041BF8B32BEB59D5539E48A /* juno-16-stop.m4a */ = {isa = PBXFileReference; path = "juno-16-stop.m4a"; sourceTree = ""; }; 5070E437DB5D188D16BBAACB /* juno-30-stop.m4a */ = {isa = PBXFileReference; path = "juno-30-stop.m4a"; sourceTree = ""; }; - 50FFCDBD1FB2CF0A81A68D18 /* HotkeyStepView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HotkeyStepView.swift; sourceTree = ""; }; 51193F08C115C6AB13BECC07 /* juno-21-stop.m4a */ = {isa = PBXFileReference; path = "juno-21-stop.m4a"; sourceTree = ""; }; 541EE5FFF00A40A486501EDF /* juno-72-start.m4a */ = {isa = PBXFileReference; path = "juno-72-start.m4a"; sourceTree = ""; }; 54244D8B2EFFCBDC803F49E2 /* juno-1-stop.m4a */ = {isa = PBXFileReference; path = "juno-1-stop.m4a"; sourceTree = ""; }; @@ -688,6 +695,7 @@ 72144FD6362B731D322BA316 /* juno-125-stop.m4a */ = {isa = PBXFileReference; path = "juno-125-stop.m4a"; sourceTree = ""; }; 724B039E044BE4B2E3EAA3C5 /* juno-83-start.m4a */ = {isa = PBXFileReference; path = "juno-83-start.m4a"; sourceTree = ""; }; 731519E33F2E9EAB57F6ABF6 /* juno-20-start.m4a */ = {isa = PBXFileReference; path = "juno-20-start.m4a"; sourceTree = ""; }; + 754D13A631535995AF41AA5B /* ReadAloudSettingsTab.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReadAloudSettingsTab.swift; sourceTree = ""; }; 763E3951E68EBB37F89C7F84 /* juno-97-start.m4a */ = {isa = PBXFileReference; path = "juno-97-start.m4a"; sourceTree = ""; }; 767217AE992554FDF6D67A4D /* rom1a-10-stop.m4a */ = {isa = PBXFileReference; path = "rom1a-10-stop.m4a"; sourceTree = ""; }; 77C74C74945D0A990578D911 /* rom1b-10-start.m4a */ = {isa = PBXFileReference; path = "rom1b-10-start.m4a"; sourceTree = ""; }; @@ -720,7 +728,7 @@ 886848A89BE3512619263E54 /* juno-94-start.m4a */ = {isa = PBXFileReference; path = "juno-94-start.m4a"; sourceTree = ""; }; 8887DCE124B065F8313AC59F /* juno-9-start.m4a */ = {isa = PBXFileReference; path = "juno-9-start.m4a"; sourceTree = ""; }; 88CC243509230D21563FDF89 /* juno-127-start.m4a */ = {isa = PBXFileReference; path = "juno-127-start.m4a"; sourceTree = ""; }; - 8999B37D9418B7A6923BD358 /* BoundTriggerKey.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = BoundTriggerKey.swift; sourceTree = ""; }; + 898FC470C399A329D901DE30 /* KeysLiveUpdateUITests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = KeysLiveUpdateUITests.swift; sourceTree = ""; }; 899D9AD955B936F6BFC4E293 /* juno-52-start.m4a */ = {isa = PBXFileReference; path = "juno-52-start.m4a"; sourceTree = ""; }; 89F61F7352130DAAA530D836 /* juno-107-stop.m4a */ = {isa = PBXFileReference; path = "juno-107-stop.m4a"; sourceTree = ""; }; 8A319EF914358A938C87389D /* juno-28-stop.m4a */ = {isa = PBXFileReference; path = "juno-28-stop.m4a"; sourceTree = ""; }; @@ -764,9 +772,11 @@ A2C5021E8F72182F331DA397 /* juno-3-start.m4a */ = {isa = PBXFileReference; path = "juno-3-start.m4a"; sourceTree = ""; }; A4AFAD52CAC3825D7A603DC5 /* juno-100-start.m4a */ = {isa = PBXFileReference; path = "juno-100-start.m4a"; sourceTree = ""; }; A58B6E01B13C27D1DA52EAFE /* juno-126-start.m4a */ = {isa = PBXFileReference; path = "juno-126-start.m4a"; sourceTree = ""; }; + A5A03BCF719B616FFF41455C /* SoundPackCatalog+ReadAloud.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = "SoundPackCatalog+ReadAloud.swift"; sourceTree = ""; }; A640F1C2A7EBAF3B0F8FFB83 /* juno-120-start.m4a */ = {isa = PBXFileReference; path = "juno-120-start.m4a"; sourceTree = ""; }; A72EE584A3704166E932D950 /* juno-87-start.m4a */ = {isa = PBXFileReference; path = "juno-87-start.m4a"; sourceTree = ""; }; A82A191B20F222EBC479143E /* rom1a-4-start.m4a */ = {isa = PBXFileReference; path = "rom1a-4-start.m4a"; sourceTree = ""; }; + A82DEA00FB07C1630A62871C /* ReadAloudSettingsUITests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReadAloudSettingsUITests.swift; sourceTree = ""; }; A8AB9F55940B496EF7A8B633 /* juno-91-start.m4a */ = {isa = PBXFileReference; path = "juno-91-start.m4a"; sourceTree = ""; }; A8FCDA16949B9CE1323E24F8 /* rom1a-27-start.m4a */ = {isa = PBXFileReference; path = "rom1a-27-start.m4a"; sourceTree = ""; }; A95FFA1F1C07E8FAF8FEC0B8 /* juno-36-start.m4a */ = {isa = PBXFileReference; path = "juno-36-start.m4a"; sourceTree = ""; }; @@ -779,6 +789,7 @@ ACFD76434CCFED8E1B6FB977 /* juno-99-start.m4a */ = {isa = PBXFileReference; path = "juno-99-start.m4a"; sourceTree = ""; }; AD303A24D4133CE3F1F1DE23 /* juno-22-start.m4a */ = {isa = PBXFileReference; path = "juno-22-start.m4a"; sourceTree = ""; }; AD5F1BDEB1DB3658DDB96247 /* juno-70-start.m4a */ = {isa = PBXFileReference; path = "juno-70-start.m4a"; sourceTree = ""; }; + AE6928CEB0A1A92241762DB6 /* KeysSettingsTab.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = KeysSettingsTab.swift; sourceTree = ""; }; AFCA36ECEFBA3D7C7B1AB1B7 /* rom1a-31-start.m4a */ = {isa = PBXFileReference; path = "rom1a-31-start.m4a"; sourceTree = ""; }; B0CB1B169A47290B83AD44C0 /* rom1b-20-stop.m4a */ = {isa = PBXFileReference; path = "rom1b-20-stop.m4a"; sourceTree = ""; }; B1CBD49CFC9AD1BF802F4625 /* UpdaterModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = UpdaterModel.swift; sourceTree = ""; }; @@ -792,15 +803,16 @@ B67F392ADB8F0C2D4BB6827F /* juno-114-start.m4a */ = {isa = PBXFileReference; path = "juno-114-start.m4a"; sourceTree = ""; }; B67F817AD357801080D26FFF /* rom1b-2-stop.m4a */ = {isa = PBXFileReference; path = "rom1b-2-stop.m4a"; sourceTree = ""; }; B75ED921A405BB7A0B0564A1 /* juno-11-stop.m4a */ = {isa = PBXFileReference; path = "juno-11-stop.m4a"; sourceTree = ""; }; + B9A7E9E1922395ABBA81D943 /* ReadAloudFlowsUITests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ReadAloudFlowsUITests.swift; sourceTree = ""; }; BBA9C9322D0AF3D9C529F06F /* MenuBarScene.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MenuBarScene.swift; sourceTree = ""; }; BBF8A81BBD03B233A8C2B845 /* rom1a-6-stop.m4a */ = {isa = PBXFileReference; path = "rom1a-6-stop.m4a"; sourceTree = ""; }; BC17D9B01FDBC9667A7CD78A /* rom1a-22-start.m4a */ = {isa = PBXFileReference; path = "rom1a-22-start.m4a"; sourceTree = ""; }; BCE9C2C9715E2C1C96683AE0 /* juno-40-stop.m4a */ = {isa = PBXFileReference; path = "juno-40-stop.m4a"; sourceTree = ""; }; BD38CC94DC42A35A4174316A /* rom1b-1-start.m4a */ = {isa = PBXFileReference; path = "rom1b-1-start.m4a"; sourceTree = ""; }; BEEF91CD94662F4344ECFDE9 /* rom1b-27-start.m4a */ = {isa = PBXFileReference; path = "rom1b-27-start.m4a"; sourceTree = ""; }; - C10286F2D7AD2A279FBBEBFC /* SelectionSpeechRouter.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SelectionSpeechRouter.swift; sourceTree = ""; }; C19DBE4661B752927D16B261 /* BlurtBrand.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = BlurtBrand.swift; sourceTree = ""; }; C1BD23E2476F46A7BD75BAB9 /* rom1b-31-start.m4a */ = {isa = PBXFileReference; path = "rom1b-31-start.m4a"; sourceTree = ""; }; + C23288EE0E6273C0D7603791 /* HarnessDriving.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HarnessDriving.swift; sourceTree = ""; }; C2CB5D2E464BFD7F8BABE760 /* rom1b-7-stop.m4a */ = {isa = PBXFileReference; path = "rom1b-7-stop.m4a"; sourceTree = ""; }; C38C73970E1D1978494F7BAE /* juno-10-start.m4a */ = {isa = PBXFileReference; path = "juno-10-start.m4a"; sourceTree = ""; }; C4B6AC536FED090B4D935619 /* LiquidGlass.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = LiquidGlass.swift; sourceTree = ""; }; @@ -879,6 +891,7 @@ EAC096B44D037E7BB119609A /* AccountSignIn.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AccountSignIn.swift; sourceTree = ""; }; EB08EC40A267C71B185EC782 /* juno-109-stop.m4a */ = {isa = PBXFileReference; path = "juno-109-stop.m4a"; sourceTree = ""; }; EB504AAB7330EE4D4C1E4948 /* rom1a-26-start.m4a */ = {isa = PBXFileReference; path = "rom1a-26-start.m4a"; sourceTree = ""; }; + EB55696AF62110C04E63B601 /* ObservedKeyBindings.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ObservedKeyBindings.swift; sourceTree = ""; }; EBB8641EF9B0108DC0B103B7 /* juno-120-stop.m4a */ = {isa = PBXFileReference; path = "juno-120-stop.m4a"; sourceTree = ""; }; EC0CF2AE01C6EE41101CBB74 /* juno-17-stop.m4a */ = {isa = PBXFileReference; path = "juno-17-stop.m4a"; sourceTree = ""; }; EC6584FE7C94EF604CA33BE3 /* rom1a-19-stop.m4a */ = {isa = PBXFileReference; path = "rom1a-19-stop.m4a"; sourceTree = ""; }; @@ -949,6 +962,7 @@ 08A7417835D8538AC4A7E2FB /* DictationComposition.swift */, 0566F4FC45C72A6DA586126E /* Info.plist */, 78C5D6E7BD14D68FEB9BA215 /* SoundPackCatalog.swift */, + A5A03BCF719B616FFF41455C /* SoundPackCatalog+ReadAloud.swift */, 6072A6B0AC8219B5484326B3 /* UITestSupport.swift */, 7D6DE85EA50E7CD4CC977211 /* Branding */, DF92BBDB4B9C15FAE100B783 /* Hotkey */, @@ -1380,8 +1394,13 @@ D88509B0396487E159AA6FDD /* AccessibilityAuditUITests.swift */, 3A0683C1BE5AD05E4C724956 /* BlurtUITestSupport.swift */, 2C0A58F1CA999E6DF7742A87 /* DictationPipelineUITests.swift */, + C23288EE0E6273C0D7603791 /* HarnessDriving.swift */, + 3041943C2A14446C97466CA4 /* KeysAndReadAloudUITests.swift */, + 898FC470C399A329D901DE30 /* KeysLiveUpdateUITests.swift */, 850AFD4ECA01A669BDD91959 /* LaunchPerformanceUITests.swift */, 63869ED0BF971F732668349B /* MenuBarUITests.swift */, + B9A7E9E1922395ABBA81D943 /* ReadAloudFlowsUITests.swift */, + A82DEA00FB07C1630A62871C /* ReadAloudSettingsUITests.swift */, 44DD60B425DF100431946DAB /* README.md */, 655BB30FDBC7C45D0AF00F91 /* ReadyViewUITests.swift */, C6EFB0086ED63DB182AFFC8F /* SettingsUITests.swift */, @@ -1394,11 +1413,12 @@ isa = PBXGroup; children = ( D643CB1D491E00A2182907F8 /* BlurtCommands.swift */, - 8999B37D9418B7A6923BD358 /* BoundTriggerKey.swift */, C7C95B4C80443E2A6EFE05E7 /* DeveloperSection.swift */, 48C4CFC6271703797187A269 /* KeyCap.swift */, - 00D3C74B2EFEC7A63A7E27A8 /* KeyStylesRow.swift */, + AE6928CEB0A1A92241762DB6 /* KeysSettingsTab.swift */, 01F10B3261EA8F8616B8595A /* MainWindowRoot.swift */, + EB55696AF62110C04E63B601 /* ObservedKeyBindings.swift */, + 754D13A631535995AF41AA5B /* ReadAloudSettingsTab.swift */, 383E03AB60BA21AAD92655D5 /* ReadyView.swift */, A0EC4E3D73862F8E4DB6F47D /* RecentDictationsSection.swift */, 254D4D3CAC1C114EC51C96C8 /* SettingRow.swift */, @@ -1461,6 +1481,7 @@ D8F69C40E4A93A6B3433BAC0 /* Shared */ = { isa = PBXGroup; children = ( + 3879273E32399C14F3533899 /* KeyActionName.swift */, A0C585C5F805520ADA502A8F /* UITestIdentifiers.swift */, ); path = Shared; @@ -1478,7 +1499,6 @@ isa = PBXGroup; children = ( 3258A84E37C372993B1572EE /* DictationKeyTap.swift */, - C10286F2D7AD2A279FBBEBFC /* SelectionSpeechRouter.swift */, ); path = Hotkey; sourceTree = ""; @@ -1508,7 +1528,6 @@ E75844EDF6969530EFB56CF4 /* AccountRowView.swift */, EAC096B44D037E7BB119609A /* AccountSignIn.swift */, 3DA3A304528FD1C7FE6C2B94 /* APIKeyStepView.swift */, - 50FFCDBD1FB2CF0A81A68D18 /* HotkeyStepView.swift */, C59092F9220BBE7465068325 /* KeyTermsStepView.swift */, 4BE9BA7182953D06AAD913AE /* MicrophoneStepView.swift */, E8DE60178ABA2CC7F4EB6401 /* PermissionsStepView.swift */, @@ -2041,8 +2060,14 @@ 6D5CE0B54CB0E0055467B489 /* AccessibilityAuditUITests.swift in Sources */, EA50EC15CDECF534464D9208 /* BlurtUITestSupport.swift in Sources */, CE73DE20498B04ED1DDE6BAE /* DictationPipelineUITests.swift in Sources */, + B79801F65857FFF5829AE13A /* HarnessDriving.swift in Sources */, + 583D87E4396443D2DD0E4ED9 /* KeyActionName.swift in Sources */, + 4D344136EED62ED843F3BE76 /* KeysAndReadAloudUITests.swift in Sources */, + D928EB58EC2072A46170CC9D /* KeysLiveUpdateUITests.swift in Sources */, CBA0D3727CA1C62D0665F51B /* LaunchPerformanceUITests.swift in Sources */, B6B9598209C21737FC80D9FC /* MenuBarUITests.swift in Sources */, + 3E5C59CDCE131B9C602814AC /* ReadAloudFlowsUITests.swift in Sources */, + F74265E4664F68358FABC862 /* ReadAloudSettingsUITests.swift in Sources */, D05EC88A9F0FA1DF6F79A3DC /* ReadyViewUITests.swift in Sources */, 9549DA14B4FC5091F052B5B6 /* SettingsUITests.swift in Sources */, 08CF46B3518827BBA0139D07 /* TextShortcutsUITests.swift in Sources */, @@ -2063,7 +2088,6 @@ 8D2BA6E9FAC605D678A73164 /* AppDelegate.swift in Sources */, 034A91EE923BF5EA71438B7D /* BlurtBrand.swift in Sources */, 76C1876DB538F8E190EB509E /* BlurtCommands.swift in Sources */, - 121DC6CD82DEAC7CC0F5AC24 /* BoundTriggerKey.swift in Sources */, 68990A3D1D120CC927789338 /* BrandOrb.swift in Sources */, 28CA968C0302328A8C1EDA74 /* CheckForUpdatesButton.swift in Sources */, A5D51CD762229A68757463C8 /* CueSoundPlayer.swift in Sources */, @@ -2071,23 +2095,25 @@ 0F1317CCFA2A3245DAA96141 /* DictationComposition.swift in Sources */, 1C40B5DB92262E05808ECCEA /* DictationKeyTap.swift in Sources */, B5A7CEA6517E5A1E4D77623F /* FloatingPanel.swift in Sources */, - F42E6D4E62345772C5A49A08 /* HotkeyStepView.swift in Sources */, + 22526AFA400A8E77982C3B3A /* KeyActionName.swift in Sources */, 92041B42C67CE21AF82B6863 /* KeyCap.swift in Sources */, - 2638A7B2FF8239CF3D861F0C /* KeyStylesRow.swift in Sources */, 9B7CB19D131F1374FCA0A986 /* KeyTermsStepView.swift in Sources */, + 1C89CB94C7D10B67BD0573DF /* KeysSettingsTab.swift in Sources */, E8AE03AA4DCD807D2BB8D660 /* LiquidGlass.swift in Sources */, 7F838018EA0ACE7170FC32C4 /* MainWindowRoot.swift in Sources */, AD32046355138298E7FC40D0 /* MenuBarScene.swift in Sources */, EC0B15FB6EAAF91396A8BF50 /* MicrophoneStepView.swift in Sources */, + AFA3982826E71CE491DD82E1 /* ObservedKeyBindings.swift in Sources */, 74BD28AA54484635D62FA797 /* OverlayPillContent.swift in Sources */, 9437C292A6D7FE533338073D /* OverlayView.swift in Sources */, 8FE4B9B139BB9557FBC5589B /* OverlayWindowController.swift in Sources */, D750C1BF16502EEC92182ECD /* PermissionsStepView.swift in Sources */, + 24607B2B47E779840B7C225C /* ReadAloudSettingsTab.swift in Sources */, A13335FE97FA2E60A4F5CABC /* ReadyView.swift in Sources */, B5ADC67B78CA3EA70D6BAEB3 /* RecentDictationsSection.swift in Sources */, - 4563B6FCC9AA36F95CFD9D88 /* SelectionSpeechRouter.swift in Sources */, 6A1402BFED3758AAD9C84ADF /* SettingRow.swift in Sources */, 1D1100AB3D47EFC80D47B04B /* SettingsWindowRoot.swift in Sources */, + 526C973DCC1F588F91DEF1E5 /* SoundPackCatalog+ReadAloud.swift in Sources */, 8F3F1CC7203A148A68DFCCFF /* SoundPackCatalog.swift in Sources */, 69E27229D8C64D7119918B7A /* SoundStepView.swift in Sources */, 0708BB71FB0A48748BA508C0 /* StyleRow.swift in Sources */, diff --git a/App/Blurt/Blurt/AppCoordinator+UITesting.swift b/App/Blurt/Blurt/AppCoordinator+UITesting.swift index 06d33e64..f3308566 100644 --- a/App/Blurt/Blurt/AppCoordinator+UITesting.swift +++ b/App/Blurt/Blurt/AppCoordinator+UITesting.swift @@ -1,20 +1,33 @@ #if UITEST_HOOKS + import BlurtEngine + // MARK: - UI-test dictation drivers // - // XCUITest can't synthesize the lone right-modifier `flagsChanged` event the - // real trigger relies on — and the `CGEventTap` that reads it needs the - // process to be Accessibility-trusted, which the CI test host isn't. The test - // harness window instead calls `session.press()`/`release()`/`cancel()` - // directly, driving the *same* `DictationSession` (and its - // press-time missing-key readiness check) the key tap's closures reach via - // `session.submit(_:)`. Compiled only in Debug, so nothing here ships in the - // notarized Release build. + // XCUITest can't synthesize the lone-modifier `flagsChanged` event the real + // triggers rely on — and the `CGEventTap` that reads it needs the process to + // be Accessibility-trusted, which the CI test host isn't. The test harness + // window drives the *real* key tap's gate and callbacks instead, for any + // trigger key, so the whole path — the tap, the read-aloud router, the + // session — runs as a keypress would. Compiled only in Debug, so nothing here + // ships in the notarized Release build. extension AppCoordinator { - /// Fire a synthetic dictation-key press/release through the *real* key tap - /// (gate + callbacks), not the session directly. Lets the leak exercise - /// (`scripts/leaks.sh`) cover the DictationKeyTap object graph a real - /// keypress would build. No-ops if the tap wasn't created. - func simulateDictationPressForTesting() { keyTap?.simulatePressForTesting() } - func simulateDictationReleaseForTesting() { keyTap?.simulateReleaseForTesting() } + /// Fire a synthetic press/release of `key` (the bound key unless given) + /// through the real key tap. No-ops if the tap wasn't created. + func simulateKeyPressForTesting(_ key: TriggerKey? = nil) { keyTap?.simulatePressForTesting(key) } + func simulateKeyReleaseForTesting(_ key: TriggerKey? = nil, asTap: Bool = false) { + keyTap?.simulateReleaseForTesting(key, asTap: asTap) + } + + /// Sets how `key` is pressed, as the Keys tab would, and rebinds the tap. + func setKeyActivationForTesting(_ activation: TriggerActivation, for key: TriggerKey) { + TriggerKeyActivationStore().setActivation(activation, for: key) + dictationBindingChanged() + } + + /// Sets what `key` does, as the Keys tab would, and rebinds the tap. + func setKeyChoiceForTesting(_ choice: TriggerKeyChoice, for key: TriggerKey) { + TriggerKeyStyleStore().setChoice(choice, for: key) + dictationBindingChanged() + } } #endif diff --git a/App/Blurt/Blurt/AppCoordinator.swift b/App/Blurt/Blurt/AppCoordinator.swift index 70d8371d..f48d5506 100644 --- a/App/Blurt/Blurt/AppCoordinator.swift +++ b/App/Blurt/Blurt/AppCoordinator.swift @@ -1,3 +1,4 @@ +import AppKit import AssemblyAI import BlurtEngine import Foundation @@ -62,6 +63,13 @@ final class AppCoordinator { /// .isCapturing`) — the ready screen disables the style switcher on it, and /// its `.recording` half (via `menuBarStatus`) drives the listening state. private(set) var isCapturing = false + /// The last phase rendered, so the pill can go back to it when a read ends. + @ObservationIgnored private var lastPhase = PipelinePhase.idle + /// Whether a read or answer is playing — the pill shows it; UI tests read it. + private(set) var isReadingAloud = false + /// What reads aloud and how the selection is read, when not the real ones: + /// UI tests run the whole read-aloud path offline. Nil in production. + @ObservationIgnored private let readAloudSeams: ReadAloudSeams? /// `components` defaults to the production pipeline, its transcriber dictating /// against the account's `dictationOrigin`; `apiKey` defaults to the @@ -73,9 +81,11 @@ final class AppCoordinator { onSetupBlocked: @escaping @MainActor () -> Void, components: DictationComponents? = nil, apiKey: APIKeyModel = APIKeyModel(), - account: AssemblyAIAccount? = nil + account: AssemblyAIAccount? = nil, + readAloud: ReadAloudSeams? = nil ) { self.onSetupBlocked = onSetupBlocked + self.readAloudSeams = readAloud self.apiKey = apiKey let info = Bundle.main.infoDictionary ?? [:] let account = @@ -154,14 +164,17 @@ final class AppCoordinator { /// than a Carbon global hotkey: the latter leaks the trigger's auto-repeat key /// events into the focused app while held. private func startDictationDriver() { - // Every trigger command goes through the read-aloud router. With the - // experimental switch off, it forwards each one to `session.submit` exactly - // as before. - let router = SelectionSpeechRouter(session: session) + // Every trigger command goes through the read-aloud router, which forwards + // a dictating key's commands to `session.submit` and handles read and ask + // keys itself. + let router = + readAloudSeams.map { + SelectionSpeechRouter(session: session, speaker: $0.speaker, readSelection: $0.readSelection) + } ?? SelectionSpeechRouter(session: session) keyTap = DictationKeyTap( // The key rides the press, and the session resolves its style for that one // dictation (`PressStyle`); the main window's selection is never written. - onStart: { key in router.submit(.press(key: key)) }, + onStart: { key in router.press(key: key) }, onStop: { router.submit(.release) }, onCancel: { router.submit(.cancel) }, onLatch: { router.keyLatched() }, @@ -170,6 +183,25 @@ final class AppCoordinator { ) router.syncGateAfterSession = { [weak self] in self?.keyTap?.syncAfterTerminalPhase() } router.resetGate = { [weak self] in self?.keyTap?.resetGate() } + // A read wears the dictation pill — orb and meter, the meter following the + // voice — between the same start and stop cues. Pill first, then the cue, + // as `render(_:)` orders them. + router.speechStarted = { [weak self] in + self?.isReadingAloud = true + self?.overlay?.show(state: .speaking) + self?.cues.speechStarted() + } + router.speechEnded = { [weak self] in + guard let self else { return } + self.isReadingAloud = false + // Back to what the session is doing — a capture or transcription still + // under way — and never a dictation's old notice (the engine's rule). + self.overlay?.show(state: self.lastPhase.overlayStateAfterSpeech) + self.cues.speechEnded() + } + router.speechLevel = { [weak self] level in self?.overlay?.pushLevel(level) } + // The pill just goes away on a failed read, so say something went wrong. + router.speechFailed = { _ in NSSound.beep() } speechRouter = router // Deliberately *not* installed here: `CGEvent.tapCreate` for keystrokes is // itself what surfaces the system permission prompt, so creating the tap at @@ -268,7 +300,12 @@ final class AppCoordinator { /// otherwise (see `GlobeKeyOverride`). Also run from `start()`, so the relaunch /// after a settings reset restores the user's setting. private func syncGlobeKeyOverride() { - GlobeKeyOverride().sync(boundKeys: TriggerKeyStyleStore().enabledKeys) + #if UITEST_HOOKS + // UI tests set fn to every action; the 🌐 setting it writes is the Mac's + // own, one per login, and no test may change it. + if UITestMode.isActive { return } + #endif + GlobeKeyOverride().sync(boundKeys: TriggerKeyBindings().enabledKeys) } // MARK: - Dictation render @@ -282,6 +319,17 @@ final class AppCoordinator { cues.packChanged() } + /// Read-aloud's cue pack changed in Settings: reload and preview it. + func readAloudSoundPackChanged() { + cues.readPackChanged() + } + + /// The Read Aloud tab picked a voice or a speed: play a sample in it, as a cue + /// sound plays when it's picked. + func readAloudStyleChanged() { + speechRouter?.previewSample() + } + private func render(_ phase: PipelinePhase) { // A setup blocker is a state, not a fault: the engine projections below // render it as calm idle (no red flash) and the menu bar ignores it — the only @@ -295,7 +343,12 @@ final class AppCoordinator { // front of the visual state change. Pure phase→pill mapping lives in the // engine (unit-tested there); .failed resolves to .error, which the pill // flashes red then auto-reverts to idle. - overlay?.show(state: phase.overlayState) + // A read wears the pill until it ends (the engine's rule — see + // `overlayState(whileSpeaking:)`). + lastPhase = phase + if let state = phase.overlayState(whileSpeaking: speechRouter?.isSpeaking ?? false) { + overlay?.show(state: state) + } // Mirror the phase onto the menu bar indicator (mapping lives in the engine, // unit-tested alongside `overlayState`). menuBarStatus = phase.menuBarStatus @@ -319,3 +372,10 @@ final class AppCoordinator { speechRouter?.sessionPhaseChanged(phase) } } + +/// Stand-ins for read-aloud's speaker and selection read, so a host can run the +/// read-aloud path with no network, audio or Accessibility (Blurt's UI tests). +struct ReadAloudSeams { + let speaker: any ReadAloudSpeaker + let readSelection: @Sendable () async -> String? +} diff --git a/App/Blurt/Blurt/AppDelegate.swift b/App/Blurt/Blurt/AppDelegate.swift index 9fa2d3ac..f01df890 100644 --- a/App/Blurt/Blurt/AppDelegate.swift +++ b/App/Blurt/Blurt/AppDelegate.swift @@ -116,7 +116,12 @@ final class AppDelegate: NSObject, NSApplicationDelegate { apiKey: apiKey, // No configuration: the suite covers the paste-a-key path, and sign-in // would open a real browser. - account: AssemblyAIAccount(configuration: nil, apiKey: apiKey, store: .inMemory())) + account: AssemblyAIAccount(configuration: nil, apiKey: apiKey, store: .inMemory()), + // Read-aloud offline: a canned selection, and a speaker that records + // what it's asked to read instead of streaming TTS. + readAloud: ReadAloudSeams( + speaker: UITestReadAloudSpeaker(), + readSelection: { [selection = UITestIdentifiers.cannedSelection] in selection })) } else { coord = AppCoordinator(onSetupBlocked: onSetupBlocked) } @@ -255,9 +260,9 @@ final class AppDelegate: NSObject, NSApplicationDelegate { // Accessibility trust, but its gate/callback graph is still exercised.) coord.showOverlay() for _ in 0..<5 { - coord.simulateDictationPressForTesting() + coord.simulateKeyPressForTesting() try? await Task.sleep(for: .milliseconds(120)) // let press() reach .recording - coord.simulateDictationReleaseForTesting() + coord.simulateKeyReleaseForTesting() try? await Task.sleep(for: .milliseconds(180)) // let transcribe→inject settle } coord.hideOverlay() diff --git a/App/Blurt/Blurt/CueSoundPlayer.swift b/App/Blurt/Blurt/CueSoundPlayer.swift index f412da66..d19d9b36 100644 --- a/App/Blurt/Blurt/CueSoundPlayer.swift +++ b/App/Blurt/Blurt/CueSoundPlayer.swift @@ -1,23 +1,15 @@ import AVFoundation import BlurtEngine -/// Owns the record start/stop cue chimes: loading the selected pack, pre-rolling -/// so the first chime never stalls the recording pill, previewing on a pack -/// change, and firing on the recording edge. Kept out of `AppCoordinator`'s body -/// so chime behavior can change without churning the session↔UI wiring. +/// Owns the cue chimes — dictation's start/stop pair and read-aloud's own, each +/// a `CuePair` from its own setting: loading the selected pack, pre-rolling so +/// the first chime never stalls the pill, previewing on a pack change, and +/// firing on the recording edge or a read's start and end. Kept out of +/// `AppCoordinator`'s body so chime behavior can change without churning the +/// session↔UI wiring. final class CueSoundPlayer { - private var startSound: AVAudioPlayer? - private var stopSound: AVAudioPlayer? - /// The pack the current `startSound`/`stopSound` were decoded from, so `prime()` - /// can skip re-decoding when nothing changed. `nil` until the first load. - private var loadedPack: SoundPack? - /// Monotonic ticket for in-flight decodes: only the newest one installs its - /// players. This is the staleness check, **not** `loadedPack` equality — which - /// cannot discriminate two loads of the *same* pack, and a route change forces - /// exactly that. Without it, a decode already in flight when the route changed - /// could land after the re-prime and install players primed against the old - /// output route — the stall the re-prime exists to prevent. - private var loadGeneration = 0 + private let dictation = CuePair(store: .dictation) + private let readAloud = CuePair(store: .readAloud) /// Edge-detector deciding when the start/stop chimes fire. The mapping from a /// pipeline phase to a cue lives in the engine (`RecordingCueGate`), where /// `swift test` covers it; this player just plays whatever it resolves to. @@ -27,18 +19,13 @@ final class CueSoundPlayer { /// output-only profile, which drops the output format the players were primed /// against — so the very next chime is the one that stalls, and that's the /// chime at the start of a dictation. - /// Set when the output route changes, cleared when the re-prime actually runs. - /// See `transition(for:)` for why the reload waits. - private var needsReprime = false + /// When each pair reloads after an output-route change (the engine's + /// `CueReprimeSchedule`; see `transition(for:)` for why dictation's waits). + private var reprime = CueReprimeSchedule() /// Kept alive for the app's lifetime; assignment is the use. Nil until /// `prime()` starts it, which is also what makes starting idempotent. private var routeObserver: Task? - /// The cues are deliberate UI accents, not music — they are normalized to a - /// hot peak, so play them well below full scale so they read as a soft chime - /// rather than blasting at the system output level. - private nonisolated static let cueVolume: Float = 0.35 - /// (Re)loads and pre-rolls the cue players for the selected sound pack so /// the first start/stop chime adds no latency to the pill; called from the /// "app is ready" transition, well before the hot path. `.none` (or a @@ -46,14 +33,17 @@ final class CueSoundPlayer { /// genuinely cheap on repeat: the decoded players are cached per pack, so a /// re-prime for the already-loaded pack returns immediately instead of /// re-decoding the AAC. A real pack switch flows through `packChanged()` with a - /// new selection, so `loadCurrentPack` reloads. + /// new selection, so `CuePair.load` reloads. /// /// Fire-and-forget: the actual decode runs off the main actor (see `decode`) and /// nothing waits on it — the players only need to be ready before the *first /// dictation*, which is always well after this launch-time call, so the AAC /// decode never sits on the main thread during startup. func prime() { - Task { await loadCurrentPack() } + Task { + await dictation.load() + await readAloud.load() + } startObservingRoute() } @@ -87,8 +77,7 @@ final class CueSoundPlayer { // keeps the player alive — the same reason `MicCapture`'s meter task // rebinds across its sleep. guard let self else { return } - // Recorded, not acted on — see `transition(for:)`. - self.needsReprime = true + self.reload(self.reprime.routeChanged()) } } } @@ -104,40 +93,14 @@ final class CueSoundPlayer { AudioRouteMonitor() } - /// Reloads the players for a newly selected pack and previews the new voice - /// (start, then stop a beat apart) so the choice is audible immediately — - /// after the reload lands, so the preview uses the freshly decoded players. - /// Silent for the `.none` pack (all players are nil, which `play(_:)` skips). + /// Previews a newly selected dictation pack (`CuePair.preview`). func packChanged() { - Task { - await loadCurrentPack() - play(startSound) - try? await Task.sleep(for: .milliseconds(380)) - play(stopSound) - } + Task { await dictation.preview() } } - /// Decodes and installs the cue players for the current selection if they aren't - /// already loaded. The `loadedPack` guard makes repeat calls cheap; the decode - /// itself hops off the main actor. Returns once the players are assigned. - /// `force` skips the already-loaded short-circuit, for a re-prime where the - /// selection is unchanged and only the output route moved. - private func loadCurrentPack(force: Bool = false) async { - let pack = SoundPackStore(catalog: .blurt).soundPack - guard force || pack != loadedPack else { return } - loadGeneration += 1 - let generation = loadGeneration - loadedPack = pack - let players = await Self.decode(pack) - // Re-check after the off-actor decode: whoever asked last owns the players. - // Both are written synchronously above (no await between read and write), so - // exactly one in-flight decode passes this guard — the newest. Ticketed - // rather than compared against `loadedPack`, so two loads of the same pack - // (a rapid pack switch back, or any forced route re-prime) can still tell - // each other apart. - guard generation == loadGeneration else { return } - startSound = players.start - stopSound = players.stop + /// Previews a newly selected read-aloud pack. + func readPackChanged() { + Task { await readAloud.preview() } } /// Fires the start/stop cue on the recording edge. Call once per rendered @@ -145,8 +108,8 @@ final class CueSoundPlayer { /// the engine's `RecordingCueGate`). func transition(for phase: PipelinePhase) { switch cueGate.cue(for: phase) { - case .start: play(startSound) - case .stop: play(stopSound) + case .start: dictation.playStart() + case .stop: dictation.playStop() case nil: break } // A pending route re-prime is acted on at exactly two points. @@ -172,12 +135,90 @@ final class CueSoundPlayer { // Terminal phases stay a trigger for everything else. A flip caused by *this* // press arrives mid-bring-up, too late for this dictation's chime either way, // and between dictations the reload costs nothing anyone is waiting on and - // coalesces a burst of flips into one decode. - guard phase == .connecting || phase.isTerminal, needsReprime else { return } - needsReprime = false - Task { await loadCurrentPack(force: true) } + // coalesces a burst of flips into one decode. Read-aloud's pair reloads at + // once when nothing is capturing (reads produce no phases to wait for), + // else at the capture's end, so it never competes with a bring-up. + reload(reprime.phase(phase)) + } + + /// Reloads the pairs `reloads` names, each against the current route. + private func reload(_ reloads: CueReprimeSchedule.Reloads) { + if reloads.dictation { Task { await dictation.load(force: true) } } + if reloads.readAloud { Task { await readAloud.load(force: true) } } } + /// Read-aloud's start and stop, from its own pack, so a read never sounds + /// like a recording starting. + func speechStarted() { readAloud.playStart() } + func speechEnded() { readAloud.playStop() } +} + +/// One setting's start/stop chimes: its store, the decoded players, and the +/// load that keeps them current. +private final class CuePair { + private let store: SoundPackStore + private var start: AVAudioPlayer? + private var stop: AVAudioPlayer? + /// The pack the current players were decoded from, so a repeat load can skip + /// re-decoding when nothing changed. `nil` until the first load. + private var loaded: SoundPack? + /// Monotonic ticket for in-flight decodes: only the newest one installs its + /// players. This is the staleness check, **not** `loaded` equality — which + /// cannot discriminate two loads of the *same* pack, and a route change forces + /// exactly that. Without it, a decode already in flight when the route changed + /// could land after the re-prime and install players primed against the old + /// output route — the stall the re-prime exists to prevent. + private var generation = 0 + + /// The beat between a preview's start and stop chimes. + private static let previewGap = Duration.milliseconds(380) + + /// The cues are deliberate UI accents, not music — they are normalized to a + /// hot peak, so play them well below full scale so they read as a soft chime + /// rather than blasting at the system output level. + private nonisolated static let cueVolume: Float = 0.35 + + init(store: SoundPackStore) { + self.store = store + } + + /// Decodes and installs the players for the current selection if they aren't + /// already loaded. The decode itself hops off the main actor. Returns once + /// the players are assigned. `force` skips the already-loaded short-circuit, + /// for a re-prime where the selection is unchanged and only the output route + /// moved. + func load(force: Bool = false) async { + let pack = store.soundPack + guard force || pack != loaded else { return } + generation += 1 + let ticket = generation + loaded = pack + let players = await Self.decode(pack) + // Re-check after the off-actor decode: whoever asked last owns the players. + // Both are written synchronously above (no await between read and write), so + // exactly one in-flight decode passes this guard — the newest. Ticketed + // rather than compared against `loaded`, so two loads of the same pack (a + // rapid pack switch back, or any forced route re-prime) can still tell + // each other apart. + guard ticket == generation else { return } + start = players.start + stop = players.stop + } + + /// Reloads for a newly selected pack and previews it (start, then stop a beat + /// apart) so the choice is audible immediately — after the reload lands, so + /// the preview uses the freshly decoded players. Silent for `.none` (both + /// players are nil, which `play(_:)` skips). + func preview() async { + await load() + playStart() + try? await Task.sleep(for: Self.previewGap) + playStop() + } + + func playStart() { Self.play(start) } + func playStop() { Self.play(stop) } + /// The decoded, pre-rolled players for a pack. Non-`Sendable` (holds /// `AVAudioPlayer`), so `decode` hands it back via `sending`. `nonisolated` so it /// can be built inside the off-main `decode` (the app defaults to MainActor @@ -215,7 +256,7 @@ final class CueSoundPlayer { /// first means a cue replays cleanly even if the previous play hasn't been /// reset, and keeping this off the visual path (callers reveal the pill /// first) guarantees the sound never delays the overlay. - private func play(_ sound: AVAudioPlayer?) { + private static func play(_ sound: AVAudioPlayer?) { guard let sound else { return } sound.currentTime = 0 sound.play() diff --git a/App/Blurt/Blurt/Hotkey/DictationKeyTap.swift b/App/Blurt/Blurt/Hotkey/DictationKeyTap.swift index 37442da7..5bbe8f1b 100644 --- a/App/Blurt/Blurt/Hotkey/DictationKeyTap.swift +++ b/App/Blurt/Blurt/Hotkey/DictationKeyTap.swift @@ -4,10 +4,11 @@ import os /// Drives the lone-modifier dictation triggers from a `CGEventTap`. /// -/// Watches `flagsChanged` for the bound modifiers (e.g. right ⌘, keycode 54, plus -/// any trigger key given a style — `TriggerKeyStyleStore`) to detect down/up, and -/// `keyDown` for any *other* key to spot a modifier combo -/// (⌘C, ⌘V…). The per-event decision lives in the engine — `DictationKeyRouter` +/// Watches `flagsChanged` for every trigger modifier given an action (e.g. right ⌘, +/// keycode 54, dictating and left ⌥ reading aloud), each under its own +/// tap/hold mode — `TriggerKeyBindings`, to detect down/up, and +/// `keyDown` for any *other* key, or a mouse-button down, to spot a modifier +/// combo (⌘C, ⌘V, ⌘-click…). The per-event decision lives in the engine — `DictationKeyRouter` /// (keycode relevance + down/up edge dedup) over `DictationKeyGate` (tap/hold /// semantics) — so this type only reduces each `CGEvent` to a router event and /// owns the tap lifecycle. @@ -78,12 +79,11 @@ final class DictationKeyTap { self.onRecordingDiscarded = onRecordingDiscarded // Every half of the binding comes from its store, not a hard-coded // `.rightCommand` / `.tapOrHold`: `fromPersisted` owns the unset default, and - // restating it here is the same mistake `BoundTriggerKey` and `HotkeyStepView` + // restating it here is the same mistake `ObservedKeyBindings` and `KeysSettingsTab` // were each corrected away from — the tap would name the old key while the // picker, ready screen, and menu bar all named the new one. `refreshBinding()` // re-reads this, but nothing enforces that it runs before the first event. - self.router = DictationKeyRouter( - triggerKeyCodes: Self.boundKeyCodes(), activation: TriggerActivationStore().activation) + self.router = DictationKeyRouter(activations: Self.boundActivations()) } deinit { @@ -115,7 +115,14 @@ final class DictationKeyTap { CGEvent.tapEnable(tap: tap, enable: true) return true } - let mask = (1 << CGEventType.keyDown.rawValue) | (1 << CGEventType.flagsChanged.rawValue) + // Mouse-button downs too: a modifier held through a click is a + // modifier-click (⌘-click a link, ⌥-click in Finder), which must cancel the + // press rather than read as a lone tap of the trigger. Listen-only, so + // clicks pass through untouched. + let mask = + (1 << CGEventType.keyDown.rawValue) | (1 << CGEventType.flagsChanged.rawValue) + | (1 << CGEventType.leftMouseDown.rawValue) | (1 << CGEventType.rightMouseDown.rawValue) + | (1 << CGEventType.otherMouseDown.rawValue) guard let created = CGEvent.tapCreate( tap: .cgSessionEventTap, @@ -184,14 +191,13 @@ final class DictationKeyTap { _ = router.reset() } - /// Re-read the bound trigger keys and activation mode into the router. Call - /// after the user rebinds any of them. The router's reset reports a discarded live + /// Re-read the trigger keys and their modes into the router. Call after the + /// user changes any of them in the Keys tab. The router's reset reports a discarded live /// recording: rebinding mid-dictation means the old key's up-event will never /// match, so the capture must be cancelled, not left to run out the /// auto-release cap. func refreshBinding() { - let activation = TriggerActivationStore().activation - if router.rebind(triggerKeyCodes: Self.boundKeyCodes(), activation: activation) { + if router.rebind(activations: Self.boundActivations()) { onRecordingDiscarded() } } @@ -227,6 +233,13 @@ final class DictationKeyTap { return .fromFlags(keyCode: keyCode, rawFlags: event.flags.rawValue) case .keyDown: return .keyDown(keyCode: keyCode) + case .leftMouseDown, .rightMouseDown, .otherMouseDown: + #if UITEST_HOOKS + // A simulated press is "held" while the test clicks the harness's + // release button; that click isn't a modifier-click. + if isSimulatingPress { return nil } + #endif + return .pointerDown default: return nil } @@ -248,32 +261,41 @@ final class DictationKeyTap { /// The trigger key that owns the router's gate, if any. private var activeKey: TriggerKey? { router.activeKeyCode.flatMap(TriggerKey.init(rawValue:)) } - /// Every key that dictates: the bound one plus each key given a style. - private static func boundKeyCodes() -> Set { - Set(TriggerKeyStyleStore().enabledKeys.map(\.keyCode)) + /// Every key that does something (dictate, read aloud, ask), with its tap/hold mode. + private static func boundActivations() -> [Int: TriggerActivation] { + Dictionary(uniqueKeysWithValues: TriggerKeyBindings().activations.map { ($0.key.keyCode, $0.value) }) } #if UITEST_HOOKS + /// Between `simulatePressForTesting()` and its release: clicks don't cancel. + private var isSimulatingPress = false + /// Test seam: drive the real gate + callback dispatch for a synthetic - /// lone-modifier press, bypassing the `CGEventTap` (whose creation needs - /// Accessibility trust an automated run doesn't have). Pairs with - /// `simulateReleaseForTesting()` to run the same press→hold→release path a - /// real keypress would — used by the leak exercise (`scripts/leaks.sh`) so the - /// DictationKeyTap → DictationKeyGate → onStart/onStop object graph is - /// covered, not just the session the coordinator drives directly. - func simulatePressForTesting() { - _ = router.reset() - dispatch( - router.handle( - .flagsChanged(keyCode: TriggerKeyStore().triggerKey.keyCode, triggerFlagIsOn: true), at: .seconds(0))) + /// lone-modifier press of `key` (the bound key unless given), bypassing the + /// `CGEventTap` (whose creation needs Accessibility trust an automated run + /// doesn't have). Pairs with `simulateReleaseForTesting` to run the same + /// press→release path a real keypress would — used by the UI tests, for + /// every trigger key and action, and by the leak exercise + /// (`scripts/leaks.sh`) so the DictationKeyTap → DictationKeyGate → + /// onStart/onStop object graph is covered, not just the session the + /// coordinator drives directly. A key that does nothing (Off) isn't + /// watched, so its press starts nothing, as a real one wouldn't. + func simulatePressForTesting(_ key: TriggerKey? = nil) { + // No reset first: a press over a latched recording (a tapped ask's or + // dictation's second tap) must reach the gate as a real one would. + isSimulatingPress = true + let keyCode = (key ?? TriggerKeyStore().triggerKey).keyCode + dispatch(router.handle(.flagsChanged(keyCode: keyCode, triggerFlagIsOn: true), at: .seconds(0))) } - /// Completes the synthetic cycle as a hold (past the threshold), so the gate - /// emits `.stop` and `onStop` fires. - func simulateReleaseForTesting() { + /// Completes the synthetic cycle: as a hold (past the threshold, so the + /// gate emits `.stop`) or a tap (`.latch` under tap-or-hold). + func simulateReleaseForTesting(_ key: TriggerKey? = nil, asTap: Bool = false) { + isSimulatingPress = false + let keyCode = (key ?? TriggerKeyStore().triggerKey).keyCode dispatch( router.handle( - .flagsChanged(keyCode: TriggerKeyStore().triggerKey.keyCode, triggerFlagIsOn: false), at: .seconds(2))) + .flagsChanged(keyCode: keyCode, triggerFlagIsOn: false), at: asTap ? .milliseconds(100) : .seconds(2))) } #endif } diff --git a/App/Blurt/Blurt/Hotkey/SelectionSpeechRouter.swift b/App/Blurt/Blurt/Hotkey/SelectionSpeechRouter.swift deleted file mode 100644 index 8b74a181..00000000 --- a/App/Blurt/Blurt/Hotkey/SelectionSpeechRouter.swift +++ /dev/null @@ -1,110 +0,0 @@ -import AppKit -import BlurtEngine - -/// Sits between the trigger key and the dictation session, and turns a press -/// over selected text into "read it aloud" when the experimental switch is on -/// (`SelectionSpeechStore`), or, holding the key with asking on -/// (`SelectionAskStore`), into "answer what I'm saying about it". When the -/// switch is off, every command goes straight to `DictationSession.submit`. -/// -/// The decisions are the engine's `SelectionSpeechRouting`. This class carries -/// out the actions it returns: the selection read, the hold timer, the speech -/// task, and the gate resets. -/// -/// The cost: while the switch is on, a dictation press starts one AX read later. -/// That's a few milliseconds in a native app, but can be tens to hundreds in an -/// Electron app, and up to the AX timeout for each round trip against a hung -/// one. Recording starts at the press, so a slow read clips the first words. -/// An ask doesn't pay it: its recording starts at the hold, after the start -/// chime the user waits for. -final class SelectionSpeechRouter { - private static let logger = HostIdentity.current.logger("SelectionSpeech") - - private let session: DictationSession - private let speaker = SelectionSpeaker() - private var routing = SelectionSpeechRouting() - private var speech: Task? - /// The pending `holdElapsed`, restarted by each press that may ask. - private var holdTimer: Task? - /// `DictationKeyTap.syncAfterTerminalPhase`: a dictation ended without a key event. - var syncGateAfterSession: () -> Void = {} - /// `DictationKeyTap.resetGate`: a read ended without a key event. - var resetGate: () -> Void = {} - - init(session: DictationSession) { - self.session = session - } - - func submit(_ command: DictationSession.Command) { - perform( - routing.submit( - command, readAloudEnabled: SelectionSpeechStore().isEnabled, askEnabled: SelectionAskStore().isEnabled)) - } - - /// `DictationKeyTap`'s latch: the press was a tap. - func keyLatched() { - perform(routing.keyLatched()) - } - - func sessionPhaseChanged(_ phase: PipelinePhase) { - perform(routing.sessionPhaseChanged(phase)) - } - - private func perform(_ actions: [SelectionSpeechRouting.Action]) { - for action in actions { - switch action { - case .forward(let command): session.submit(command) - case .readSelection: - Task { [weak self] in - let selection = await SelectionSpeaker.focusedSelection() - guard let self else { return } - self.perform(self.routing.selectionResolved(selection)) - } - case .startHoldTimer: startHoldTimer() - case .speak(let text): - // Read per press, like the switch itself, so a Settings change applies - // to the next read. - let style = ReadAloudStyleStore().style - run { [speaker] in try await speaker.speak(text, style: style) } - case .answer(let request, let selection): - let style = ReadAloudStyleStore().style - run { [speaker] in try await speaker.answer(request, about: selection, style: style) } - case .stopSpeech: - speech?.cancel() - speech = nil - case .syncGateAfterSession: syncGateAfterSession() - case .resetGate: resetGate() - } - } - } - - /// One timer at a time: a newer press cancels the older one's, so a timer - /// left over from a quick tap can't mark the next press as a hold. - private func startHoldTimer() { - holdTimer?.cancel() - holdTimer = Task { [weak self] in - try? await Task.sleep(for: SelectionSpeechRouting.askHoldDelay) - guard !Task.isCancelled, let self else { return } - self.holdTimer = nil - self.perform(self.routing.holdElapsed()) - } - } - - /// Runs one read or answer as the current speech task. - private func run(_ work: @escaping @Sendable () async throws -> Void) { - speech = Task { [weak self] in - do { - try await work() - } catch is CancellationError { - } catch { - Self.logger.error("read-aloud failed: \(error.localizedDescription, privacy: .public)") - NSSound.beep() - } - // A read this router stopped was cancelled, and a newer one may already - // be playing, so only a read that ended on its own reports back. - guard !Task.isCancelled, let self else { return } - self.speech = nil - self.perform(self.routing.speechFinished()) - } - } -} diff --git a/App/Blurt/Blurt/MenuBar/MenuBarScene.swift b/App/Blurt/Blurt/MenuBar/MenuBarScene.swift index d177297e..a06c431b 100644 --- a/App/Blurt/Blurt/MenuBar/MenuBarScene.swift +++ b/App/Blurt/Blurt/MenuBar/MenuBarScene.swift @@ -30,11 +30,10 @@ struct MenuBarContent: View { @Environment(\.openSettings) private var openSettings // Observed (as the ready screen does) so the reminder line updates live when - // the dictation key is rebound in Settings — see `BoundTriggerKey`. - @BoundTriggerKey private var triggerKey - @AppStorage(TriggerActivationStore.defaultsKey) private var activationRaw = "" - - private var activation: TriggerActivation { .fromPersisted(activationRaw) } + // the dictation key or its mode changes in Settings (`ObservedKeyBindings`). + @ObservedKeyBindings private var bindings + private var triggerKey: TriggerKey { bindings.bound } + private var activation: TriggerActivation { bindings.activation(for: bindings.bound) } var body: some View { // Disabled informational row: the dictation trigger is an invisible lone diff --git a/App/Blurt/Blurt/Overlay/OverlayView.swift b/App/Blurt/Blurt/Overlay/OverlayView.swift index 45768ec3..31012b9d 100644 --- a/App/Blurt/Blurt/Overlay/OverlayView.swift +++ b/App/Blurt/Blurt/Overlay/OverlayView.swift @@ -97,7 +97,8 @@ struct OverlayView: View { // which is the whole point. waitingContent("Connecting") .transition(.opacity) - case .recording: + case .recording, .speaking: + // A read wears the same pill, the meter following the voice instead. // The orb beside the live waveform, its ring sweeping as it does in every // other state — the orb is one object and doesn't change behaviour with // the content next to it. The meter carries the level; the ring carries diff --git a/App/Blurt/Blurt/SoundPackCatalog+ReadAloud.swift b/App/Blurt/Blurt/SoundPackCatalog+ReadAloud.swift new file mode 100644 index 00000000..4a6287c3 --- /dev/null +++ b/App/Blurt/Blurt/SoundPackCatalog+ReadAloud.swift @@ -0,0 +1,15 @@ +import BlurtEngine + +extension SoundPackCatalog { + /// Read-aloud's cue voices: the same as dictation's, defaulting to the + /// Juno-106's Brass, so a read sounds unlike a recording starting. Apart from + /// the generated catalog file, which is the generator's to write. + static let blurtReadAloud = SoundPackCatalog.blurt.defaulting(to: "juno-0") +} + +extension SoundPackStore { + /// Each cue's store, tied to its catalog here once, so a read-aloud setting + /// can't be read under dictation's default or the other way round. + static var dictation: SoundPackStore { SoundPackStore(catalog: .blurt) } + static var readAloud: SoundPackStore { SoundPackStore(catalog: .blurtReadAloud, cue: .readAloud) } +} diff --git a/App/Blurt/Blurt/UITestSupport.swift b/App/Blurt/Blurt/UITestSupport.swift index 6c304e2a..0e78eb96 100644 --- a/App/Blurt/Blurt/UITestSupport.swift +++ b/App/Blurt/Blurt/UITestSupport.swift @@ -62,6 +62,26 @@ private(set) var pastedText = "" func recordPaste(_ text: String) { pastedText = text } + + /// The style instructions the last dictation request carried, or "—" for + /// none: how a test sees which style a key dictated with. + private(set) var sentStyle = "—" + + func recordStyle(_ instructions: String?) { sentStyle = instructions ?? "—" } + + /// What the stub speaker was last asked to read ("—" before any read). + private(set) var spokenText = "—" + + func recordSpoken(_ text: String) { spokenText = text } + + /// The voice and speed the stub speaker was last asked to use, e.g. + /// "aai-jane @ 1.0" ("—" before any read). + private(set) var spokenStyle = "—" + + func recordSpokenStyle(_ style: ReadAloudStyle) { spokenStyle = "\(style.voiceID) @ \(style.rate)" } + + /// The trigger key the harness's press/release buttons drive. + var selectedKey = TriggerKey.rightCommand } // MARK: - Pipeline doubles @@ -100,7 +120,10 @@ // context needs no draining now — the streaming route settles it before // the request opens, so it arrives as a value. for await _ in frames {} - return await MainActor.run { UITestState.shared.cannedTranscript } + return await MainActor.run { + UITestState.shared.recordStyle(context?.styleInstructions) + return UITestState.shared.cannedTranscript + } } } @@ -115,6 +138,26 @@ } } + /// Stub read-aloud speaker: records what it's asked to read, then "plays" + /// until stopped (or a while passes), so a test can watch a read start, run + /// and be interrupted — no network, no audio. + nonisolated struct UITestReadAloudSpeaker: ReadAloudSpeaker { + func speak(_ text: String, style: ReadAloudStyle, onLevel: (@Sendable (Float) -> Void)?) async throws { + await MainActor.run { + UITestState.shared.recordSpoken(text) + UITestState.shared.recordSpokenStyle(style) + } + try await Task.sleep(for: .seconds(20)) + } + + func answer( + _ request: String, about selection: String, style: ReadAloudStyle, onLevel: (@Sendable (Float) -> Void)? + ) async throws { + await MainActor.run { UITestState.shared.recordSpoken("Answer to: \(request)") } + try await Task.sleep(for: .seconds(20)) + } + } + // The real Keychain stays untouched under UI testing via the engine's // `InMemoryAPIKeyStore` (`Sources/BlurtEngine/Config/APIKeyGateway.swift`), // injected as the coordinator's key store in `AppDelegate`. @@ -188,17 +231,38 @@ .accessibilityIdentifier(UITestIdentifiers.cancelButton) } - // A second row that drives dictation through the *real* key tap - // (DictationKeyTap → DictationKeyGate → the coordinator's onStart/onStop), - // rather than calling the session directly like Start/Stop above. Lets a - // UI test exercise the hotkey path end to end without synthesizing the - // lone-modifier CGEvent (which needs Accessibility trust the test host - // lacks). Press then Release is a hold, so it records then transcribes. + // Drives the picked trigger key through the *real* key tap + // (DictationKeyTap → DictationKeyGate → the read-aloud router → the + // session), rather than calling the session directly like Start/Stop + // above. Lets a UI test exercise every key's path end to end without + // synthesizing the lone-modifier CGEvent (which needs Accessibility + // trust the test host lacks). Press then Release is a hold; Tap + // Release lets go quickly, as a tap. + Picker("Key", selection: $state.selectedKey) { + ForEach(TriggerKey.allCases, id: \.self) { key in + Text(key.label).tag(key) + } + } + .accessibilityIdentifier(UITestIdentifiers.harnessKeyPicker) + HStack(spacing: 8) { + Button("Dictate") { coordinator?.setKeyChoiceForTesting(.dictate, for: state.selectedKey) } + .accessibilityIdentifier(UITestIdentifiers.harnessSetDictate) + Button("Read") { coordinator?.setKeyChoiceForTesting(.readAloud, for: state.selectedKey) } + .accessibilityIdentifier(UITestIdentifiers.harnessSetReadAloud) + Button("Ask") { coordinator?.setKeyChoiceForTesting(.ask, for: state.selectedKey) } + .accessibilityIdentifier(UITestIdentifiers.harnessSetAsk) + Button("Off") { coordinator?.setKeyChoiceForTesting(.off, for: state.selectedKey) } + .accessibilityIdentifier(UITestIdentifiers.harnessSetOff) + Button("Tap Mode") { coordinator?.setKeyActivationForTesting(.tap, for: state.selectedKey) } + .accessibilityIdentifier(UITestIdentifiers.harnessSetTapMode) + } HStack(spacing: 8) { - Button("Hotkey Press") { coordinator?.simulateDictationPressForTesting() } + Button("Hotkey Press") { coordinator?.simulateKeyPressForTesting(state.selectedKey) } .accessibilityIdentifier(UITestIdentifiers.hotkeyPressButton) - Button("Hotkey Release") { coordinator?.simulateDictationReleaseForTesting() } + Button("Hotkey Release") { coordinator?.simulateKeyReleaseForTesting(state.selectedKey) } .accessibilityIdentifier(UITestIdentifiers.hotkeyReleaseButton) + Button("Tap Release") { coordinator?.simulateKeyReleaseForTesting(state.selectedKey, asTap: true) } + .accessibilityIdentifier(UITestIdentifiers.hotkeyTapReleaseButton) } // The live pipeline status, mirrored off the same `menuBarStatus` the @@ -215,6 +279,25 @@ .accessibilityIdentifier(UITestIdentifiers.pastedLabel) } + // Read-aloud: what the stub speaker was asked to read, whether a read is + // playing, and the style the last dictation request carried. + LabeledContent("Spoken") { + Text(state.spokenText) + .accessibilityIdentifier(UITestIdentifiers.spokenLabel) + } + LabeledContent("Reading") { + Text(coordinator?.isReadingAloud == true ? "yes" : "no") + .accessibilityIdentifier(UITestIdentifiers.readingLabel) + } + LabeledContent("Voice") { + Text(state.spokenStyle) + .accessibilityIdentifier(UITestIdentifiers.spokenStyleLabel) + } + LabeledContent("Style") { + Text(state.sentStyle) + .accessibilityIdentifier(UITestIdentifiers.sentStyleLabel) + } + // Mirrors the newest entry of `AppCoordinator.recentDictations` (the // ready-window "Recent" list) so a test can watch it populate on a // completed dictation. Placeholder "—" stands in for the empty list so diff --git a/App/Blurt/Blurt/Wizard/BoundTriggerKey.swift b/App/Blurt/Blurt/Wizard/BoundTriggerKey.swift deleted file mode 100644 index 67d73ce4..00000000 --- a/App/Blurt/Blurt/Wizard/BoundTriggerKey.swift +++ /dev/null @@ -1,22 +0,0 @@ -import BlurtEngine -import SwiftUI - -/// Live-updating read of the bound dictation trigger key, for the views that -/// only *display* it ("Tap or hold right ⌘ to blurt"). -/// -/// Wraps the `@AppStorage` + `TriggerKey.fromPersisted` pair that three views — -/// the ready screen, the menu bar menu, and the permissions footer — each spelled -/// out, along with a restated `TriggerKey.rightCommand.rawValue` default that -/// `fromPersisted` already owns (an unset default reads as 0, which it maps to -/// right ⌘). Reading the raw keycode through `@AppStorage`, rather than calling -/// `TriggerKeyStore()` once, is what makes these views re-render when the key is -/// rebound in the separate Settings window. -/// -/// A `DynamicProperty` rather than a `View` because two of the three call sites -/// need the value inside a larger sentence, not a `Text` of its own. -@propertyWrapper -struct BoundTriggerKey: DynamicProperty { - @AppStorage(TriggerKeyStore.defaultsKey) private var keyCode = 0 - - var wrappedValue: TriggerKey { TriggerKey.fromPersisted(keyCode) } -} diff --git a/App/Blurt/Blurt/Wizard/DeveloperSection.swift b/App/Blurt/Blurt/Wizard/DeveloperSection.swift index 34c3283b..948083a5 100644 --- a/App/Blurt/Blurt/Wizard/DeveloperSection.swift +++ b/App/Blurt/Blurt/Wizard/DeveloperSection.swift @@ -3,8 +3,8 @@ import BlurtEngine import OSLog import SwiftUI -// The Advanced pane's four sections: updates (version, check, Sparkle's two -// preferences), experimental features, developer mode, and the start-over button. All are +// The Advanced pane's three sections: updates (version, check, Sparkle's two +// preferences), developer mode, and the start-over button. All are // Settings-only (none gates setup, so none is a wizard step), and they live here // rather than in `SettingsWindowRoot` because that file is at the repo's // file-length limit. @@ -24,20 +24,24 @@ struct UpdatesSection: View { Section { // Ellipsis: it opens Sparkle's window, where an update still waits on the // user's go-ahead — the same rule as "Reset…" below. - SettingRow(title: model.versionLabel, systemImage: "info.circle") { + SettingRow(title: "Version", systemImage: "info.circle", subtitle: model.versionLabel) { Button("Check for Updates…") { model.checkForUpdates() } .disabled(!model.canCheckForUpdates) .accessibilityIdentifier(UITestIdentifiers.updateCheck) } Toggle(isOn: $model.automaticallyChecksForUpdates) { - SettingLabel(title: "Check for updates automatically", systemImage: "clock") + SettingLabel( + title: "Automatic checks", systemImage: "clock", + subtitle: "Check for new versions in the background.") } .disabled(!model.isEnabled) .accessibilityIdentifier(UITestIdentifiers.updateAutoCheck) // Installing on its own only means anything while Blurt is looking on its // own, so the second switch follows the first. Toggle(isOn: $model.automaticallyDownloadsUpdates) { - SettingLabel(title: "Download and install updates automatically", systemImage: "arrow.down.circle") + SettingLabel( + title: "Automatic installation", systemImage: "arrow.down.circle", + subtitle: "Download and install new versions automatically.") } .disabled(!model.isEnabled || !model.automaticallyChecksForUpdates) } header: { @@ -50,89 +54,6 @@ struct UpdatesSection: View { } } -/// The Advanced pane's experimental features — read-aloud (see -/// `SelectionSpeechStore`), asking about the selection -/// (`SelectionAskStore`) and how a read sounds (`ReadAloudStyleStore`). The -/// router reads the same defaults these controls write at every press, so a -/// change applies to the next one. -struct ExperimentalSection: View { - @AppStorage(SelectionSpeechStore.defaultsKey) private var selectionSpeech = false - @AppStorage(SelectionAskStore.defaultsKey) private var selectionAsk = SelectionAskStore.defaultValue - @AppStorage(ReadAloudStyleStore.speedDefaultsKey) private var speed = ReadAloudStyleStore.defaultSpeed - @AppStorage(ReadAloudStyleStore.skipsJargonDefaultsKey) - private var skipsJargon = ReadAloudStyleStore.defaultSkipsJargon - @BoundTriggerKey private var triggerKey - @AppStorage(TriggerActivationStore.defaultsKey) private var activationRaw = "" - - /// Read-aloud rides the trigger's own tap/hold gate, so its instructions - /// follow the activation mode. With asking on, a hold asks instead of reading, - /// so reading is a tap's job, or, where every press is a hold, a hold with - /// nothing said. - private var howToUse: String { - let activation = TriggerActivation.fromPersisted(activationRaw) - let key = triggerKey.label - guard selectionAsk else { - return "Select text and \(activation.startVerb) \(key) to hear it. \(activation.stopHint)" - } - switch activation { - case .hold: return "Select text and hold \(key) without speaking to hear it. Press again to stop." - case .tapOrHold, .tap: return "Select text and tap \(key) to hear it. Tap again to stop." - } - } - - /// The picker reads the stored speed through the same snap the press does, so - /// a `defaults write` value off the list shows as the speed that will play. - private var speedSelection: Binding { - Binding(get: { ReadAloudStyleStore.nearestChoice(to: speed) }, set: { speed = $0 }) - } - - var body: some View { - Section { - Toggle(isOn: $selectionSpeech) { - SettingLabel(title: "Read selected text aloud", systemImage: "speaker.wave.2") - } - .accessibilityIdentifier(UITestIdentifiers.selectionSpeechToggle) - // These rows only change reads, so they only appear while there are - // reads to change. - if selectionSpeech { - Toggle(isOn: $selectionAsk) { - SettingLabel(title: "Hold to ask about the selection", systemImage: "questionmark.bubble") - } - .accessibilityIdentifier(UITestIdentifiers.selectionAskToggle) - PickerSettingRow( - title: "Speed", systemImage: "hare", - accessibilityID: UITestIdentifiers.readAloudSpeedPicker, selection: speedSelection - ) { - ForEach(ReadAloudStyleStore.speedChoices, id: \.self) { choice in - Text(ReadAloudStyleStore.speedLabel(choice)).tag(choice) - } - } - Toggle(isOn: $skipsJargon) { - SettingLabel(title: "Skip code, links, and long numbers", systemImage: "text.badge.minus") - } - .accessibilityIdentifier(UITestIdentifiers.readAloudSkipsJargonToggle) - } - } header: { - Text("Experimental") - } footer: { - Text("\(howToUse) With nothing selected, the key dictates as usual.") - // Each note is its own paragraph. Asking says where the selection and the - // request go, since that is a second service seeing both. - if selectionSpeech, selectionAsk { - Text( - "Hold \(triggerKey.label) over a selection and say what you want, like “summarize this,” to hear " - + "the answer. Asking sends the selection and what you said to AssemblyAI’s LLM Gateway.") - } - // While skipping is on, the selection goes to the gateway before it's read. - if selectionSpeech, skipsJargon { - Text( - "To skip code, links, and long numbers, Blurt sends the selection to AssemblyAI’s LLM Gateway " - + "before reading it.") - } - } - } -} - /// The Advanced pane's developer-mode switch. /// /// Developer mode is an opt-in: while on, every completed dictation is appended @@ -146,7 +67,9 @@ struct MaintenanceSection: View { var body: some View { Section { Toggle(isOn: $developerMode) { - SettingLabel(title: "Developer mode", systemImage: "hammer") + SettingLabel( + title: "Developer mode", systemImage: "hammer", + subtitle: "Keep local logs of dictations and failures.") } .accessibilityIdentifier(UITestIdentifiers.developerToggle) if developerMode { @@ -167,8 +90,6 @@ struct MaintenanceSection: View { + "and each failure to \(DictationLog.defaultErrorDisplayPath)" ) .textSelection(.enabled) - } else { - Text("Developer mode keeps a local log of each dictation and failure.") } } } @@ -244,7 +165,10 @@ struct ResetSection: View { Section { // Ellipsis for the same reason as "Connect…" and "Add Style…": the button // opens something rather than completing the action. - SettingRow(title: "Reset Blurt", systemImage: "arrow.counterclockwise") { + SettingRow( + title: "Reset Blurt", systemImage: "arrow.counterclockwise", + subtitle: "Remove app data and return to setup." + ) { // Red text, because `role: .destructive` alone draws a bordered // button no differently from "Check for Updates…" in the sections // above — and this one deletes the key, every setting and the logs. diff --git a/App/Blurt/Blurt/Wizard/KeyStylesRow.swift b/App/Blurt/Blurt/Wizard/KeyStylesRow.swift deleted file mode 100644 index ad6d2733..00000000 --- a/App/Blurt/Blurt/Wizard/KeyStylesRow.swift +++ /dev/null @@ -1,181 +0,0 @@ -import BlurtEngine -import SwiftUI - -/// The Key Styles row in a section of its own, under the styles it picks from. -/// Not a row of Custom Styles: that section is disabled while enhanced -/// transcripts are off, and turning a key on or off still works then (the -/// sheet disables only the styles). -struct KeyStylesSection: View { - let coordinator: AppCoordinator - - var body: some View { - Section { - KeyStylesRow(coordinator: coordinator) - } footer: { - Text("Use a different style with each key, without switching styles in the main window.") - } - } -} - -/// The Key Styles row: a status and an "Edit…" button, like the API key row, -/// whose sheet gives each trigger key its own style (or Off, for keys other -/// than the bound one). On the Styles pane, under the styles it picks from, -/// rather than beside the dictation key on General, which has to stay short -/// enough for a small display. Settings-only, so it is not a wizard step. -struct KeyStylesRow: View { - var coordinator: AppCoordinator - - @BoundTriggerKey private var bound - - // Observed so the summary follows a change, and decoded through the stores - // that own both encodings. - @AppStorage(TriggerKeyStyleStore.defaultsKey) private var keyStylesRaw = "" - @AppStorage(StyleProfileStore.defaultsKey) private var profilesRaw = "" - - @State private var isPresentingKeyStyles = false - - /// One "right ⌥: Casual" entry per trigger key, named as the sheet names it, - /// and whether that key is moved off its default (Off, or the main window's - /// style for the bound key). All of them are the Edit… button's spoken - /// summary; the changed ones are the row's status. - private static func keyStyleEntries( - bound: TriggerKey, styles: [TriggerKey: String], profiles: [StyleProfile] - ) -> [(text: String, isChanged: Bool)] { - TriggerKey.allCases.map { key in - let choice = TriggerKeyStyleStore.choice(for: key, bound: bound, styles: styles, profiles: profiles) - return ( - text: "\(key.label): \(KeyStylesSheet.name(of: choice, profiles: profiles))", - isChanged: choice != TriggerKeyStyleStore.defaultChoice(for: key, bound: bound) - ) - } - } - - var body: some View { - let entries = Self.keyStyleEntries( - bound: bound, styles: TriggerKeyStyleStore.styles(decoding: keyStylesRaw), - profiles: StyleProfileStore().profiles(decoding: profilesRaw)) - let changed = entries.filter(\.isChanged).map(\.text) - SettingRow(title: "Key Styles", systemImage: "keyboard") { - HStack(spacing: 8) { - // The keys set up beyond the default, or the main window's style. - Text(changed.isEmpty ? KeyStylesSheet.mainWindowName : changed.joined(separator: ", ")) - .foregroundStyle(.secondary) - .lineLimit(1) - .truncationMode(.tail) - Button("Edit…") { isPresentingKeyStyles = true } - .accessibilityValue(entries.map(\.text).joined(separator: ", ")) - .accessibilityIdentifier(UITestIdentifiers.keyStylesButton) - } - } - .sheet(isPresented: $isPresentingKeyStyles) { - KeyStylesSheet(bound: bound) { coordinator.dictationBindingChanged() } - } - } -} - -/// Which style each trigger key dictates with, as a sheet from the Key Styles -/// row: one pop-up per key offering Off (except for the bound key, which -/// always dictates), the main window's style, Default, or a profile. Follows -/// `StyleProfileEditorSheet`'s shape — headline, explanation, content, and the -/// default action at the trailing edge — with only Done, since each choice is -/// written as it is made. While enhanced transcripts are off the styles are -/// disabled, as on the Styles pane; turning a key on or off still works. -private struct KeyStylesSheet: View { - let bound: TriggerKey - /// A key was turned on or off, so the tap must rebind (see `setStyle`). - let onEnabledKeysChanged: () -> Void - - @Environment(\.dismiss) private var dismiss - @AppStorage(TriggerKeyStyleStore.defaultsKey) private var keyStylesRaw = "" - @AppStorage(StyleProfileStore.defaultsKey) private var profilesRaw = "" - @AppStorage(EnhancedTranscriptsStore.defaultsKey) - private var enhancedTranscripts = EnhancedTranscriptsStore.defaultValue - - static let offName = "Off" - static let mainWindowName = "Same as main window" - - /// A choice's display name: Off, the main window's style, Default, or a - /// profile — `choice` is one `TriggerKeyStyleStore.choice` has normalized. - static func name(of choice: String, profiles: [StyleProfile]) -> String { - switch choice { - case TriggerKeyStyleStore.offChoice: offName - case TriggerKeyStyleStore.currentStyleID: mainWindowName - default: StyleProfileStore.active(in: profiles, id: choice)?.name ?? StyleProfileStore.defaultStyleName - } - } - - var body: some View { - let profiles = StyleProfileStore().profiles(decoding: profilesRaw) - let styles = TriggerKeyStyleStore.styles(decoding: keyStylesRaw) - VStack(alignment: .leading, spacing: 16) { - header - Form { - Section { - ForEach(TriggerKey.allCases, id: \.self) { key in - PickerSettingRow( - title: key.label, systemImage: "keyboard", - accessibilityID: UITestIdentifiers.keyStylePicker(key.keyCode), - selection: choice(for: key, styles: styles, profiles: profiles) - ) { - if key != bound { Text(Self.offName).tag(TriggerKeyStyleStore.offChoice) } - Text(Self.mainWindowName).tag(TriggerKeyStyleStore.currentStyleID) - Group { - Text(StyleProfileStore.defaultStyleName).tag(StyleProfileStore.defaultStyleID) - ForEach(profiles) { profile in - Text(profile.name).tag(profile.id.uuidString) - } - } - .disabled(!enhancedTranscripts) - } - } - } footer: { - if !enhancedTranscripts { - Text(StylesInertNotice.settingsText) - } - } - } - .formStyle(.grouped) - .scrollDisabled(true) - .fixedSize(horizontal: false, vertical: true) - HStack { - Spacer() - Button("Done") { dismiss() } - .glassButtonStyleCompat(prominent: true) - .keyboardShortcut(.defaultAction) - .accessibilityIdentifier(UITestIdentifiers.keyStylesDone) - } - // Escape closes it too. The other sheets get that from their Cancel button; - // every choice here is already saved, so the same shortcut rides a hidden one. - .background { - Button("Close") { dismiss() } - .keyboardShortcut(.cancelAction) - .hidden() - } - } - .padding(20) - .frame(width: 420) - } - - private var header: some View { - VStack(alignment: .leading, spacing: 6) { - Text("Key Styles") - .font(.headline) - Text("Pick a style for each key. Turn another key on to dictate with it too.") - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - } - - /// One key's pop-up: `offChoice` is Off, otherwise a style id (see `TriggerKeyStyleStore`). - private func choice( - for key: TriggerKey, styles: [TriggerKey: String], profiles: [StyleProfile] - ) -> Binding { - Binding( - get: { TriggerKeyStyleStore.choice(for: key, bound: bound, styles: styles, profiles: profiles) }, - set: { newValue in - // Through the store, which owns the encoding (see `HotkeyStepView.selection`). - // Only a key turned on or off rebinds the tap; a style is read at the next press. - if TriggerKeyStyleStore().setStyle(newValue, for: key) { onEnabledKeysChanged() } - }) - } -} diff --git a/App/Blurt/Blurt/Wizard/KeysSettingsTab.swift b/App/Blurt/Blurt/Wizard/KeysSettingsTab.swift new file mode 100644 index 00000000..0f5d0d86 --- /dev/null +++ b/App/Blurt/Blurt/Wizard/KeysSettingsTab.swift @@ -0,0 +1,160 @@ +import BlurtEngine +import SwiftUI + +/// The Keys tab: per trigger key, an action pop-up — Off, dictate (with the +/// main window's style, no custom style, or a profile), read the selection +/// aloud, or ask about it — and whether it's tapped, held, or either. A pane +/// of its own after General, so every key's choice is in view and General +/// stays short enough for a small display. Changes are persisted and pushed to +/// the event tap immediately. How read-aloud sounds is its own tab +/// (`ReadAloudSettingsTab`). +/// +/// Reads go through `TriggerKeyBindings` (one decode per render) and writes +/// through the stores, never the raw `@AppStorage` slots: the stores own how a +/// choice and a mode are encoded, and the bound-key rule — taking the only +/// dictating key off dictation is refused, and taking the bound key off it +/// moves the binding first — lives in `TriggerKeyStyleStore.setChoice`, where +/// it is tested. Assigning a raw slot directly would leave the store's write +/// with no production caller, so an encoding change would keep `swift test` +/// green while this view kept writing the old form. +struct KeysSettingsTab: View { + let coordinator: AppCoordinator + + @ObservedKeyBindings private var bindings + @AppStorage(StyleProfileStore.defaultsKey) private var profilesRaw = "" + @AppStorage(EnhancedTranscriptsStore.defaultsKey) + private var enhancedTranscripts = EnhancedTranscriptsStore.defaultValue + + private static let keyboardSettingsURL = URL( + string: "x-apple.systempreferences:com.apple.Keyboard-Settings.extension")! + + /// Shared widths keep the column headings aligned with the native pickers. + private static let actionWidth: CGFloat = 150 + private static let activationWidth: CGFloat = 108 + + /// Each row's icon: the key's own symbol. + static func symbol(for key: TriggerKey) -> String { + switch key { + case .rightCommand, .leftCommand: "command" + case .rightOption, .leftOption: "option" + case .leftControl: "control" + case .function: "globe" + } + } + + var body: some View { + let bindings = bindings + let profiles = StyleProfileStore().profiles(decoding: profilesRaw) + let dictating = bindings.dictatingKeys + SettingsPane { + Section { + ForEach(TriggerKey.allCases, id: \.self) { key in + row(for: key, bindings: bindings, profiles: profiles, isOnlyDictatingKey: bindings.isOnlyDictatingKey(key)) + } + } header: { + HStack(spacing: 8) { + Text("Key") + Spacer(minLength: 12) + Text("Action") + .frame(width: Self.actionWidth, alignment: .trailing) + Text("Activation") + .frame(width: Self.activationWidth, alignment: .trailing) + } + } footer: { + Text("Dictate uses your current writing style. Choose No custom style to use only Blurt’s standard cleanup.") + // The only dictating key's other choices are disabled; say why. + if let only = dictating.first, dictating.count == 1 { + Text("To reassign \(only.spokenName), set another key to Dictate first.") + } + if !enhancedTranscripts { + Text(StylesInertNotice.settingsText) + } + // `GlobeKeyOverride` turns off macOS's own 🌐 action while fn does + // anything, but that write only lands at the next log-in, so until then + // fn double-fires. Lead with that symptom rather than the setting's name, + // and link to the pane that applies the change now. + if bindings.enabledKeys.contains(.function) { + Text( + "Until you next log in, pressing fn may also open the emoji picker or Apple " + + "Dictation. To fix that now, open Keyboard Settings and set “Press 🌐 key to” " + + "to Do Nothing.") + Link("Open Keyboard Settings…", destination: Self.keyboardSettingsURL) + } + } + } + } + + private func row( + for key: TriggerKey, bindings: TriggerKeyBindings, profiles: [StyleProfile], isOnlyDictatingKey: Bool + ) -> some View { + let action = bindings.action(for: key) + return SettingRow(title: key.spokenName, systemImage: Self.symbol(for: key)) { + HStack(spacing: 8) { + // Every item names its action on its own, since a closed pop-up shows + // only the one chosen, without the menu's dividers around it. The + // hidden titles are what VoiceOver reads for each pop-up. The only + // dictating key offers only dictation — left out rather than disabled, + // since a menu pop-up doesn't reliably disable single items; the + // footer says why. + Picker("Action for \(key.fullName)", selection: choice(for: key, bindings: bindings, profiles: profiles)) { + if !isOnlyDictatingKey { + Text("Off").tag(TriggerKeyChoice.off) + Divider() + } + Text("Dictate").tag(TriggerKeyChoice.dictate) + Group { + Text("Dictate: No custom style").tag(TriggerKeyChoice.dictate(style: StyleProfileStore.defaultStyleID)) + ForEach(profiles) { profile in + Text("Dictate: \(profile.name)").tag(TriggerKeyChoice.dictate(style: profile.id.uuidString)) + } + } + .disabled(!enhancedTranscripts) + if !isOnlyDictatingKey { + Divider() + Text(KeyActionName.readAloud).tag(TriggerKeyChoice.readAloud) + Text(KeyActionName.ask).tag(TriggerKeyChoice.ask) + } + } + .labelsHidden() + .pickerStyle(.menu) + .lineLimit(1) + .frame(width: Self.actionWidth, alignment: .trailing) + .accessibilityIdentifier(UITestIdentifiers.keyStylePicker(key.keyCode)) + Picker("Tap or hold for \(key.fullName)", selection: activation(for: key, bindings: bindings)) { + ForEach(TriggerActivation.allCases, id: \.self) { mode in + Text(mode.label).tag(mode) + } + } + .labelsHidden() + .pickerStyle(.menu) + .frame(width: Self.activationWidth, alignment: .trailing) + .disabled(action == .off) + .accessibilityIdentifier(UITestIdentifiers.keyActivationPicker(key.keyCode)) + } + } + } + + /// One key's action pop-up. + private func choice( + for key: TriggerKey, bindings: TriggerKeyBindings, profiles: [StyleProfile] + ) -> Binding { + Binding( + get: { bindings.choice(for: key, profiles: profiles) }, + set: { newValue in + // Refused only for the only dictating key, whose other choices the + // pop-up disables. + guard TriggerKeyStyleStore().setChoice(newValue, for: key) else { return } + coordinator.dictationBindingChanged() + }) + } + + /// One key's tap/hold pop-up. + private func activation(for key: TriggerKey, bindings: TriggerKeyBindings) -> Binding { + Binding( + get: { bindings.activation(for: key) }, + set: { newValue in + TriggerKeyActivationStore().setActivation(newValue, for: key) + coordinator.dictationBindingChanged() + }) + } +} diff --git a/App/Blurt/Blurt/Wizard/ObservedKeyBindings.swift b/App/Blurt/Blurt/Wizard/ObservedKeyBindings.swift new file mode 100644 index 00000000..d2c794a1 --- /dev/null +++ b/App/Blurt/Blurt/Wizard/ObservedKeyBindings.swift @@ -0,0 +1,24 @@ +import BlurtEngine +import SwiftUI + +/// Live-updating read of every trigger key's setting (`TriggerKeyBindings`), +/// for the views that show them: the Keys tab, the ready screen, the menu bar, +/// the permissions page and the Read Aloud tab. +/// +/// Observes the four raw slots the bindings are built from through +/// `@AppStorage`, rather than reading `TriggerKeyBindings()` once, so a view +/// re-renders when the separate Settings window changes any of them. Each slot's unset default +/// (0, "") is "nothing persisted": the defaults belong to `TriggerKeyBindings`, +/// not to literals here. +@propertyWrapper +struct ObservedKeyBindings: DynamicProperty { + @AppStorage(TriggerKeyStore.defaultsKey) private var boundKeyCode = 0 + @AppStorage(TriggerKeyStyleStore.defaultsKey) private var choicesRaw = "" + @AppStorage(TriggerKeyActivationStore.defaultsKey) private var modesRaw = "" + @AppStorage(TriggerActivationStore.defaultsKey) private var sharedModeRaw = "" + + var wrappedValue: TriggerKeyBindings { + TriggerKeyBindings( + boundKeyCode: boundKeyCode, choicesRaw: choicesRaw, modesRaw: modesRaw, sharedModeRaw: sharedModeRaw) + } +} diff --git a/App/Blurt/Blurt/Wizard/ReadAloudSettingsTab.swift b/App/Blurt/Blurt/Wizard/ReadAloudSettingsTab.swift new file mode 100644 index 00000000..fa519250 --- /dev/null +++ b/App/Blurt/Blurt/Wizard/ReadAloudSettingsTab.swift @@ -0,0 +1,115 @@ +import BlurtEngine +import SwiftUI + +/// The Read Aloud tab: its own pane, before Styles, rather than part of the +/// Keys tab — which keys read is a key's business, how a read sounds isn't. +struct ReadAloudSettingsTab: View { + let coordinator: AppCoordinator + + var body: some View { + SettingsPane { + ReadAloudSection(coordinator: coordinator) + } + } +} + +/// How read-aloud sounds — voice, speed, and skipping jargon +/// (`ReadAloudStyleStore`) — for every key that reads aloud or answers. The +/// Read Aloud tab's one section. The +/// router reads the same defaults at each press, so a change applies to the +/// next one. +struct ReadAloudSection: View { + var coordinator: AppCoordinator + + // Observed so the section follows the Keys tab: with no key reading aloud + // or asking, these settings do nothing, and asking adds its own note. + @ObservedKeyBindings private var bindings + + @AppStorage(ReadAloudStyleStore.speedDefaultsKey) private var speed = ReadAloudStyleStore.defaultSpeed + @AppStorage(ReadAloudStyleStore.voiceDefaultsKey) private var voiceID = ReadAloudStyleStore.defaultVoiceID + @AppStorage(ReadAloudStyleStore.skipsJargonDefaultsKey) + private var skipsJargon = ReadAloudStyleStore.defaultSkipsJargon + + /// The picker reads the stored speed through the same snap the press does, so + /// a `defaults write` value off the list shows as the speed that will play. + /// Picking a speed plays the sample at it, as picking a voice does. + private var speedSelection: Binding { + Binding( + get: { ReadAloudStyleStore.nearestChoice(to: speed) }, + set: { + speed = $0 + coordinator.readAloudStyleChanged() + }) + } + + /// Same idea for the voice: a stored id the catalog doesn't list shows as the + /// default voice, which is what the press will speak with. Picking one plays + /// a sample, as the cue-sound picker does. + private var voiceSelection: Binding { + Binding( + get: { ReadAloudStyleStore.knownVoiceID(voiceID) }, + set: { + voiceID = $0 + coordinator.readAloudStyleChanged() + }) + } + + var body: some View { + let bindings = bindings + let readsOrAsks = bindings.readsOrAsks + let asks = bindings.asks + Section { + Group { + PickerSettingRow( + title: "Voice", systemImage: "person.wave.2", + accessibilityID: UITestIdentifiers.readAloudVoicePicker, + subtitle: "Choose a voice to hear a preview.", selection: voiceSelection + ) { + ForEach(ReadAloudVoice.Group.allCases, id: \.self) { group in + Section(group.rawValue) { + ForEach(ReadAloudVoice.catalog.filter { $0.group == group }) { voice in + Text(voice.label).tag(voice.id) + } + } + } + } + PickerSettingRow( + title: "Speed", systemImage: "hare", + accessibilityID: UITestIdentifiers.readAloudSpeedPicker, + subtitle: "How quickly text is read.", selection: speedSelection + ) { + ForEach(ReadAloudStyleStore.speedChoices, id: \.self) { choice in + Text(ReadAloudStyleStore.speedLabel(choice)).tag(choice) + } + } + Toggle(isOn: $skipsJargon) { + SettingLabel( + title: "Skip technical text", systemImage: "text.badge.minus", + subtitle: "Omits code, links, and long numbers.") + } + .accessibilityIdentifier(UITestIdentifiers.readAloudSkipsJargonToggle) + } + .disabled(!readsOrAsks) + } header: { + Text("Playback") + } footer: { + // Each note is its own paragraph, and each says where the text goes: reading + // aloud is on by default (left ⌥), so the first one always shows. + if TriggerKey.allCases.contains(where: { bindings.action(for: $0) == .readAloud }) { + Text("Select text, then use a key assigned to \(KeyActionName.readAloud).") + } else if !readsOrAsks { + Text("Assign a \(KeyActionName.readAloud) or \(KeyActionName.ask) key in Keys to enable playback.") + } + if asks { + Text("Select text, then use an \(KeyActionName.ask) key and speak your request.") + Text("Your selection and spoken request are sent to AssemblyAI’s LLM Gateway to generate an answer.") + } + if readsOrAsks { + Text("Selected text is sent to AssemblyAI to read aloud.") + if skipsJargon { + Text("Skip technical text also sends your selection to AssemblyAI’s LLM Gateway before playback.") + } + } + } + } +} diff --git a/App/Blurt/Blurt/Wizard/ReadyView.swift b/App/Blurt/Blurt/Wizard/ReadyView.swift index 444b915a..9ee26b34 100644 --- a/App/Blurt/Blurt/Wizard/ReadyView.swift +++ b/App/Blurt/Blurt/Wizard/ReadyView.swift @@ -2,8 +2,8 @@ import BlurtEngine import SwiftUI /// The "you're all set" screen shown in the main window once setup is complete: -/// the wordmark over a grouped form — the shortcut and the style in effect, -/// then the recent dictations — and a bottom bar with the attribution links +/// the wordmark over a grouped form — the keys you've configured and the +/// style in effect, then the recent dictations — and a bottom bar with the attribution links /// and a Settings button, the window's own door to the Settings scene /// alongside the app-menu "Settings…" (⌘,) and the menu-bar item. /// @@ -15,14 +15,13 @@ import SwiftUI struct ReadyView: View { var coordinator: AppCoordinator var openSettings: () -> Void - // Observed (not read once) so changing the dictation key in the separate - // Settings window re-renders this window's keycap live — see `BoundTriggerKey`. - @BoundTriggerKey private var triggerKey - /// Tap or hold, tap, or hold — observed like the key, so the shortcut row's - /// words follow a change made in Settings. Raw slot, decoded through - /// `fromPersisted`, which owns the unset default. - @AppStorage(TriggerActivationStore.defaultsKey) private var activationRaw = "" - private var activation: TriggerActivation { .fromPersisted(activationRaw) } + // Observed (not read once) so a change in the separate Settings window — + // the dictation key, any key's action or mode — re-renders this window's + // rows live (`ObservedKeyBindings`). + @ObservedKeyBindings private var bindings + private var triggerKey: TriggerKey { bindings.bound } + /// The dictation key's tap/hold mode, which the shortcut row's words follow. + private var activation: TriggerActivation { bindings.activation(for: bindings.bound) } /// Kept as view state because Settings is a separate scene: its defaults /// write can otherwise leave this already-open window displaying the old @@ -41,13 +40,16 @@ struct ReadyView: View { // second reading here. let active = StyleProfileStore.active(in: profiles, id: rawActiveID) VStack(spacing: 0) { - ReadyBrandingView() - .padding(.top, 24) - .padding(.bottom, 2) + MainWindowBranding() + .padding(.top, 16) Form { Section { shortcutRow + // Every other key given an action in the Keys tab, in its order. + ForEach(bindings.extraKeys, id: \.self) { key in + extraKeyRow(key, profiles: profiles) + } // Always present, even with no custom styles, so the active treatment // remains visible from the main window. StyleRow(profiles: profiles, activeID: active?.id) @@ -90,24 +92,26 @@ struct ReadyView: View { /// the bound key as a keycap beside a word on what it does. The keycap fills /// with the accent the moment the key goes down (`isCapturing`, which /// includes the mic bring-up) — the window's proof the hotkey is registered — - /// while the words swap from the configured gesture ("Tap or hold", "Tap", - /// or "Hold" — `TriggerActivation.label`) to the live phase. + /// with the gesture guidance beneath the title and the live phase beside + /// the key during capture and transcription. /// /// The phase is gated on the same stream the overlay pill renders /// (`coordinator.menuBarStatus`, whose `.recording` deliberately excludes /// the mic bring-up) — during "Connecting…" nothing is captured yet, so /// claiming "Listening" would invite unrecoverable speech. private var shortcutRow: some View { - SettingRow(title: "Shortcut", systemImage: "keyboard") { + SettingRow(title: "Dictate", systemImage: "keyboard", subtitle: activation.hint(for: .dictate)) { HStack(spacing: 8) { phaseText .foregroundStyle(.secondary) KeyCap(label: triggerKey.keycapLabel, isLit: coordinator.isCapturing) + .frame(minWidth: 64, alignment: .trailing) } } - .help(activation.guidance) + .padding(.vertical, 3) + .help(activation.hint(for: .dictate)) .accessibilityElement(children: .ignore) - .accessibilityLabel("Shortcut: \(triggerKey.spokenName). \(phaseDescription)") + .accessibilityLabel("Dictate: \(triggerKey.spokenName). \(phaseDescription)") // Esc cancels the in-flight dictation — same command the overlay's owner // submits (`.cancel`), scoped to this window by living on a button in it. // Present through all of capture (not just `.recording`), so Esc during @@ -121,8 +125,35 @@ struct ReadyView: View { } } - /// The shortcut row's trailing words: how to use the key at rest, then what - /// the dictation is doing. + /// One extra key's row: what it does leading — with the style a dictating + /// key uses when it isn't the main window's — and trailing its tap/hold mode + /// beside its keycap. No live phase: that stays on the dictation row above, + /// whichever key started the dictation. + private func extraKeyRow(_ key: TriggerKey, profiles: [StyleProfile]) -> some View { + let choice = bindings.choice(for: key, profiles: profiles) + let (title, symbol): (String, String) = + switch choice { + case .readAloud: (KeyActionName.readAloud, "speaker.wave.2") + case .ask: (KeyActionName.ask, "questionmark.bubble") + case .dictate(TriggerKeyChoice.mainWindowStyle), .off: ("Dictate", "keyboard") + case .dictate(let style): + ( + "Dictate: \(StyleProfileStore.active(in: profiles, id: style)?.name ?? StyleProfileStore.defaultStyleName)", + "keyboard" + ) + } + let hint = bindings.activation(for: key).hint(for: choice.action) + return SettingRow(title: title, systemImage: symbol, subtitle: hint) { + KeyCap(label: key.keycapLabel) + .frame(minWidth: 64, alignment: .trailing) + } + .padding(.vertical, 3) + .help(hint) + .accessibilityElement(children: .ignore) + .accessibilityLabel("\(title): \(key.spokenName). \(hint)") + } + + /// The shortcut row's trailing status, shown only during an active dictation. @ViewBuilder private var phaseText: some View { switch coordinator.menuBarStatus { @@ -138,7 +169,7 @@ struct ReadyView: View { case .transcribing: Text("Transcribing…") case .idle: - Text(activation.label) + EmptyView() } } @@ -148,7 +179,7 @@ struct ReadyView: View { switch coordinator.menuBarStatus { case .recording: "Listening. \(activation.finishHint) Escape cancels." case .transcribing: "Transcribing." - case .idle: activation.guidance + case .idle: activation.hint(for: .dictate) } } @@ -157,13 +188,14 @@ struct ReadyView: View { /// A standard push button — Settings is a place visited once, and ⌘, and the /// app menu already reach it, so it takes neither the accent nor glass. private var bottomBar: some View { - HStack { + HStack(alignment: .firstTextBaseline) { footerLinks Spacer() Button(action: openSettings) { Label("Settings…", systemImage: "gearshape") .labelStyle(.titleAndIcon) } + .controlSize(.small) } } @@ -193,6 +225,7 @@ struct ReadyView: View { } } .font(.caption) + .tint(.secondary) } } @@ -217,14 +250,14 @@ enum BlurtLinks { /// The `blurt` wordmark over the form: the brand-green mark /// (`Design/brand/blurt-ready-logo.png`, a 720×180 rasterization of the design's -/// vector wordmark, so its 104×26 pt slot is fed nearly 7× the pixels it needs and +/// vector wordmark, so its 72×18 pt slot is fed nearly 7× the pixels it needs and /// stays crisp at any display scale). Smoothly interpolated — it's curved /// letterforms now, not the pixel-art mark it replaced, which needed /// nearest-neighbor to keep its pixels square. A header mark, not the window's /// identity — the standard titlebar names the app — which is why it simply /// omits itself if the PNG can't load rather than swapping in a fallback /// identity view. -private struct ReadyBrandingView: View { +struct MainWindowBranding: View { /// Loaded once for the process rather than per `body` evaluation: this view /// sits in `ReadyView`, whose body re-runs on every new dictation /// (`recentDictations.entries`), and a bundle lookup plus a PNG decode is not @@ -254,7 +287,7 @@ private struct ReadyBrandingView: View { .interpolation(.high) .resizable() .scaledToFit() - .frame(maxWidth: 104) + .frame(maxWidth: 72) .foregroundStyle(BlurtBrand.accent) .accessibilityLabel("Blurt logo") } diff --git a/App/Blurt/Blurt/Wizard/RecentDictationsSection.swift b/App/Blurt/Blurt/Wizard/RecentDictationsSection.swift index bc407779..7d4f4dc0 100644 --- a/App/Blurt/Blurt/Wizard/RecentDictationsSection.swift +++ b/App/Blurt/Blurt/Wizard/RecentDictationsSection.swift @@ -22,7 +22,7 @@ struct RecentDictationsSection: View { /// Each transcript gets a comfortable pointer target even though its text /// stays one line. Three of these slots preserve the section's height. - private static let rowHeight: CGFloat = 36 + private static let rowHeight: CGFloat = 28 /// How often the relative timestamps re-render. Half the engine's "just now" /// window, so a row can't read as stale for longer than that window lasts — @@ -61,20 +61,21 @@ struct RecentDictationsSection: View { /// only says nothing has happened yet: it names the one thing to try, in the /// order the user does it. private var emptyPrompt: some View { - VStack(spacing: 7) { - Text("Start your first dictation") + VStack(spacing: 8) { + Text("Ready when you are") .font(.headline) .foregroundStyle(.primary) HStack(spacing: 5) { - Text("Focus a text field, then \(activation.startVerb)") + Text("Click a text field, then \(activation.startVerb)") KeyCap(label: triggerKey.keycapLabel) } + .font(.callout) } .foregroundStyle(.secondary) .fixedSize() .accessibilityElement(children: .ignore) .accessibilityLabel( - "Start your first dictation. Focus a text field, then \(activation.startVerb) \(triggerKey.spokenName)." + "Ready when you are. Click a text field, then \(activation.startVerb) \(triggerKey.spokenName)." ) .accessibilityAddTraits(.isStaticText) .accessibilityIdentifier(UITestIdentifiers.recentEmptyPrompt) diff --git a/App/Blurt/Blurt/Wizard/SettingRow.swift b/App/Blurt/Blurt/Wizard/SettingRow.swift index 29f6b09a..8701d7cf 100644 --- a/App/Blurt/Blurt/Wizard/SettingRow.swift +++ b/App/Blurt/Blurt/Wizard/SettingRow.swift @@ -78,11 +78,12 @@ struct PickerSettingRow: View { var title: String var systemImage: String var accessibilityID: String + var subtitle: String? @Binding var selection: Value @ViewBuilder var options: () -> Options var body: some View { - SettingRow(title: title, systemImage: systemImage) { + SettingRow(title: title, systemImage: systemImage, subtitle: subtitle) { // The picker keeps its own (hidden) title so VoiceOver reads a meaningful // name for the pop-up button rather than an empty string. Picker(title, selection: $selection, content: options) diff --git a/App/Blurt/Blurt/Wizard/SettingsWindowRoot.swift b/App/Blurt/Blurt/Wizard/SettingsWindowRoot.swift index 714e631b..75be0160 100644 --- a/App/Blurt/Blurt/Wizard/SettingsWindowRoot.swift +++ b/App/Blurt/Blurt/Wizard/SettingsWindowRoot.swift @@ -3,7 +3,7 @@ import SwiftUI /// Root view of the `Settings` scene. A `TabView` at the root of a `Settings` /// scene renders as the standard macOS preferences window — a segmented toolbar -/// of panes (General / Styles / Vocabulary / Advanced), each sized to its own content. This is the +/// of panes (General / Keys / Read Aloud / Styles / Vocabulary / Advanced), each sized to its own content. This is the /// HIG-native answer to a settings screen that outgrows one pane: keeping every /// pane short means the window never has to grow past a small display (a single /// stacked `Form` did, stranding the bottom section off-screen). Each pane @@ -20,7 +20,11 @@ struct SettingsWindowRoot: View { TabView { GeneralSettingsTab(coordinator: coordinator) .tabItem { Label(UITestIdentifiers.generalSettingsTab, systemImage: "gearshape") } - StylesSettingsTab(coordinator: coordinator) + KeysSettingsTab(coordinator: coordinator) + .tabItem { Label(UITestIdentifiers.keysSettingsTab, systemImage: "keyboard") } + ReadAloudSettingsTab(coordinator: coordinator) + .tabItem { Label(UITestIdentifiers.readAloudSettingsTab, systemImage: "speaker.wave.2") } + StylesSettingsTab() .tabItem { Label(UITestIdentifiers.stylesSettingsTab, systemImage: "textformat") } VocabularySettingsTab() .tabItem { @@ -50,8 +54,8 @@ struct SettingsPane: View { } } -/// The everyday setup a user changes: the AssemblyAI key, the dictation -/// shortcut, the microphone, and the cue sound. Key terms live on Vocabulary, +/// The everyday setup a user changes: the AssemblyAI account, the microphone, +/// and the cue sounds. The trigger keys are their own tab (`KeysSettingsTab`). Key terms live on Vocabulary, /// beside the text shortcuts — both are "words Blurt should know". private struct GeneralSettingsTab: View { let coordinator: AppCoordinator @@ -59,32 +63,27 @@ private struct GeneralSettingsTab: View { var body: some View { SettingsPane { APIKeyStepView(apiKey: coordinator.apiKey, account: coordinator.account) - HotkeyStepView(coordinator: coordinator) MicrophoneStepView() SoundStepView(coordinator: coordinator) } } } -/// The enhanced-transcripts switch, the user's style profiles, and which style -/// each trigger key dictates with — their own pane rather than an Advanced -/// section. The switch leads because styles only shape the enhanced rewrite: +/// The enhanced-transcripts switch and the user's style profiles — their own +/// pane rather than an Advanced section. The switch leads because styles only shape the enhanced rewrite: /// with it off they're disabled, and the switch that re-enables them sits /// directly above rather than on another pane. private struct StylesSettingsTab: View { - let coordinator: AppCoordinator - var body: some View { SettingsPane { TranscriptionSection() StyleProfilesSection() - KeyStylesSection(coordinator: coordinator) } } } -/// The occasional stuff: checking for an update, experimental features -/// (read-aloud), the developer-mode log toggle, and the +/// The occasional stuff: checking for an update, the developer-mode log +/// toggle, and the /// start-over button. Kept out of General so /// the common pane stays short. private struct AdvancedSettingsTab: View { @@ -94,7 +93,6 @@ private struct AdvancedSettingsTab: View { var body: some View { SettingsPane { UpdatesSection(model: updaterModel) - ExperimentalSection() MaintenanceSection() ResetSection(coordinator: coordinator) } @@ -116,38 +114,39 @@ private struct TranscriptionSection: View { private var enhancedTranscripts = EnhancedTranscriptsStore.defaultValue // "" is "nothing persisted"; the unset default belongs to - // `RewriteMode.fromPersisted`, as `HotkeyStepView`'s activation slot does. + // `RewriteMode.fromPersisted`, as the key slots' defaults belong to `TriggerKeyBindings`. @AppStorage(RewriteModeStore.defaultsKey) private var rewriteModeRaw = "" private var rewriteMode: Binding { Binding( get: { RewriteMode.fromPersisted(rewriteModeRaw) }, - // Through the store, which owns the encoding (see `HotkeyStepView`). + // Through the store, which owns the encoding (see `KeysSettingsTab`). set: { RewriteModeStore().mode = $0 }) } var body: some View { Section { Toggle(isOn: $enhancedTranscripts) { - SettingLabel(title: "Enhanced transcripts", systemImage: "wand.and.stars") + SettingLabel( + title: "Enhanced transcripts", systemImage: "wand.and.stars", + subtitle: "Removes filler words and fixes punctuation.") } .accessibilityIdentifier(UITestIdentifiers.enhancedTranscriptsToggle) // Picks the model for the rewrite the switch above pastes, so it is // disabled with it, as the styles below are. PickerSettingRow( title: "Dictation mode", systemImage: "speedometer", - accessibilityID: UITestIdentifiers.rewriteModePicker, selection: rewriteMode + accessibilityID: UITestIdentifiers.rewriteModePicker, + subtitle: "Fast for speed. Smart for better rewrites.", selection: rewriteMode ) { ForEach(RewriteMode.allCases, id: \.self) { mode in Text(mode.label).tag(mode) } } .disabled(!enhancedTranscripts) + .help(RewriteMode.guidance) } footer: { - Text( - "Polishes each dictation before pasting — removing filler words and fixing punctuation. " - + "Turn off to paste your words exactly as spoken.") - Text(RewriteMode.guidance) + Text("Turn off Enhanced transcripts to paste the transcript without cleanup or custom styling.") } } } @@ -199,20 +198,23 @@ private struct StyleProfilesSection: View { } // Ellipsis for the same reason as the API-key row's "Connect…": the // action needs more input before it completes. - Button("Add Style…") { editing = StyleProfile(name: "", instructions: "") } - .disabled(profiles.count >= StyleProfileStore.limit) - .accessibilityIdentifier(UITestIdentifiers.styleProfileAdd) + SettingRow( + title: profiles.isEmpty ? "Create your first style" : "New style", systemImage: "text.badge.plus", + subtitle: "For example, casual tone and lowercase text." + ) { + Button("Add Style…") { editing = StyleProfile(name: "", instructions: "") } + .disabled(profiles.count >= StyleProfileStore.limit) + .accessibilityIdentifier(UITestIdentifiers.styleProfileAdd) + } } header: { - Text("Custom Styles") + Text("Custom styles") } footer: { // The caveat *replaces* the explanation rather than joining it: with // enhanced transcripts off the rewrite a style shapes is discarded // unread, so pointing at the switch just above is the useful half. Text( enhancedTranscripts - ? "A style tells Blurt how to polish your dictations — for example, “casual, all " - + "lowercase” for chat. Switch between styles in the main window. Create up to " - + "\(StyleProfileStore.limit) styles." + ? "Create up to \(StyleProfileStore.limit) styles for tone and formatting. Choose the active style in the main window." : StylesInertNotice.settingsText) } .disabled(!enhancedTranscripts) @@ -262,9 +264,9 @@ private struct StyleProfileEditorSheet: View { var body: some View { VStack(alignment: .leading, spacing: 16) { VStack(alignment: .leading, spacing: 6) { - Text("Custom Style") + Text(isExisting ? "Edit Style" : "Add Style") .font(.headline) - Text("Tell Blurt how to polish each dictation, including its casing, tone, and emoji use.") + Text("Choose a name and describe the tone and formatting you want.") .foregroundStyle(.secondary) .fixedSize(horizontal: false, vertical: true) } @@ -287,7 +289,7 @@ private struct StyleProfileEditorSheet: View { VStack(alignment: .leading, spacing: 6) { Text("Name") .font(.subheadline.weight(.semibold)) - TextField("", text: $name, prompt: Text("Casual")) + TextField("", text: $name, prompt: Text("e.g. Casual")) .lineLimit(1) .disableAutocorrection(true) .focused($nameFocused) @@ -303,8 +305,7 @@ private struct StyleProfileEditorSheet: View { } } - /// The multi-line instruction field, with the byte counter directly beneath - /// the field it measures. + /// The multi-line instruction field, with its limit usage beneath it. private var instructionsField: some View { VStack(alignment: .leading, spacing: 6) { Text("Instructions") @@ -313,13 +314,13 @@ private struct StyleProfileEditorSheet: View { // there's no faked placeholder over a TextEditor. TextField( text: $instructions, - prompt: Text("Add fitting emojis sparingly, or always write in lowercase"), + prompt: Text("Use a casual tone, write in lowercase, and add emojis sparingly."), axis: .vertical ) { Text("Instructions") } .labelsHidden() - .lineLimit(2...6) + .lineLimit(3...6) .font(.body) .disableAutocorrection(true) .accessibilityIdentifier(UITestIdentifiers.styleProfileInstructions) @@ -333,16 +334,20 @@ private struct StyleProfileEditorSheet: View { // The API caps the instruction, so the room left is finite — show it // rather than truncating silently at the limit. Counted in UTF-8 bytes, // the unit the limit is enforced in. - Text("\(instructions.utf8.count)/\(StyleProfileStore.characterLimit)") + Text("\(instructionUsage)% of limit used") .font(.caption) .monospacedDigit() .foregroundStyle(.secondary) .frame(maxWidth: .infinity, alignment: .trailing) .accessibilityLabel( - "\(instructions.utf8.count) of \(StyleProfileStore.characterLimit) characters used") + "\(instructionUsage) percent of instruction limit used") } } + private var instructionUsage: Int { + instructions.utf8.count * 100 / StyleProfileStore.characterLimit + } + private var buttonRow: some View { HStack(spacing: 12) { if isExisting { diff --git a/App/Blurt/Blurt/Wizard/Steps/AccountRowView.swift b/App/Blurt/Blurt/Wizard/Steps/AccountRowView.swift index fe575125..d3d7cad6 100644 --- a/App/Blurt/Blurt/Wizard/Steps/AccountRowView.swift +++ b/App/Blurt/Blurt/Wizard/Steps/AccountRowView.swift @@ -51,17 +51,9 @@ struct AccountRowView: View { } private var row: some View { - SettingRow(title: "Account", systemImage: "person.crop.circle") { + let status = account.status.statusText ?? "Connect your AssemblyAI account." + return SettingRow(title: "Account", systemImage: "person.crop.circle", subtitle: status) { HStack(spacing: 12) { - if let statusText = account.status.statusText { - Text(statusText) - .foregroundStyle(.secondary) - // A long work address mustn't push Sign Out off the row; the full - // text is on hover. - .lineLimit(1) - .truncationMode(.middle) - .help(statusText) - } if isWorking { ProgressView().controlSize(.small) .accessibilityLabel(account.status.statusText == nil ? "Signing in" : "Signing out") @@ -70,12 +62,13 @@ struct AccountRowView: View { Button("Sign Out", action: signOut) .disabled(isWorking) } else { - Button("Sign In with AssemblyAI…", action: startSignIn) + Button("Sign In…", action: startSignIn) .glassButtonStyleCompat(prominent: isPrimary) .disabled(isWorking) } } } + .help(status) } private func startSignIn() { diff --git a/App/Blurt/Blurt/Wizard/Steps/HotkeyStepView.swift b/App/Blurt/Blurt/Wizard/Steps/HotkeyStepView.swift deleted file mode 100644 index 9763e7c0..00000000 --- a/App/Blurt/Blurt/Wizard/Steps/HotkeyStepView.swift +++ /dev/null @@ -1,102 +0,0 @@ -import BlurtEngine -import SwiftUI - -/// The dictation-key section of the setup/settings screen. Two menu pickers: which -/// lone modifier triggers dictation, and whether it activates by tap (toggle), -/// hold (push-to-talk), or both; changes are persisted and pushed to the event tap -/// immediately. Giving the other trigger keys their own styles is the Styles -/// pane's `KeyStylesRow`. -struct HotkeyStepView: View { - var coordinator: AppCoordinator - - // `0` is "no keycode persisted", not a default binding: the unset default belongs - // to `TriggerKey.fromPersisted` (below), which maps any unknown keycode to right - // ⌘. Restating `TriggerKey.rightCommand.rawValue` here would give the empty slot - // two answers, and this one would win for an unset key — so a change to the - // engine's default would leave this picker showing the old binding while the - // ready screen and menu bar showed the new one. Matches `@BoundTriggerKey`. - @AppStorage(TriggerKeyStore.defaultsKey) private var triggerKeyCode = 0 - - // Same rule for the activation half: "" is "nothing persisted", and the unset - // default belongs to `TriggerActivation.fromPersisted`, not a literal here. - @AppStorage(TriggerActivationStore.defaultsKey) private var activationRaw = "" - - // Observed so the fn footer follows a key turned on in Key Styles, and decoded - // through the store that owns the encoding. - @AppStorage(TriggerKeyStyleStore.defaultsKey) private var keyStylesRaw = "" - - private var selection: Binding { - Binding( - get: { - TriggerKey.fromPersisted(triggerKeyCode) - }, - set: { newValue in - // Write through the store, not the raw `@AppStorage` slot: the store owns - // how a `TriggerKey` is encoded, and `@AppStorage` is here to *observe* the - // key so this view re-renders (it picks up the store's external write). - // Assigning `triggerKeyCode` directly left `TriggerKeyStore`'s setter with - // no production caller, so a change to the encoding — versioning the key, - // storing the case name, a migration — would keep `swift test` green while - // the picker silently kept writing the old form. - TriggerKeyStore().triggerKey = newValue - coordinator.dictationBindingChanged() - }) - } - - private var activation: Binding { - Binding( - get: { - TriggerActivation.fromPersisted(activationRaw) - }, - set: { newValue in - // Write through the store for the same reason as `selection` above. - TriggerActivationStore().activation = newValue - coordinator.dictationBindingChanged() - }) - } - - private static let keyboardSettingsURL = URL( - string: "x-apple.systempreferences:com.apple.Keyboard-Settings.extension")! - - var body: some View { - let bound = selection.wrappedValue - let styles = TriggerKeyStyleStore.styles(decoding: keyStylesRaw) - Section { - PickerSettingRow( - title: "Dictation key", systemImage: "keyboard", - accessibilityID: UITestIdentifiers.hotkeyPicker, selection: selection - ) { - ForEach(TriggerKey.allCases, id: \.self) { key in - Text(key.label).tag(key) - } - } - PickerSettingRow( - title: "Activation", systemImage: "hand.tap", - accessibilityID: UITestIdentifiers.activationPicker, selection: activation - ) { - ForEach(TriggerActivation.allCases, id: \.self) { mode in - Text(mode.label).tag(mode) - } - } - } header: { - Text("Shortcut") - } footer: { - // The engine owns the wording (`TriggerActivation.guidance`), so the hint - // always describes the mode the gate will actually apply. Each variant is - // no longer than the tap-or-hold sentence shipped here before the picker - // existed, so the footer never grows the pane. - Text(TriggerActivation.fromPersisted(activationRaw).guidance) - // `GlobeKeyOverride` turns off macOS's own 🌐 action while fn is bound, but - // that write only lands at the next log-in, so until then fn double-fires. - // Lead with that symptom rather than the setting's name, and link to the - // pane that applies the change now. - if TriggerKeyStyleStore.enabledKeys(bound: bound, styles: styles).contains(.function) { - Text( - "Until you next log in, pressing fn may also open the emoji picker or Apple " - + "Dictation. To fix that now, open Keyboard Settings and set “Press 🌐 key to” " - + "to Do Nothing.") - Link("Open Keyboard Settings…", destination: Self.keyboardSettingsURL) - } - } - } -} diff --git a/App/Blurt/Blurt/Wizard/Steps/KeyTermsStepView.swift b/App/Blurt/Blurt/Wizard/Steps/KeyTermsStepView.swift index d9b65879..f856bd61 100644 --- a/App/Blurt/Blurt/Wizard/Steps/KeyTermsStepView.swift +++ b/App/Blurt/Blurt/Wizard/Steps/KeyTermsStepView.swift @@ -28,10 +28,10 @@ struct KeyTermsStepView: View { // accessibility without repeating the section header inline. TextField( text: $text, - prompt: Text("AssemblyAI, Kubernetes, Anthropic, Blurt"), + prompt: Text("e.g. AssemblyAI, Kubernetes, Blurt"), axis: .vertical ) { - Text("Key Terms") + Text("Key terms") } .labelsHidden() .lineLimit(2...6) @@ -39,9 +39,9 @@ struct KeyTermsStepView: View { .disableAutocorrection(true) .accessibilityIdentifier(UITestIdentifiers.keyTermsField) } header: { - Text("Key Terms") + Text("Key terms") } footer: { - Text("Add names, jargon, and product terms to help Blurt spell them correctly.") + Text("Separate names and specialist terms with commas to help Blurt spell them correctly.") } } } diff --git a/App/Blurt/Blurt/Wizard/Steps/MicrophoneStepView.swift b/App/Blurt/Blurt/Wizard/Steps/MicrophoneStepView.swift index b3b26109..ba208afe 100644 --- a/App/Blurt/Blurt/Wizard/Steps/MicrophoneStepView.swift +++ b/App/Blurt/Blurt/Wizard/Steps/MicrophoneStepView.swift @@ -11,7 +11,7 @@ import SwiftUI struct MicrophoneStepView: View { // Empty means "no device pinned" — the unset default belongs to // `MicDeviceSelection.fromPersisted`, which reads it as "same as system". See - // `HotkeyStepView` for why the view must not restate it. + // `KeysSettingsTab` for why the view must not restate it. @AppStorage(MicDeviceStore.defaultsKey) private var micDeviceUID = "" /// The input devices present when the pane appeared, re-read each time it @@ -27,18 +27,17 @@ struct MicrophoneStepView: View { /// rather than an optional array because SwiftLint's /// `discouraged_optional_collection` (opted into repo-wide) forbids the latter. @State private var devicesLoaded = false - /// The current default input's name, for the "Same as system (…)" label; nil - /// (no device, or the read failed) drops the parenthetical. + /// The current default input's name, shown beneath the row title. @State private var systemDefaultName: String? /// The selection as the engine's own type, not the raw slot. Binding the /// `String` meant spelling ""-means-system a second time (in a `.tag("")`), /// which is exactly what `MicDeviceSelection` exists to own — the same shape - /// `SoundStepView` and `HotkeyStepView` already use for their pickers. + /// `SoundStepView` and `KeysSettingsTab` already use for their pickers. private var selection: Binding { Binding( get: { MicDeviceSelection.fromPersisted(micDeviceUID) }, - // Write through the store (see `HotkeyStepView` for why): the store owns + // Write through the store (see `KeysSettingsTab` for why): the store owns // the encoding, `@AppStorage` observes the key to re-render. set: { MicDeviceStore().selection = $0 }) } @@ -53,17 +52,20 @@ struct MicrophoneStepView: View { return .pinned(uid: micDeviceUID) } - private var systemDefaultLabel: String { - systemDefaultName.map { "Same as system (\($0))" } ?? "Same as system" + private var deviceDescription: String { + if micDeviceUID.isEmpty || missingPin != nil { + return systemDefaultName.map { "Using \($0)." } ?? "Uses your system’s default microphone." + } + return "Used for voice recording." } var body: some View { Section { PickerSettingRow( title: "Input device", systemImage: "mic", - accessibilityID: UITestIdentifiers.micPicker, selection: selection + accessibilityID: UITestIdentifiers.micPicker, subtitle: deviceDescription, selection: selection ) { - Text(systemDefaultLabel).tag(MicDeviceSelection.systemDefault) + Text("Same as system").tag(MicDeviceSelection.systemDefault) ForEach(devices) { device in Text(device.name).tag(MicDeviceSelection.pinned(uid: device.uid)) } @@ -75,7 +77,7 @@ struct MicrophoneStepView: View { Text("Microphone") } footer: { Text( - "Blurt uses this microphone for dictation. If it’s unavailable, Blurt uses the system default." + "If this microphone is unavailable, Blurt uses the system default." ) } // Off the main actor, and `.task` rather than `.onAppear` to have somewhere diff --git a/App/Blurt/Blurt/Wizard/Steps/PermissionsStepView.swift b/App/Blurt/Blurt/Wizard/Steps/PermissionsStepView.swift index 55318498..4036b7a5 100644 --- a/App/Blurt/Blurt/Wizard/Steps/PermissionsStepView.swift +++ b/App/Blurt/Blurt/Wizard/Steps/PermissionsStepView.swift @@ -12,8 +12,9 @@ struct PermissionsStepView: View { /// Observed rather than read once via `TriggerKeyStore()`: Settings is reachable /// with ⌘, while this page is showing, so a one-shot read left the footer naming /// the old key after a rebind until something else re-rendered the view. Same - /// `BoundTriggerKey` as `ReadyView` / `MenuBarScene`. - @BoundTriggerKey private var triggerKey + /// `ObservedKeyBindings` as `ReadyView` / `MenuBarScene`. + @ObservedKeyBindings private var bindings + private var triggerKey: TriggerKey { bindings.bound } /// Set when the user taps a settings button so the section can show a /// "waiting for you to come back" cue until the poll sees the grant. @@ -33,16 +34,18 @@ struct PermissionsStepView: View { private static let stuckGrantHintDelay: Duration = .seconds(12) var body: some View { - Group { + Section("Permissions") { microphoneSection accessibilitySection } } private var microphoneSection: some View { - Section { + VStack(alignment: .leading, spacing: 6) { permissionRow( - PermissionInfo(label: "Microphone", symbol: "mic.fill", buttonLabel: "Allow Microphone Access"), + PermissionInfo( + label: "Microphone", symbol: "mic.fill", buttonLabel: "Allow Access", + detail: "Records when you use \(triggerKey.keycapLabel)."), granted: controller.permissions.microphone, action: { Task { @@ -59,37 +62,31 @@ struct PermissionsStepView: View { } } ) - } header: { - Text("Permissions") - } footer: { settingsFooter( opened: openedMicrophoneSettings, granted: controller.permissions.microphone, - waiting: "Waiting for you to turn on Blurt under Microphone…", - description: - "Blurt records only after you start dictating with \(triggerKey.label)." + waiting: "Waiting for you to turn on Blurt under Microphone…" ) } } private var accessibilitySection: some View { - Section { + VStack(alignment: .leading, spacing: 6) { permissionRow( PermissionInfo( - label: "Accessibility", symbol: "accessibility", buttonLabel: "Open Accessibility Settings…"), + label: "Accessibility", symbol: "accessibility", buttonLabel: "Open Settings…", + detail: "Pastes dictations into the active app."), granted: controller.permissions.accessibility, action: { openedAccessibilitySettings = true PermissionsChecker.openAccessibilitySettings() } ) - } footer: { VStack(alignment: .leading, spacing: 4) { settingsFooter( opened: openedAccessibilitySettings, granted: controller.permissions.accessibility, - waiting: "Waiting for you to turn on Blurt in the Accessibility list…", - description: "Blurt uses Accessibility to paste transcripts into the active app." + waiting: "Waiting for you to turn on Blurt in the Accessibility list…" ) if accessibilityGrantLooksStuck && openedAccessibilitySettings && !controller.permissions.accessibility @@ -120,9 +117,9 @@ struct PermissionsStepView: View { /// Shared footer for a settings-backed permission row: a "waiting for you to /// come back" cue while the grant is outstanding after the user opened - /// Settings, otherwise the explanatory description. + /// Settings. The explanation stays beneath the row title. @ViewBuilder - private func settingsFooter(opened: Bool, granted: Bool, waiting: String, description: String) + private func settingsFooter(opened: Bool, granted: Bool, waiting: String) -> some View { if opened && !granted { @@ -130,8 +127,8 @@ struct PermissionsStepView: View { ProgressView().controlSize(.small) Text(waiting) } - } else { - Text(description) + .font(.callout) + .foregroundStyle(.secondary) } } @@ -141,6 +138,7 @@ struct PermissionsStepView: View { let label: String let symbol: String let buttonLabel: String + let detail: String } @ViewBuilder @@ -149,17 +147,16 @@ struct PermissionsStepView: View { granted: Bool, action: @escaping () -> Void ) -> some View { - SettingRow(title: info.label, systemImage: info.symbol) { + SettingRow(title: info.label, systemImage: info.symbol, subtitle: info.detail) { if granted { HStack(spacing: 4) { Image(systemName: "checkmark.circle.fill").foregroundStyle(BlurtBrand.accent) Text("Granted").foregroundStyle(.secondary) } } else { - // Prominent (brand-green fill), like the API key row's button: the one - // action that clears this row's outstanding requirement. Button(info.buttonLabel, action: action) - .glassButtonStyleCompat(prominent: true) + .glassButtonStyleCompat(prominent: false) + .accessibilityLabel("\(info.label): \(info.buttonLabel)") } } } diff --git a/App/Blurt/Blurt/Wizard/Steps/SoundStepView.swift b/App/Blurt/Blurt/Wizard/Steps/SoundStepView.swift index 8cb47847..56775cd6 100644 --- a/App/Blurt/Blurt/Wizard/Steps/SoundStepView.swift +++ b/App/Blurt/Blurt/Wizard/Steps/SoundStepView.swift @@ -1,17 +1,19 @@ import BlurtEngine import SwiftUI -/// The sound-cue section of the settings screen. A menu picker chooses the -/// record-start/stop voice (or None); the choice is persisted and the cue -/// players are reloaded immediately. +/// The sound-cue section of the settings screen. Two menu pickers choose the +/// start/stop voice (or None) for dictation and for read-aloud — its own pair, +/// so a read never sounds like a recording starting. Each choice is persisted, +/// its players reloaded, and a preview played immediately. struct SoundStepView: View { var coordinator: AppCoordinator // Empty means "no pack persisted" — the unset default belongs to // `SoundPackCatalog.fromPersisted` (below), which names no known voice for `""` - // and so resolves the catalog's default. See `HotkeyStepView` for why the view + // and so resolves the catalog's default. See `KeysSettingsTab` for why the view // must not restate it. @AppStorage(SoundPackStore.defaultsKey) private var soundPackID = "" + @AppStorage(SoundPackStore.readAloudDefaultsKey) private var readAloudPackID = "" private var selection: Binding { Binding( @@ -19,30 +21,54 @@ struct SoundStepView: View { SoundPackCatalog.blurt.fromPersisted(soundPackID) }, set: { newValue in - // Write through the store (see `HotkeyStepView` for why): the store owns the + // Write through the store (see `KeysSettingsTab` for why): the store owns the // encoding, `@AppStorage` observes the key to re-render. - SoundPackStore(catalog: .blurt).soundPack = newValue + SoundPackStore.dictation.soundPack = newValue coordinator.soundPackChanged() }) } + /// Read-aloud's pack, the same way, through its own catalog default. + private var readAloudSelection: Binding { + Binding( + get: { SoundPackCatalog.blurtReadAloud.fromPersisted(readAloudPackID) }, + set: { newValue in + SoundPackStore.readAloud.soundPack = newValue + coordinator.readAloudSoundPackChanged() + }) + } + + /// Both pickers' options: None, then every voice by synth. One catalog's + /// voices, so the two lists can't drift apart. + @ViewBuilder private var packOptions: some View { + Text(SoundPack.none.label).tag(SoundPack.none) + ForEach(SoundPackCatalog.blurt.groups, id: \.self) { group in + Section(group) { + ForEach(SoundPackCatalog.blurt.voices(in: group)) { pack in + Text(pack.label).tag(pack) + } + } + } + } + var body: some View { Section { PickerSettingRow( - title: "Cue sound", systemImage: "speaker.wave.2", - accessibilityID: UITestIdentifiers.soundPicker, selection: selection + title: "Dictation", systemImage: "speaker.wave.2", + accessibilityID: UITestIdentifiers.soundPicker, + subtitle: "Start and stop recording.", selection: selection ) { - Text(SoundPack.none.label).tag(SoundPack.none) - ForEach(SoundPackCatalog.blurt.groups, id: \.self) { group in - Section(group) { - ForEach(SoundPackCatalog.blurt.voices(in: group)) { pack in - Text(pack.label).tag(pack) - } - } - } + packOptions + } + PickerSettingRow( + title: "Read aloud", systemImage: "speaker.wave.2.bubble", + accessibilityID: UITestIdentifiers.readAloudSoundPicker, + subtitle: "Start and stop playback.", selection: readAloudSelection + ) { + packOptions } } header: { - Text("Sound") + Text("Sounds") } footer: { Text("Choose None to turn off start and stop sounds.") } diff --git a/App/Blurt/Blurt/Wizard/StyleRow.swift b/App/Blurt/Blurt/Wizard/StyleRow.swift index 053bfea2..4db0d2ac 100644 --- a/App/Blurt/Blurt/Wizard/StyleRow.swift +++ b/App/Blurt/Blurt/Wizard/StyleRow.swift @@ -31,27 +31,21 @@ struct StyleRow: View { } var body: some View { - SettingRow(title: "Style", systemImage: "textformat") { + SettingRow( + title: "Writing style", systemImage: "textformat", subtitle: "Choose how your dictations are written." + ) { if profiles.isEmpty { // With no alternative there is nothing to pick. A passive value avoids // presenting a one-item pop-up that promises a choice it can't offer. Text(StyleProfileStore.defaultStyleName) .foregroundStyle(.secondary) } else { - // The picker keeps its own (hidden) title so VoiceOver reads a - // meaningful name for the pop-up rather than an empty string. - Picker("Style", selection: choice) { - // Writes go through the store — never the raw slot — so the store - // keeps owning how a choice is encoded (a profile's id, or the - // store's "default" sentinel; see `HotkeyStepView.selection` for the - // precedent). + Picker("Writing style", selection: choice) { Text(StyleProfileStore.defaultStyleName).tag(Choice.defaultStyle) ForEach(profiles) { profile in Text(profile.name).tag(Choice.profile(profile.id)) } } - // Explicit, not inherited: the pop-up is the whole point (see the type's - // note on why segmented can't work here). .pickerStyle(.menu) .labelsHidden() .accessibilityIdentifier(UITestIdentifiers.styleProfilePickerFromMain) @@ -115,7 +109,7 @@ struct StyleRow: View { /// Settings. struct StylesInertNotice: View { /// The same caveat inside Settings, where the Enhanced transcripts switch is - /// on screen — the Styles pane's and the Key Styles sheet's footer. + /// on screen — the Styles pane's and the Keys tab's footer. static let settingsText = "Turn on Enhanced transcripts to use styles." /// Observed so the notice follows the toggle in the Settings window live. diff --git a/App/Blurt/Blurt/Wizard/TextShortcutsSection.swift b/App/Blurt/Blurt/Wizard/TextShortcutsSection.swift index 47bce690..09bf253c 100644 --- a/App/Blurt/Blurt/Wizard/TextShortcutsSection.swift +++ b/App/Blurt/Blurt/Wizard/TextShortcutsSection.swift @@ -59,11 +59,10 @@ struct TextShortcutsSection: View { .buttonStyle(.borderless) } } header: { - Text("Text Shortcuts") + Text("Text shortcuts") } footer: { Text( - "Say a phrase while dictating and it's replaced with your saved text — " - + "for example, “personal email” becomes your address.") + "Replace spoken phrases with saved text. For example, say “personal email” to insert your address.") } .sheet(item: $editing) { shortcut in TextShortcutEditorSheet(shortcut: shortcut, among: shortcuts) @@ -102,8 +101,14 @@ struct TextShortcutsSection: View { .onDeleteCommand { if let selection { remove(selection) } } .overlay { if shortcuts.isEmpty { - Text("No shortcuts") - .foregroundStyle(.secondary) + VStack(spacing: 6) { + Text("No text shortcuts yet") + .font(.headline) + Text("Click + to add a phrase and its replacement.") + .font(.callout) + .foregroundStyle(.secondary) + } + .allowsHitTesting(false) } } .frame(height: 160) diff --git a/App/Blurt/Blurt/Wizard/WizardView.swift b/App/Blurt/Blurt/Wizard/WizardView.swift index 89293f57..47b65375 100644 --- a/App/Blurt/Blurt/Wizard/WizardView.swift +++ b/App/Blurt/Blurt/Wizard/WizardView.swift @@ -1,4 +1,3 @@ -import AppKit import SwiftUI /// The first-run setup screen. A single page showing everything setup needs at @@ -20,6 +19,8 @@ struct WizardView: View { var body: some View { VStack(spacing: 0) { + MainWindowBranding() + .padding(.top, 16) header Form { // API key first — the reason setup leads with it: a user arriving from @@ -41,43 +42,23 @@ struct WizardView: View { } private var header: some View { - // Center the mark against the two-line text block rather than pinning it to - // the first line's top — top-alignment left the icon reading high over the - // taller title+subtitle stack. A little more spacing gives it room. - HStack(alignment: .center, spacing: 12) { - OnboardingBrandMark() - - VStack(alignment: .leading, spacing: 4) { - Text("Set up Blurt") - .font(.title2) - .fontWeight(.bold) - Text("Add your API key, then allow Microphone and Accessibility access.") - .font(.body) - .foregroundStyle(.secondary) - } + VStack(alignment: .leading, spacing: 4) { + Text("Set up Blurt") + .font(.title2) + .fontWeight(.bold) + Text( + coordinator.account.isAvailable + ? "Sign in, then allow microphone and Accessibility access." + : "Add your API key, then allow microphone and Accessibility access." + ) + .font(.body) + .foregroundStyle(.secondary) } .frame(maxWidth: .infinity, alignment: .leading) .padding(.horizontal, 20) // The standard titlebar provides the traffic-light clearance, so the top // inset is just breathing room between the bar and the header. - .padding(.top, 12) + .padding(.top, 20) .padding(.bottom, 4) } } - -private struct OnboardingBrandMark: View { - /// 60 pt, the size the design draws it at (a 60×60 icon against the title and - /// its two-line subtitle). It was 38, which left the mark reading as a favicon - /// beside the heading rather than as the app introducing itself — this is the - /// first thing shown on first run. - private static let size: CGFloat = 60 - - var body: some View { - Image(nsImage: NSApplication.shared.applicationIconImage) - .resizable() - .interpolation(.high) - .scaledToFit() - .frame(width: Self.size, height: Self.size) - .accessibilityHidden(true) - } -} diff --git a/App/Blurt/BlurtUITests/BlurtUITestSupport.swift b/App/Blurt/BlurtUITests/BlurtUITestSupport.swift index 097459bf..48bc4b79 100644 --- a/App/Blurt/BlurtUITests/BlurtUITestSupport.swift +++ b/App/Blurt/BlurtUITests/BlurtUITestSupport.swift @@ -58,6 +58,8 @@ class BlurtUITestCase: XCTestCase { func openSettingsWindow(timeout: TimeInterval = 10) -> XCUIElement { let paneTitles = [ UITestIdentifiers.generalSettingsTab, + UITestIdentifiers.keysSettingsTab, + UITestIdentifiers.readAloudSettingsTab, UITestIdentifiers.stylesSettingsTab, UITestIdentifiers.vocabularySettingsTab, UITestIdentifiers.advancedSettingsTab, diff --git a/App/Blurt/BlurtUITests/HarnessDriving.swift b/App/Blurt/BlurtUITests/HarnessDriving.swift new file mode 100644 index 00000000..edcdac74 --- /dev/null +++ b/App/Blurt/BlurtUITests/HarnessDriving.swift @@ -0,0 +1,108 @@ +import XCTest + +/// Driving the UI-test harness (`UITestHarnessView`): picking a trigger key, +/// setting the transcript, dictating, reading aloud, and reading the harness's +/// read-outs. Shared by the suites that exercise the keys and read-aloud. +extension BlurtUITestCase { + + /// The harness, with an API key set so presses clear the missing-key gate. + func readyHarness() -> XCUIElement { + let harness = harnessWindow() + harness.buttons[UITestIdentifiers.setKeyButton].click() + return harness + } + + /// Points the harness's press/release buttons at `key`. + func pick(_ key: String, in harness: XCUIElement) { + let picker = harness.popUpButtons[UITestIdentifiers.harnessKeyPicker] + XCTAssertTrue(picker.waitForExistence(timeout: 5), "The harness key picker not found") + picker.click() + picker.menuItems[key].click() + } + + func setTranscript(_ text: String, in harness: XCUIElement) { + let field = harness.textFields[UITestIdentifiers.transcriptField] + field.click() + field.typeKey("a", modifierFlags: .command) + // Typing nothing wouldn't replace the selection, so clear it first. + field.typeKey(.delete, modifierFlags: []) + field.typeText(text) + } + + /// A hold of the picked key: records, then pastes `transcript`. + func dictate(in harness: XCUIElement, expecting transcript: String, _ message: String) { + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + waitForLabel(statusLabel(in: harness), equals: UITestIdentifiers.statusRecording, "\(message): no recording") + harness.buttons[UITestIdentifiers.hotkeyReleaseButton].click() + waitForLabel(pastedLabel(in: harness), equals: transcript, timeout: 15, "\(message): nothing pasted") + waitForLabel(statusLabel(in: harness), equals: UITestIdentifiers.statusIdle) + } + + /// A tap of left ⌥ (reading aloud by default), until the read is playing. + func startRead(in harness: XCUIElement) { + pick("left ⌥", in: harness) + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + harness.buttons[UITestIdentifiers.hotkeyTapReleaseButton].click() + waitForLabel(spoken(in: harness), equals: UITestIdentifiers.cannedSelection, "The selection should be read") + waitForLabel(reading(in: harness), equals: "yes", "The read should be playing") + } + + func addStyle(named name: String, instructions: String, in settings: XCUIElement) { + let styles = selectSettingsTab(settings, named: UITestIdentifiers.stylesSettingsTab) + styles.buttons[UITestIdentifiers.styleProfileAdd].click() + let sheet = styles.sheets.firstMatch + XCTAssertTrue(sheet.waitForExistence(timeout: 5), "The Add Style sheet should open") + fill(name, into: UITestIdentifiers.styleProfileName, in: sheet) + fill(instructions, into: UITestIdentifiers.styleProfileInstructions, in: sheet) + sheet.buttons[UITestIdentifiers.styleProfileSave].click() + XCTAssertTrue(sheet.waitForNonExistence(timeout: 5), "Saving should dismiss the style sheet") + } + + func fill(_ text: String, into identifier: String, in sheet: XCUIElement) { + let field = sheet.anyDescendant(identified: identifier) + XCTAssertTrue(field.waitForExistence(timeout: 5), "Field \(identifier) not found") + field.click() + field.typeText(text) + } + + func statusLabel(in harness: XCUIElement) -> XCUIElement { + harness.staticTexts[UITestIdentifiers.statusLabel] + } + + func pastedLabel(in harness: XCUIElement) -> XCUIElement { + harness.staticTexts[UITestIdentifiers.pastedLabel] + } + + func reading(in harness: XCUIElement) -> XCUIElement { + harness.staticTexts[UITestIdentifiers.readingLabel] + } + + func spoken(in harness: XCUIElement) -> XCUIElement { + harness.staticTexts[UITestIdentifiers.spokenLabel] + } + + /// A read-out's string. An empty SwiftUI `Text` isn't in the accessibility + /// tree at all, so a read-out that isn't there reads as "". + func text(of element: XCUIElement) -> String { + guard element.exists else { return "" } + return (element.value as? String).flatMap { $0.isEmpty ? nil : $0 } ?? element.label + } + + func status(in harness: XCUIElement) -> String { text(of: statusLabel(in: harness)) } + func pasted(in harness: XCUIElement) -> String { text(of: pastedLabel(in: harness)) } + func sentStyle(in harness: XCUIElement) -> String { + text(of: harness.staticTexts[UITestIdentifiers.sentStyleLabel]) + } + + func spokenStyle(in harness: XCUIElement) -> String { + text(of: harness.staticTexts[UITestIdentifiers.spokenStyleLabel]) + } + + /// Waits for `element`'s string to contain `fragment`. + func waitForText(_ element: XCUIElement, containing fragment: String, timeout: TimeInterval = 10, _ message: String) { + let predicate = NSPredicate(format: "label CONTAINS %@ OR value CONTAINS %@", fragment, fragment) + let result = XCTWaiter().wait( + for: [XCTNSPredicateExpectation(predicate: predicate, object: element)], timeout: timeout) + XCTAssertEqual(result, .completed, message) + } +} diff --git a/App/Blurt/BlurtUITests/KeysAndReadAloudUITests.swift b/App/Blurt/BlurtUITests/KeysAndReadAloudUITests.swift new file mode 100644 index 00000000..960f760e --- /dev/null +++ b/App/Blurt/BlurtUITests/KeysAndReadAloudUITests.swift @@ -0,0 +1,137 @@ +import XCTest + +/// Every trigger key and what it can do, end to end through the real key tap +/// (the harness drives its gate and callbacks — see `UITestHarnessView`): each +/// key dictating, a key turned off, the read-aloud and ask keys, the dictation +/// key interrupting a read, a key's own dictation style, and text shortcuts +/// expanding in what's pasted. Read-aloud runs offline: the selection is +/// `UITestIdentifiers.cannedSelection`, and the stub speaker reports what it +/// was asked to read on the harness's Spoken read-out. +final class KeysAndReadAloudUITests: BlurtUITestCase { + /// Each `TriggerKey`'s label, as the harness's key picker lists it. Spelled + /// out because this bundle doesn't link the engine. + private let allKeys = ["right ⌘", "right ⌥", "fn", "left ⌘", "left ⌥", "left ⌃"] + + // MARK: - Every key + + /// Each key, set to dictate, records on a press and pastes on its release. + /// Each but the bound key, set to off, then starts nothing. + func testEveryKeyDictatesAndCanBeTurnedOff() { + let harness = readyHarness() + for key in allKeys { + pick(key, in: harness) + harness.buttons[UITestIdentifiers.harnessSetDictate].click() + let transcript = "Dictated with \(key)." + setTranscript(transcript, in: harness) + dictate(in: harness, expecting: transcript, "\(key) should dictate") + } + for key in allKeys.dropFirst() { + pick(key, in: harness) + harness.buttons[UITestIdentifiers.harnessSetOff].click() + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + sleep(1) + XCTAssertEqual(status(in: harness), UITestIdentifiers.statusIdle, "\(key), turned off, should start nothing") + harness.buttons[UITestIdentifiers.hotkeyReleaseButton].click() + } + } + + // MARK: - Read aloud and ask + + /// Left ⌥ reads aloud by default: a tap over a selection reads it, and + /// nothing is recorded or pasted. Its next tap stops the read. + func testReadAloudKeyReadsTheSelection() { + let harness = readyHarness() + startRead(in: harness) + XCTAssertEqual(status(in: harness), UITestIdentifiers.statusIdle, "A read should not record") + XCTAssertEqual(pasted(in: harness), "", "A read should paste nothing") + + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + harness.buttons[UITestIdentifiers.hotkeyTapReleaseButton].click() + waitForLabel(reading(in: harness), equals: "no", "The read key's next tap should stop the read") + } + + /// Pressing the dictation key during a read stops the read and dictates. + func testDictationKeyInterruptsARead() { + let harness = readyHarness() + startRead(in: harness) + pick("right ⌘", in: harness) + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + waitForLabel(reading(in: harness), equals: "no", "The dictation key should stop the read") + waitForLabel(statusLabel(in: harness), equals: UITestIdentifiers.statusRecording, "…and start dictating") + harness.buttons[UITestIdentifiers.hotkeyReleaseButton].click() + waitForLabel( + pastedLabel(in: harness), equals: UITestIdentifiers.defaultCannedTranscript, timeout: 15, + "The dictation should paste") + } + + /// A key set to ask, held over a selection, records the request and reads + /// the answer aloud; the request itself is never pasted. + func testAskKeyAnswersAboutTheSelection() { + let harness = readyHarness() + pick("left ⌃", in: harness) + harness.buttons[UITestIdentifiers.harnessSetAsk].click() + setTranscript("Summarize this.", in: harness) + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + waitForLabel(statusLabel(in: harness), equals: UITestIdentifiers.statusRecording, "A held ask should record") + harness.buttons[UITestIdentifiers.hotkeyReleaseButton].click() + waitForLabel( + spoken(in: harness), equals: "Answer to: Summarize this.", timeout: 15, "The answer should be read aloud") + XCTAssertEqual(pasted(in: harness), "", "The request should not be pasted") + } + + // MARK: - Styles + + /// A style made in Settings shows up in the Keys tab's menu. The bound key + /// dictates with the main window's style; a key set to "No custom style" + /// sends none. + func testEachKeyDictatesWithItsOwnStyle() { + let harness = readyHarness() + let settings = openSettingsWindow() + addStyle(named: "Casual", instructions: "keep it breezy", in: settings) + + // The window's title follows its tab, so find it afresh before switching. + let keys = selectSettingsTab(openSettingsWindow(), named: UITestIdentifiers.keysSettingsTab) + let option = keys.popUpButtons[UITestIdentifiers.keyStylePicker(61)] + XCTAssertTrue(option.waitForExistence(timeout: 5), "Right ⌥'s action pop-up not found") + option.click() + XCTAssertTrue(option.menuItems["Dictate: Casual"].waitForExistence(timeout: 5), "The new style should be offered") + option.menuItems["Dictate: No custom style"].click() + + harness.click() + pick("right ⌘", in: harness) + dictate(in: harness, expecting: UITestIdentifiers.defaultCannedTranscript, "The bound key should dictate") + XCTAssertTrue( + sentStyle(in: harness).contains("keep it breezy"), + "The bound key should send the main window's style (got: \(sentStyle(in: harness)))") + + pick("right ⌥", in: harness) + setTranscript("Plain words.", in: harness) + dictate(in: harness, expecting: "Plain words.", "Right ⌥ should dictate") + XCTAssertEqual(sentStyle(in: harness), "—", "A key set to No custom style should send none") + } + + // MARK: - Text shortcuts + + /// A text shortcut added in Settings expands in what a dictation pastes. + func testTextShortcutExpandsInADictation() { + let harness = readyHarness() + let settings = openSettingsWindow() + let vocabulary = selectSettingsTab(settings, named: UITestIdentifiers.vocabularySettingsTab) + vocabulary.anyDescendant(identified: UITestIdentifiers.textShortcutAdd).click() + let sheet = vocabulary.sheets.firstMatch + XCTAssertTrue(sheet.waitForExistence(timeout: 5), "The shortcut sheet should open") + fill("my email", into: UITestIdentifiers.textShortcutTrigger, in: sheet) + fill("alex@example.com", into: UITestIdentifiers.textShortcutExpansion, in: sheet) + sheet.buttons[UITestIdentifiers.textShortcutSave].click() + XCTAssertTrue(sheet.waitForNonExistence(timeout: 5), "Save should dismiss the sheet") + + harness.click() + setTranscript("Send it to my email.", in: harness) + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + waitForLabel(statusLabel(in: harness), equals: UITestIdentifiers.statusRecording) + harness.buttons[UITestIdentifiers.hotkeyReleaseButton].click() + waitForText( + pastedLabel(in: harness), containing: "alex@example.com", timeout: 15, + "The pasted text should carry the expansion (got: \(pasted(in: harness)))") + } +} diff --git a/App/Blurt/BlurtUITests/KeysLiveUpdateUITests.swift b/App/Blurt/BlurtUITests/KeysLiveUpdateUITests.swift new file mode 100644 index 00000000..8ddef04e --- /dev/null +++ b/App/Blurt/BlurtUITests/KeysLiveUpdateUITests.swift @@ -0,0 +1,44 @@ +import XCTest + +/// The ready screen and the menu bar follow the keys as they change, without a +/// relaunch: a key given an action gains a row, and moving dictation to +/// another key renames the dictation row and the menu bar's reminder. +final class KeysLiveUpdateUITests: BlurtUITestCase { + override var extraLaunchArguments: [String] { [UITestIdentifiers.readyLaunchArgument] } + + private func row(_ label: String) -> XCUIElement { + app.descendants(matching: .any).matching(NSPredicate(format: "label BEGINSWITH %@", label)).firstMatch + } + + func testReadyScreenAndMenuBarFollowTheKeys() { + // Both windows looked up directly, as ReadyViewUITests does: the shared + // window helpers each close every window but the one they return. + let harness = app.windows[UITestIdentifiers.harnessWindowTitle] + XCTAssertTrue(harness.waitForExistence(timeout: 10), "Harness window not presented") + XCTAssertTrue( + app.windows[UITestIdentifiers.mainWindowTitle].waitForExistence(timeout: 10), "Ready window not presented") + harness.buttons[UITestIdentifiers.setKeyButton].click() + + pick("left ⌃", in: harness) + harness.buttons[UITestIdentifiers.harnessSetAsk].click() + XCTAssertTrue( + row("\(KeyActionName.ask): Left Control.").waitForExistence(timeout: 5), + "The ready screen should gain a row for the key that asks") + + pick("right ⌥", in: harness) + harness.buttons[UITestIdentifiers.harnessSetDictate].click() + pick("right ⌘", in: harness) + harness.buttons[UITestIdentifiers.harnessSetReadAloud].click() + XCTAssertTrue( + row("Dictate: Right Option.").waitForExistence(timeout: 5), + "Dictation moved to right ⌥, so the dictation row should name it") + XCTAssertTrue(row("\(KeyActionName.readAloud): Right Command.").exists) + + let statusItem = app.menuBars.statusItems.firstMatch + XCTAssertTrue(statusItem.waitForExistence(timeout: 15)) + statusItem.click() + let hint = app.menuItems.matching(NSPredicate(format: "title CONTAINS %@", "right ⌥ to dictate")).firstMatch + XCTAssertTrue(hint.waitForExistence(timeout: 5), "The menu bar's reminder should name right ⌥") + app.typeKey(.escape, modifierFlags: []) + } +} diff --git a/App/Blurt/BlurtUITests/ReadAloudFlowsUITests.swift b/App/Blurt/BlurtUITests/ReadAloudFlowsUITests.swift new file mode 100644 index 00000000..80f8c2be --- /dev/null +++ b/App/Blurt/BlurtUITests/ReadAloudFlowsUITests.swift @@ -0,0 +1,104 @@ +import XCTest + +/// Read-aloud and ask beyond the basics: the pill during and after a read, a +/// read held to listen on another key, a tapped ask, an ask with nothing said, +/// and a press stopping an answer. +final class ReadAloudFlowsUITests: BlurtUITestCase { + private var pill: XCUIElement { app.anyDescendant(identified: UITestIdentifiers.overlayPill) } + + // MARK: - U3: the pill + + /// A read wears the pill as "Reading aloud." and takes it down when stopped. + /// After a dictation, a read that ends must not replay the dictation's + /// "pasted" notice. + func testPillShowsAReadAndNeverReplaysAnOldNotice() { + let harness = readyHarness() + dictate(in: harness, expecting: UITestIdentifiers.defaultCannedTranscript, "The bound key should dictate") + sleep(2) // let the "Pasted" notice clear + + startRead(in: harness) + waitForLabel(pill, equals: "Reading aloud.", "The pill should show the read") + + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + harness.buttons[UITestIdentifiers.hotkeyTapReleaseButton].click() + waitForLabel(reading(in: harness), equals: "no", "The second tap should stop the read") + for _ in 0..<10 { + XCTAssertFalse( + pill.exists && pill.label == "Your dictation was pasted.", + "A read's end must not replay the earlier dictation's notice") + usleep(100_000) + } + } + + // MARK: - U4: a held read on another key + + /// Left ⌃ set to read aloud: held, it reads once the key has been down past + /// the hold delay; let go, it stops. + func testHeldReadOnAnotherKey() { + let harness = readyHarness() + pick("left ⌃", in: harness) + harness.buttons[UITestIdentifiers.harnessSetReadAloud].click() + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + waitForLabel(reading(in: harness), equals: "yes", "Holding the key should start the read") + waitForLabel(spoken(in: harness), equals: UITestIdentifiers.cannedSelection) + harness.buttons[UITestIdentifiers.hotkeyReleaseButton].click() + waitForLabel(reading(in: harness), equals: "no", "Letting go should stop a held read") + } + + // MARK: - U5: ask + + /// A tap-only ask key records from one tap to the next, like a dictation, + /// then reads the answer. + func testTappedAsk() { + let harness = readyHarness() + pick("left ⌃", in: harness) + harness.buttons[UITestIdentifiers.harnessSetAsk].click() + harness.buttons[UITestIdentifiers.harnessSetTapMode].click() + setTranscript("What does this mean?", in: harness) + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + harness.buttons[UITestIdentifiers.hotkeyTapReleaseButton].click() + waitForLabel(statusLabel(in: harness), equals: UITestIdentifiers.statusRecording, "A tap should start the ask") + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + harness.buttons[UITestIdentifiers.hotkeyTapReleaseButton].click() + waitForLabel( + spoken(in: harness), equals: "Answer to: What does this mean?", timeout: 15, "The next tap should end it") + } + + /// An ask with nothing said answers nothing — and leaves the key working. + func testAskWithNothingSaidLeavesTheKeyWorking() { + let harness = readyHarness() + pick("left ⌃", in: harness) + harness.buttons[UITestIdentifiers.harnessSetAsk].click() + setTranscript("", in: harness) + holdAsk(in: harness) + waitForLabel(statusLabel(in: harness), equals: UITestIdentifiers.statusIdle, timeout: 15) + XCTAssertEqual(text(of: spoken(in: harness)), "—", "Nothing said, so nothing answered") + + setTranscript("Try again.", in: harness) + holdAsk(in: harness) + waitForLabel(spoken(in: harness), equals: "Answer to: Try again.", timeout: 15, "The key should still ask") + } + + /// Pressing the dictation key while an answer plays stops it and dictates. + func testPressStopsAnAnswer() { + let harness = readyHarness() + pick("left ⌃", in: harness) + harness.buttons[UITestIdentifiers.harnessSetAsk].click() + setTranscript("Summarize.", in: harness) + holdAsk(in: harness) + waitForLabel(reading(in: harness), equals: "yes", timeout: 15, "The answer should play") + + pick("right ⌘", in: harness) + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + waitForLabel(reading(in: harness), equals: "no", "The press should stop the answer") + waitForLabel(statusLabel(in: harness), equals: UITestIdentifiers.statusRecording, "…and dictate") + harness.buttons[UITestIdentifiers.hotkeyReleaseButton].click() + } + + /// Holds the picked ask key through a request: press, recording, release. + private func holdAsk(in harness: XCUIElement) { + harness.buttons[UITestIdentifiers.hotkeyPressButton].click() + waitForLabel(statusLabel(in: harness), equals: UITestIdentifiers.statusRecording, "A held ask should record") + harness.buttons[UITestIdentifiers.hotkeyReleaseButton].click() + } +} diff --git a/App/Blurt/BlurtUITests/ReadAloudSettingsUITests.swift b/App/Blurt/BlurtUITests/ReadAloudSettingsUITests.swift new file mode 100644 index 00000000..ba2db078 --- /dev/null +++ b/App/Blurt/BlurtUITests/ReadAloudSettingsUITests.swift @@ -0,0 +1,96 @@ +import XCTest + +/// The Read Aloud tab and read-aloud's own sound: each setting persists, the +/// controls follow whether any key reads aloud, the privacy notes follow what +/// the keys do, and the voice and speed picked are the ones a read uses. +final class ReadAloudSettingsUITests: BlurtUITestCase { + private func readAloudTab() -> XCUIElement { + selectSettingsTab(openSettingsWindow(), named: UITestIdentifiers.readAloudSettingsTab) + } + + /// Picks `item` from `popUp`'s menu. + private func choose(_ item: String, in popUp: XCUIElement) { + XCTAssertTrue(popUp.waitForExistence(timeout: 5), "Pop-up not found") + popUp.click() + popUp.menuItems[item].click() + } + + // MARK: - U1: the read-aloud sound + + /// Read-aloud's sound starts on the Juno's Brass and is its own setting: + /// changing it leaves dictation's alone, and it sticks. + func testReadAloudSoundIsItsOwnSetting() { + let general = openSettingsWindow() + let read = general.popUpButtons[UITestIdentifiers.readAloudSoundPicker] + let dictation = general.popUpButtons[UITestIdentifiers.soundPicker] + XCTAssertTrue(read.waitForExistence(timeout: 5)) + XCTAssertEqual(read.value as? String, "Brass") + XCTAssertEqual(dictation.value as? String, "Orchestra") + + choose("Clav 1", in: read) + XCTAssertEqual(read.value as? String, "Clav 1") + XCTAssertEqual(dictation.value as? String, "Orchestra", "Read-aloud's sound must not change dictation's") + + choose("Brass 1", in: dictation) + XCTAssertEqual(read.value as? String, "Clav 1", "…nor dictation's change read-aloud's") + + let reopened = selectSettingsTab( + selectSettingsTab(general, named: UITestIdentifiers.readAloudSettingsTab), + named: UITestIdentifiers.generalSettingsTab) + XCTAssertEqual(reopened.popUpButtons[UITestIdentifiers.readAloudSoundPicker].value as? String, "Clav 1") + } + + // MARK: - U2: the Read Aloud tab + + /// Skipping technical text starts on, and a picked speed sticks. + func testDefaultsAndSpeedPersist() { + let tab = readAloudTab() + let skip = tab.anyDescendant(identified: UITestIdentifiers.readAloudSkipsJargonToggle) + XCTAssertTrue(skip.waitForExistence(timeout: 5)) + XCTAssertEqual(skip.value as? Int ?? Int(skip.value as? String ?? ""), 1, "Skip technical text should start on") + + choose("1.5×", in: tab.popUpButtons[UITestIdentifiers.readAloudSpeedPicker]) + let reopened = selectSettingsTab( + selectSettingsTab(tab, named: UITestIdentifiers.generalSettingsTab), + named: UITestIdentifiers.readAloudSettingsTab) + XCTAssertEqual(reopened.popUpButtons[UITestIdentifiers.readAloudSpeedPicker].value as? String, "1.5×") + } + + /// With no key reading aloud or asking, the controls are disabled and the tab + /// says how to turn read-aloud on; a key that asks adds the gateway note. + func testControlsAndNotesFollowTheKeys() { + let harness = readyHarness() + pick("left ⌥", in: harness) + harness.buttons[UITestIdentifiers.harnessSetOff].click() + + let tab = readAloudTab() + let voice = tab.popUpButtons[UITestIdentifiers.readAloudVoicePicker] + XCTAssertTrue(voice.waitForExistence(timeout: 5)) + XCTAssertFalse(voice.isEnabled, "With no read or ask key, the voice can't be picked") + let hint = "to enable playback" + let assign = NSPredicate(format: "value CONTAINS %@ OR label CONTAINS %@", hint, hint) + XCTAssertTrue(tab.staticTexts.matching(assign).firstMatch.exists, "The tab should say how to turn it on") + + harness.click() + pick("left ⌃", in: harness) + harness.buttons[UITestIdentifiers.harnessSetAsk].click() + let tabAgain = readAloudTab() + let gateway = NSPredicate(format: "value CONTAINS %@ OR label CONTAINS %@", "spoken request", "spoken request") + XCTAssertTrue( + tabAgain.staticTexts.matching(gateway).firstMatch.waitForExistence(timeout: 5), + "A key that asks should add the LLM Gateway note") + XCTAssertTrue(tabAgain.popUpButtons[UITestIdentifiers.readAloudVoicePicker].isEnabled) + } + + /// The voice and speed picked are the ones the next read speaks with. + func testPickedVoiceAndSpeedReachARead() { + let harness = readyHarness() + let tab = readAloudTab() + choose("Buford (British)", in: tab.popUpButtons[UITestIdentifiers.readAloudVoicePicker]) + choose("1.5×", in: tab.popUpButtons[UITestIdentifiers.readAloudSpeedPicker]) + + harness.click() + startRead(in: harness) + XCTAssertEqual(spokenStyle(in: harness), "aai-buford @ 1.5") + } +} diff --git a/App/Blurt/BlurtUITests/ReadyViewUITests.swift b/App/Blurt/BlurtUITests/ReadyViewUITests.swift index f9dec0f2..6233f689 100644 --- a/App/Blurt/BlurtUITests/ReadyViewUITests.swift +++ b/App/Blurt/BlurtUITests/ReadyViewUITests.swift @@ -14,14 +14,20 @@ final class ReadyViewUITests: BlurtUITestCase { func testReadyScreenShowsShortcutAndRecent() { mainWindow() - // The shortcut row: one element naming the bound key in words (the row + // The dictation row: one element naming the bound key in words (the row // itself draws it as a keycap) and how to use it. let shortcutPredicate = NSPredicate( - format: "label BEGINSWITH %@", "Shortcut: Right Command.") + format: "label BEGINSWITH %@", "Dictate: Right Command.") XCTAssertTrue( app.descendants(matching: .any).matching(shortcutPredicate).firstMatch .waitForExistence(timeout: 10), "Ready screen should state the dictation shortcut") + // And a row for each other configured key: left ⌥ reads aloud by default. + let readPredicate = NSPredicate( + format: "label BEGINSWITH %@", "\(KeyActionName.readAloud): Left Option.") + XCTAssertTrue( + app.descendants(matching: .any).matching(readPredicate).firstMatch.exists, + "Ready screen should list the read-aloud key") // The style row is always present. With no custom styles there isn't a real // choice, so it reads Default without drawing a one-item pop-up. @@ -169,7 +175,7 @@ final class ReadyViewUITests: BlurtUITestCase { } /// The empty Recent list's first-dictation prompt, one element whose label - /// starts "Start your first dictation" and goes on to name the bound key. + /// starts "Ready when you are" and goes on to name the bound key. private func emptyRecentPrompt(in main: XCUIElement) -> XCUIElement { main.descendants(matching: .any)[UITestIdentifiers.recentEmptyPrompt] } diff --git a/App/Blurt/BlurtUITests/SettingsUITests.swift b/App/Blurt/BlurtUITests/SettingsUITests.swift index 78b819b6..5508a3ea 100644 --- a/App/Blurt/BlurtUITests/SettingsUITests.swift +++ b/App/Blurt/BlurtUITests/SettingsUITests.swift @@ -1,9 +1,8 @@ import XCTest /// Drives the Settings window: the AssemblyAI API-key sheet (connect / reject -/// / cancel) reached from the account row, the -/// dictation-key picker and its Key Styles sheet, the sound-cue picker, and the -/// key-terms field. All +/// / cancel) reached from the account row, the Keys tab, the Read Aloud tab, +/// the sound-cue pickers, and the key-terms field. All /// offline — the API-key submit is short-circuited in UI-test mode, so these /// never reach AssemblyAI or the real Keychain. final class SettingsUITests: BlurtUITestCase { @@ -63,17 +62,20 @@ final class SettingsUITests: BlurtUITestCase { "Show key should expose a plain (non-secure) text field") } - /// The dictation-key picker changes the persisted trigger selection. - func testHotkeyPickerChangesSelection() { - let settings = openSettingsWindow() + /// Taking the bound key off dictation moves the binding to the other dictating + /// key: turn right ⌥ on, set right ⌘ to read aloud, and both stick. + func testDictationMovesToAnotherKey() { + let keys = openKeys() + let option = keys.popUpButtons[UITestIdentifiers.keyStylePicker(rightOption)] + let command = keys.popUpButtons[UITestIdentifiers.keyStylePicker(rightCommand)] - let picker = settings.popUpButtons[UITestIdentifiers.hotkeyPicker] - XCTAssertTrue(picker.waitForExistence(timeout: 10), "Hotkey picker not found") - // Default is right ⌘; switch to right ⌥ and confirm the selection sticks. - picker.click() - app.menuItems["right ⌥"].click() + option.click() + app.menuItems["Dictate"].click() + command.click() + app.menuItems[KeyActionName.readAloud].click() - XCTAssertEqual(picker.value as? String, "right ⌥") + XCTAssertEqual(command.value as? String, KeyActionName.readAloud) + XCTAssertEqual(option.value as? String, "Dictate") } /// The microphone picker exists and defaults to following the system input. @@ -234,85 +236,73 @@ final class SettingsUITests: BlurtUITestCase { "Cancel should return to the connected row") } - // MARK: - Key styles + // MARK: - Keys /// The trigger keys' keycodes (`TriggerKey`'s raw values), which suffix each - /// pop-up's identifier in the Key Styles sheet. Spelled out because this bundle + /// pop-up's identifier in the Keys tab. Spelled out because this bundle /// doesn't link the engine. private let rightCommand = 54 private let rightOption = 61 private let fnKey = 63 - - /// The Key Styles row's "Edit…" opens a sheet with a pop-up per trigger key. On a reset launch - /// right ⌘ is the bound key: it dictates with the main window's style and, since - /// the bound key always dictates, offers no Off; the other keys start Off. - func testKeyStylesSheetListsEveryKey() { - let sheet = openKeyStyles(openStylesSettings()) - - for keyCode in [rightCommand, rightOption, fnKey] { + private let leftCommand = 55 + private let leftOption = 58 + private let leftControl = 59 + + /// The Keys tab has an action and a mode pop-up per trigger key. On a reset + /// launch right ⌘ dictates and, as the only dictating key, can't be turned + /// off; left ⌥ reads aloud on a tap or hold; the rest are off, with no mode + /// to pick. + func testKeysTabListsEveryKey() { + let keys = openKeys() + + for keyCode in [rightCommand, rightOption, fnKey, leftCommand, leftOption, leftControl] { + XCTAssertTrue( + keys.popUpButtons[UITestIdentifiers.keyStylePicker(keyCode)].exists, + "The tab should list an action pop-up for keycode \(keyCode)") XCTAssertTrue( - sheet.popUpButtons[UITestIdentifiers.keyStylePicker(keyCode)].exists, - "The sheet should list a pop-up for keycode \(keyCode)") + keys.popUpButtons[UITestIdentifiers.keyActivationPicker(keyCode)].exists, + "The tab should list a mode pop-up for keycode \(keyCode)") } - XCTAssertEqual(sheet.popUpButtons[UITestIdentifiers.keyStylePicker(rightOption)].value as? String, "Off") + XCTAssertEqual(keys.popUpButtons[UITestIdentifiers.keyStylePicker(rightOption)].value as? String, "Off") + XCTAssertFalse(keys.popUpButtons[UITestIdentifiers.keyActivationPicker(rightOption)].isEnabled) + XCTAssertEqual( + keys.popUpButtons[UITestIdentifiers.keyStylePicker(leftOption)].value as? String, KeyActionName.readAloud) + XCTAssertEqual( + keys.popUpButtons[UITestIdentifiers.keyActivationPicker(leftOption)].value as? String, "Tap or hold") - let bound = sheet.popUpButtons[UITestIdentifiers.keyStylePicker(rightCommand)] - XCTAssertEqual(bound.value as? String, "Same as main window") + let bound = keys.popUpButtons[UITestIdentifiers.keyStylePicker(rightCommand)] + XCTAssertEqual(bound.value as? String, "Dictate") bound.click() - XCTAssertTrue(app.menuItems["Same as main window"].waitForExistence(timeout: 5)) - XCTAssertFalse(app.menuItems["Off"].exists, "The bound key always dictates, so it offers no Off") + XCTAssertTrue(bound.menuItems["Dictate"].waitForExistence(timeout: 5)) + XCTAssertFalse(bound.menuItems["Off"].exists, "The only dictating key can't be turned off") + XCTAssertFalse(bound.menuItems[KeyActionName.readAloud].exists, "…nor set to read aloud") // Close the menu by re-choosing the current item, leaving the setting as it was. - app.menuItems["Same as main window"].click() + bound.menuItems["Dictate"].click() } - /// Turning right ⌥ on persists: the button's spoken summary names it, and the - /// sheet shows the choice again when reopened. - func testKeyStyleChoicePersists() { - let settings = openStylesSettings() - let sheet = openKeyStyles(settings) - - sheet.popUpButtons[UITestIdentifiers.keyStylePicker(rightOption)].click() - app.menuItems["Same as main window"].click() - sheet.buttons[UITestIdentifiers.keyStylesDone].click() - XCTAssertTrue(sheet.waitForNonExistence(timeout: 5), "Done should dismiss the sheet") - - let summary = settings.buttons[UITestIdentifiers.keyStylesButton].value as? String ?? "" - XCTAssertTrue( - summary.contains("right ⌥: Same as main window"), - "The button's accessibility value should summarise the new mapping (got: \(summary))") + /// A key's mode persists: the tab shows it again after another pane. + func testKeyModePersists() { + openKeys().popUpButtons[UITestIdentifiers.keyActivationPicker(leftOption)].click() + app.menuItems["Hold"].click() - let reopened = openKeyStyles(settings) + let settings = openSettingsWindow() + let reopened = selectSettingsTab(settings, named: UITestIdentifiers.keysSettingsTab) XCTAssertEqual( - reopened.popUpButtons[UITestIdentifiers.keyStylePicker(rightOption)].value as? String, - "Same as main window", "The choice should survive closing the sheet") + reopened.popUpButtons[UITestIdentifiers.keyActivationPicker(leftOption)].value as? String, "Hold", + "The mode should survive leaving the tab") } - /// Escape closes the sheet (it has no Cancel button to carry the shortcut) and - /// leaves the Settings window open. - func testEscapeClosesKeyStylesSheet() { - let settings = openStylesSettings() - let sheet = openKeyStyles(settings) - - app.typeKey(.escape, modifierFlags: []) - - XCTAssertTrue(sheet.waitForNonExistence(timeout: 5), "Escape should dismiss the sheet") - XCTAssertTrue(settings.exists, "Escape should close only the sheet, not Settings") + /// The Read Aloud tab's voice picker starts on the service's default voice. + func testVoicePickerDefaultsToJane() { + let pane = selectSettingsTab(openSettingsWindow(), named: UITestIdentifiers.readAloudSettingsTab) + let voice = pane.popUpButtons[UITestIdentifiers.readAloudVoicePicker] + XCTAssertTrue(voice.waitForExistence(timeout: 5), "Voice picker not found") + XCTAssertEqual(voice.value as? String, "Jane") } - /// Returns the Styles pane, where the Key Styles row lives. - private func openStylesSettings() -> XCUIElement { - selectSettingsTab(openSettingsWindow(), named: UITestIdentifiers.stylesSettingsTab) - } - - /// Opens the Key Styles sheet from its row's Edit… button on the Styles - /// pane and returns it. - private func openKeyStyles(_ settings: XCUIElement) -> XCUIElement { - let button = settings.buttons[UITestIdentifiers.keyStylesButton] - XCTAssertTrue(button.waitForExistence(timeout: 10), "Key Styles row's Edit… button not found") - button.click() - let sheet = settings.sheets.firstMatch - XCTAssertTrue(sheet.waitForExistence(timeout: 5), "The Key Styles sheet should open") - return sheet + /// Opens Settings on the Keys tab and returns it. + private func openKeys() -> XCUIElement { + selectSettingsTab(openSettingsWindow(), named: UITestIdentifiers.keysSettingsTab) } // MARK: - API-key helpers diff --git a/App/Blurt/BlurtUITests/TextShortcutsUITests.swift b/App/Blurt/BlurtUITests/TextShortcutsUITests.swift index 2ebf6cff..d1df6df6 100644 --- a/App/Blurt/BlurtUITests/TextShortcutsUITests.swift +++ b/App/Blurt/BlurtUITests/TextShortcutsUITests.swift @@ -64,8 +64,8 @@ final class TextShortcutsUITests: BlurtUITestCase { addShortcut(in: pane, trigger: "work email", expansion: "me@example.com") let sheet = openSheet(in: pane, via: UITestIdentifiers.textShortcutAdd) - type("Work-Email", into: UITestIdentifiers.textShortcutTrigger, in: sheet) - type("other@example.com", into: UITestIdentifiers.textShortcutExpansion, in: sheet) + fill("Work-Email", into: UITestIdentifiers.textShortcutTrigger, in: sheet) + fill("other@example.com", into: UITestIdentifiers.textShortcutExpansion, in: sheet) let save = sheet.buttons[UITestIdentifiers.textShortcutSave] XCTAssertFalse(save.isEnabled, "Save should stay disabled for a duplicate phrase") @@ -103,19 +103,12 @@ final class TextShortcutsUITests: BlurtUITestCase { return sheet } - private func type(_ text: String, into identifier: String, in sheet: XCUIElement) { - let field = sheet.anyDescendant(identified: identifier) - XCTAssertTrue(field.waitForExistence(timeout: 5), "Field \(identifier) not found") - field.click() - field.typeText(text) - } - /// Adds one shortcut through the sheet, waiting on the sheet's disappearance /// for the reason on `SettingsUITests.connectValidKey`. private func addShortcut(in pane: XCUIElement, trigger: String, expansion: String) { let sheet = openSheet(in: pane, via: UITestIdentifiers.textShortcutAdd) - type(trigger, into: UITestIdentifiers.textShortcutTrigger, in: sheet) - type(expansion, into: UITestIdentifiers.textShortcutExpansion, in: sheet) + fill(trigger, into: UITestIdentifiers.textShortcutTrigger, in: sheet) + fill(expansion, into: UITestIdentifiers.textShortcutExpansion, in: sheet) sheet.buttons[UITestIdentifiers.textShortcutSave].click() XCTAssertTrue(sheet.waitForNonExistence(timeout: 5), "Save should dismiss the sheet") } diff --git a/App/Blurt/Config/SignIn.local.xcconfig.example b/App/Blurt/Config/SignIn.local.xcconfig.example index 8d435726..a4cf2f67 100644 --- a/App/Blurt/Config/SignIn.local.xcconfig.example +++ b/App/Blurt/Config/SignIn.local.xcconfig.example @@ -1,5 +1,7 @@ -// Copy to SignIn.local.xcconfig (gitignored) and fill in. Any one left empty -// turns sign-in off. xcconfig treats `//` as the start of a comment, so write +// Copy to SignIn.local.xcconfig (gitignored) and fill in to sign a dev build +// in to a sandbox deployment instead of production (SignIn.xcconfig includes +// this after the production values, so each one set here replaces its +// production value). Any one left empty turns sign-in off. xcconfig treats `//` as the start of a comment, so write // every URL's `//` as `/$()/`. // // BLURT_OAUTH_CLIENT_ID the Connected App's client id diff --git a/App/Blurt/Config/SignIn.xcconfig b/App/Blurt/Config/SignIn.xcconfig index 5878bd68..4dc3cbd2 100644 --- a/App/Blurt/Config/SignIn.xcconfig +++ b/App/Blurt/Config/SignIn.xcconfig @@ -1,9 +1,10 @@ // Sign in with AssemblyAI, for the Debug and Debug-Local configurations -// (see `configFiles` in project.yml). Release signs in to production instead, -// from SignIn.release.xcconfig. +// (see `configFiles` in project.yml). Dev builds sign in to production, the +// same as Release, from SignIn.release.xcconfig. // -// The values themselves are not in this repo: they name AssemblyAI's sandbox -// deployments. Copy SignIn.local.xcconfig.example to SignIn.local.xcconfig -// (gitignored) and fill it in; without it, dev builds simply don't show -// sign-in. +// To sign a dev build in to one of AssemblyAI's sandbox deployments instead, +// copy SignIn.local.xcconfig.example to SignIn.local.xcconfig (gitignored) +// and fill it in: its values, included after production's, replace them. The +// sandbox values aren't in this repo. +#include "SignIn.release.xcconfig" #include? "SignIn.local.xcconfig" diff --git a/App/Blurt/Shared/KeyActionName.swift b/App/Blurt/Shared/KeyActionName.swift new file mode 100644 index 00000000..9792bb2a --- /dev/null +++ b/App/Blurt/Shared/KeyActionName.swift @@ -0,0 +1,8 @@ +/// The names the Keys tab gives its read and ask actions. The tab's menu, the +/// ready screen's key rows, the Read Aloud tab's footers and the UI tests all +/// quote them, so they're spelled once — here in `Shared/`, where the UI tests +/// can reach them too. +enum KeyActionName { + static let readAloud = "Read Selection Aloud" + static let ask = "Ask About Selection" +} diff --git a/App/Blurt/Shared/UITestIdentifiers.swift b/App/Blurt/Shared/UITestIdentifiers.swift index 4f64b8d6..b07bab43 100644 --- a/App/Blurt/Shared/UITestIdentifiers.swift +++ b/App/Blurt/Shared/UITestIdentifiers.swift @@ -31,6 +31,8 @@ enum UITestIdentifiers { // its selected pane, so the General label doubles as the Settings window's // opening title (see `settingsWindowTitle` in BlurtUITestSupport). static let generalSettingsTab = "General" + static let keysSettingsTab = "Keys" + static let readAloudSettingsTab = "Read Aloud" static let stylesSettingsTab = "Styles" static let advancedSettingsTab = "Advanced" static let vocabularySettingsTab = "Vocabulary" @@ -41,6 +43,19 @@ enum UITestIdentifiers { static let startButton = "uitest.start" static let stopButton = "uitest.stop" static let cancelButton = "uitest.cancel" + static let harnessKeyPicker = "harness.key.picker" + static let harnessSetDictate = "harness.key.dictate" + static let harnessSetReadAloud = "harness.key.readAloud" + static let harnessSetAsk = "harness.key.ask" + static let harnessSetOff = "harness.key.off" + static let hotkeyTapReleaseButton = "harness.hotkey.tapRelease" + static let harnessSetTapMode = "harness.key.tapMode" + static let spokenStyleLabel = "harness.spokenStyle" + static let spokenLabel = "harness.spoken" + static let readingLabel = "harness.reading" + static let sentStyleLabel = "harness.sentStyle" + /// What the stub selection read returns: the text a read-aloud key reads. + static let cannedSelection = "The selected paragraph to read aloud." static let hotkeyPressButton = "uitest.hotkeyPress" static let hotkeyReleaseButton = "uitest.hotkeyRelease" static let statusLabel = "uitest.status" @@ -62,20 +77,17 @@ enum UITestIdentifiers { static let apiKeyCancel = "settings.apiKey.cancel" static let apiKeyError = "settings.apiKey.error" static let keyTermsField = "settings.keyTerms.field" - static let hotkeyPicker = "settings.hotkey.picker" - static let activationPicker = "settings.activation.picker" - static let keyStylesButton = "settings.keyStyles.button" - static let keyStylesDone = "settings.keyStyles.done" - /// One per trigger key in the Key Styles sheet, suffixed with its keycode. + /// One pair per trigger key on the Keys tab, suffixed with its keycode. static func keyStylePicker(_ keyCode: Int) -> String { "settings.keyStyles.picker.\(keyCode)" } + static func keyActivationPicker(_ keyCode: Int) -> String { "settings.keyStyles.activation.\(keyCode)" } static let micPicker = "settings.mic.picker" static let soundPicker = "settings.sound.picker" + static let readAloudSoundPicker = "settings.readAloudSound.picker" static let developerToggle = "settings.developer.toggle" static let developerShowLogs = "settings.developer.showLogs" static let enhancedTranscriptsToggle = "settings.enhancedTranscripts.toggle" static let rewriteModePicker = "settings.rewriteMode.picker" - static let selectionSpeechToggle = "settings.selectionSpeech.toggle" - static let selectionAskToggle = "settings.selectionAsk.toggle" + static let readAloudVoicePicker = "settings.readAloud.voice" static let readAloudSpeedPicker = "settings.readAloud.speed" static let readAloudSkipsJargonToggle = "settings.readAloud.skipsJargon" static let updateCheck = "settings.update.check" diff --git a/App/Blurt/project.yml b/App/Blurt/project.yml index efe59190..071a4ed4 100644 --- a/App/Blurt/project.yml +++ b/App/Blurt/project.yml @@ -136,11 +136,11 @@ targets: Release: PRODUCT_BUNDLE_IDENTIFIER: dev.alex.blurt BLURT_APP_NAME: Blurt - # Sign in with AssemblyAI's BLURT_OAUTH_* settings. Release signs in to - # production, whose values are public and committed. The dev - # configurations name AssemblyAI's sandbox deployments, so their values - # live in a gitignored include (see Config/SignIn.xcconfig); without it - # they're unset and sign-in is off. Baked into Info.plist below and read by + # Sign in with AssemblyAI's BLURT_OAUTH_* settings. Every configuration + # signs in to production, whose values are public and committed + # (Config/SignIn.release.xcconfig). A dev build can sign in to one of + # AssemblyAI's sandbox deployments instead through a gitignored include + # (see Config/SignIn.xcconfig). Baked into Info.plist below and read by # `OAuthConfiguration.blurt`. configFiles: Debug: Config/SignIn.xcconfig diff --git a/App/BlurtiOS/BlurtiOSCore/VoiceState.swift b/App/BlurtiOS/BlurtiOSCore/VoiceState.swift index 9f7f86d3..7dd8cfd7 100644 --- a/App/BlurtiOS/BlurtiOSCore/VoiceState.swift +++ b/App/BlurtiOS/BlurtiOSCore/VoiceState.swift @@ -122,7 +122,8 @@ extension PhaseSnapshot.State { nonisolated package init(_ overlay: OverlayUIState) { self = switch overlay { - case .idle: .idle + // Reading aloud is the Mac's alone, so the iPhone never sees it. + case .idle, .speaking: .idle case .connecting: .connecting case .recording: .recording case .processing: .processing @@ -139,7 +140,7 @@ extension PhaseSnapshot { nonisolated package static func message(for overlay: OverlayUIState) -> String? { switch overlay { case .error(let message, _): message - case .idle, .connecting, .recording, .processing, .pasted, .noTarget: nil + case .idle, .connecting, .recording, .processing, .speaking, .pasted, .noTarget: nil } } } diff --git a/Sources/BlurtEngine/Account/CredentialRows.swift b/Sources/BlurtEngine/Account/CredentialRows.swift index a954c056..1b78e36e 100644 --- a/Sources/BlurtEngine/Account/CredentialRows.swift +++ b/Sources/BlurtEngine/Account/CredentialRows.swift @@ -13,7 +13,11 @@ public struct CredentialRows: Sendable, Equatable { /// The account row: offered where sign-in is configured, and kept for a /// session inherited by a build without it, so its Sign Out stays reachable. public let showsAccountRow: Bool - /// The key row, hidden while signed in: the key is the connection's to rotate. + /// The key row. Pasting a key is the legacy path: a new install where + /// sign-in is configured signs in, and the row is only for an install that + /// already has a pasted key (to change or remove it) or a build with no + /// sign-in at all. Hidden while signed in, since the key is then the + /// connection's to rotate. public let showsKeyRow: Bool /// The filled button, or nil once a key is stored (nothing is waiting on the /// user, and a lone filled button would mark a rare errand as the main action). @@ -26,6 +30,7 @@ public struct CredentialRows: Sendable, Equatable { let primary: Primary? = signedIn || isKeyStored ? nil : (isSignInAvailable ? .signIn : .connect) return CredentialRows( - showsAccountRow: isSignInAvailable || signedIn, showsKeyRow: !signedIn, primary: primary) + showsAccountRow: isSignInAvailable || signedIn, + showsKeyRow: !signedIn && (isKeyStored || !isSignInAvailable), primary: primary) } } diff --git a/Sources/BlurtEngine/Audio/SoundPackCatalog.swift b/Sources/BlurtEngine/Audio/SoundPackCatalog.swift index ddeb589a..f60d3d7e 100644 --- a/Sources/BlurtEngine/Audio/SoundPackCatalog.swift +++ b/Sources/BlurtEngine/Audio/SoundPackCatalog.swift @@ -69,6 +69,23 @@ public struct SoundPackCatalog: Sendable { defaultPack = index[defaultVoiceID] ?? SoundPack.none } + /// The same voices with a different default — a second setting drawn from + /// one catalog, such as read-aloud's cues beside dictation's. An id naming no + /// voice falls back to `SoundPack.none`, as in `init`. + public func defaulting(to voiceID: String) -> SoundPackCatalog { + SoundPackCatalog( + groups: groups, defaultPack: byID[voiceID] ?? SoundPack.none, byID: byID, voicesByGroup: voicesByGroup) + } + + private init( + groups: [String], defaultPack: SoundPack, byID: [String: SoundPack], voicesByGroup: [String: [SoundPack]] + ) { + self.groups = groups + self.defaultPack = defaultPack + self.byID = byID + self.voicesByGroup = voicesByGroup + } + /// The voices belonging to one group, in catalog order. public func voices(in group: String) -> [SoundPack] { voicesByGroup[group] ?? [] } diff --git a/Sources/BlurtEngine/Audio/SoundPackStore.swift b/Sources/BlurtEngine/Audio/SoundPackStore.swift index 7776bde0..8ad728c9 100644 --- a/Sources/BlurtEngine/Audio/SoundPackStore.swift +++ b/Sources/BlurtEngine/Audio/SoundPackStore.swift @@ -11,24 +11,37 @@ import Foundation /// stranded the sound setting outside every reset — the exact bug the roster was /// created to prevent, twice. public struct SoundPackStore { - /// UserDefaults key holding the selected pack id. Public so SwiftUI views can + /// Which cues a store holds: dictation's, or read-aloud's — its own pair, so + /// a read never sounds like a recording starting. + public enum Cue: Sendable { + case dictation, readAloud + } + + /// UserDefaults key holding dictation's pack id. Public so SwiftUI views can /// observe it directly (e.g. `@AppStorage`) and re-render on change. public static var defaultsKey: String { DefaultsKey.soundPack.key } + /// Read-aloud's pack id, the same way. + public static var readAloudDefaultsKey: String { DefaultsKey.readAloudSoundPack.key } private let catalog: SoundPackCatalog private let defaults: UserDefaults + private let key: String - public init(catalog: SoundPackCatalog, defaults: UserDefaults = .standard) { + /// `catalog` supplies the voices and what an unset setting means, so a + /// read-aloud store is given a catalog defaulting to read-aloud's pack + /// (`SoundPackCatalog.defaulting(to:)`). + public init(catalog: SoundPackCatalog, cue: Cue = .dictation, defaults: UserDefaults = .standard) { self.catalog = catalog self.defaults = defaults + self.key = cue == .dictation ? Self.defaultsKey : Self.readAloudDefaultsKey } public var soundPack: SoundPack { get { - catalog.fromPersisted(defaults.string(forKey: Self.defaultsKey)) + catalog.fromPersisted(defaults.string(forKey: key)) } nonmutating set { - defaults.set(newValue.id, forKey: Self.defaultsKey) + defaults.set(newValue.id, forKey: key) } } } diff --git a/Sources/BlurtEngine/Config/DefaultsKey.swift b/Sources/BlurtEngine/Config/DefaultsKey.swift index 4f3d6c46..ba9941a4 100644 --- a/Sources/BlurtEngine/Config/DefaultsKey.swift +++ b/Sources/BlurtEngine/Config/DefaultsKey.swift @@ -27,14 +27,14 @@ enum DefaultsKey: String, CaseIterable { case triggerKeyCode = "TriggerKeyCode" case triggerActivation = "TriggerActivation" - /// Which style each trigger key dictates with, and so which keys beyond the - /// bound one dictate at all (`TriggerKeyStyleStore`). + /// What each trigger key does — dictate with a style, read aloud, ask, or + /// nothing (`TriggerKeyStyleStore`). The raw value predates read and ask. case triggerKeyStyles = "TriggerKeyStyles" - /// Each trigger key's entry for the role it isn't in right now — kept across - /// a rebind so moving the bound key away and back loses nothing - /// (`TriggerKeyStyleStore.rebound(from:to:)`). - case triggerKeyStyleStash = "TriggerKeyStyleStash" + /// Each trigger key's tap/hold mode (`TriggerKeyActivationStore`). + case triggerKeyActivations = "TriggerKeyActivations" case soundPack = "SoundPack" + /// Read-aloud's start/stop cue voice (`SoundPackStore`, `.readAloud`). + case readAloudSoundPack = "ReadAloudSoundPack" case keyTerms = "KeyTerms" case developerMode = "DeveloperMode" case enhancedTranscripts = "EnhancedTranscripts" @@ -56,14 +56,11 @@ enum DefaultsKey: String, CaseIterable { case micDeviceUID = "MicDeviceUID" /// The JSON list of spoken-phrase → text expansions (`TextShortcutStore`). case textShortcuts = "TextShortcuts" - /// The experimental read-selection-aloud switch (`SelectionSpeechStore`). - case selectionSpeech = "SelectionSpeech" - /// Read-aloud's playback speed, and whether it rewrites the text for - /// listening first (`ReadAloudStyleStore`, which owns both). + /// Read-aloud's playback speed, whether it rewrites the text for listening + /// first, and its voice (`ReadAloudStyleStore`, which owns all three). case readAloudSpeed = "ReadAloudSpeed" case readAloudSkipsJargon = "ReadAloudSkipsJargon" - /// Whether a hold over a selection asks about it (`SelectionAskStore`). - case selectionAsk = "SelectionAsk" + case readAloudVoice = "ReadAloudVoice" /// The key this case actually reads and writes, under the configured host /// identity. A computed property rather than a stored string because the diff --git a/Sources/BlurtEngine/Config/DeveloperModeStore.swift b/Sources/BlurtEngine/Config/DeveloperModeStore.swift index 351a407f..93002834 100644 --- a/Sources/BlurtEngine/Config/DeveloperModeStore.swift +++ b/Sources/BlurtEngine/Config/DeveloperModeStore.swift @@ -19,7 +19,7 @@ public struct DeveloperModeStore { /// /// Read-only: the Settings toggle writes the slot directly through `@AppStorage` /// (that's what `defaultsKey` is public for), so a setter here had no production - /// caller — only tests, which is the hazard `HotkeyStepView` documents in the + /// caller — only tests, which is the hazard `KeysSettingsTab` documents in the /// other direction. Rather than leave a write path nothing exercised, there is /// none; seed the slot to change the switch. var isEnabled: Bool { diff --git a/Sources/BlurtEngine/Config/PressStyle.swift b/Sources/BlurtEngine/Config/PressStyle.swift index d2a48e47..8bf0471f 100644 --- a/Sources/BlurtEngine/Config/PressStyle.swift +++ b/Sources/BlurtEngine/Config/PressStyle.swift @@ -1,7 +1,7 @@ import Foundation /// The style one dictation is made with, resolved once at press from the -/// trigger key that started it (`TriggerKeyStyleStore`) — else the main +/// trigger key that started it (`TriggerKeyBindings`) — else the main /// window's selection. A value carried by that dictation (the press command /// names the key, the session resolves it), so a key's style never moves the /// main window's sticky selection, and a later press can't restyle a dictation @@ -25,7 +25,10 @@ public struct PressStyle: Sendable, Equatable { /// `key`'s style, or the main window's selection for nil, a key with no /// style, or one whose style is gone (`StyleProfileStore.profile(for:)`). init(key: TriggerKey?, defaults: UserDefaults = .standard) { - let styleID = key.flatMap { TriggerKeyStyleStore(defaults: defaults).styles[$0] } + var styleID: String? + if let key, case .dictate(let style) = TriggerKeyBindings(defaults: defaults).choice(for: key) { + styleID = style + } let profile = StyleProfileStore(defaults: defaults).profile(for: styleID) let isEnhanced = EnhancedTranscriptsStore(defaults: defaults).isEnabled self.init(instructions: profile?.appendedInstructions, name: isEnhanced ? profile?.name : nil) diff --git a/Sources/BlurtEngine/Config/ReadAloudStyleStore.swift b/Sources/BlurtEngine/Config/ReadAloudStyleStore.swift index c127fc11..067e898c 100644 --- a/Sources/BlurtEngine/Config/ReadAloudStyleStore.swift +++ b/Sources/BlurtEngine/Config/ReadAloudStyleStore.swift @@ -1,7 +1,7 @@ import Foundation -/// How one read-aloud press should sound: the playback rate, and whether the -/// text is rewritten for listening before it is spoken +/// How one read-aloud press should sound: the voice, the playback rate, and +/// whether the text is rewritten for listening before it is spoken /// (`ReadAloudLLM.rewriteForListening`). /// `ReadAloudStyleStore.style` resolves it from the settings at each press. public struct ReadAloudStyle: Sendable, Equatable { @@ -11,32 +11,44 @@ public struct ReadAloudStyle: Sendable, Equatable { /// Leave out what sounds like noise read aloud: code, file paths, links, /// email addresses, long numbers. public let skipsJargon: Bool + /// The TTS request's `voice_id`, one of `ReadAloudVoice.catalog`. + public let voiceID: String - /// Read-aloud untouched: natural pace, the selection verbatim. + /// Read-aloud untouched: the default voice at natural pace, the selection verbatim. public static let standard = ReadAloudStyle(rate: 1, skipsJargon: false) - public init(rate: Double, skipsJargon: Bool) { + public init(rate: Double, skipsJargon: Bool, voiceID: String = ReadAloudVoice.defaultID) { self.rate = rate self.skipsJargon = skipsJargon + self.voiceID = voiceID } + + /// The same voice and speed, reading the text as given: for text already + /// written to be heard (an answer, a voice sample). + public var verbatim: ReadAloudStyle { ReadAloudStyle(rate: rate, skipsJargon: false, voiceID: voiceID) } } -/// Persists how read-aloud sounds in `UserDefaults`: its playback speed and -/// whether it skips jargon, two independent settings. Untouched, it reads at -/// natural pace with nothing skipped (`ReadAloudStyle.standard`). Skipping is -/// off by default because it sends the selection to the LLM Gateway before -/// reading it. The Settings window's Advanced pane writes both slots through -/// `@AppStorage`, so like `SelectionSpeechStore` this store is read-only. +/// Persists how read-aloud sounds in `UserDefaults`: its voice, its playback +/// speed and whether it skips jargon, three independent settings. Untouched, it +/// reads in the default voice at natural pace, skipping jargon: code, paths and +/// long numbers read aloud are mostly noise. That sends the selection to the LLM +/// Gateway before reading it, which the Read Aloud footer says while it's on, +/// and a failed rewrite falls back to the selection verbatim. The Keys tab's Read Aloud section writes every slot through +/// `@AppStorage`, so this store is read-only. public struct ReadAloudStyleStore { /// UserDefaults keys, public so the Settings view can bind them directly. public static var speedDefaultsKey: String { DefaultsKey.readAloudSpeed.key } public static var skipsJargonDefaultsKey: String { DefaultsKey.readAloudSkipsJargon.key } + public static var voiceDefaultsKey: String { DefaultsKey.readAloudVoice.key } /// What an unset slot reads as. Public so the Settings view's `@AppStorage` /// defaults come from here and can't disagree with the press (the trap /// `EnhancedTranscriptsStore.defaultValue` records). public static let defaultSpeed = ReadAloudStyle.standard.rate - public static let defaultSkipsJargon = ReadAloudStyle.standard.skipsJargon + /// On, unlike `ReadAloudStyle.standard` (the verbatim style a caller gets + /// when it names none). + public static let defaultSkipsJargon = true + public static let defaultVoiceID = ReadAloudStyle.standard.voiceID /// The speeds the Settings picker offers. Capped at 3: past that the /// time-domain pitch correction smears syllables together. @@ -54,7 +66,18 @@ public struct ReadAloudStyleStore { /// The style the next press reads with. public var style: ReadAloudStyle { - ReadAloudStyle(rate: speed, skipsJargon: skipsJargon) + ReadAloudStyle( + rate: speed, skipsJargon: skipsJargon, + voiceID: Self.knownVoiceID(defaults.string(forKey: Self.voiceDefaultsKey))) + } + + /// `id` when it names a catalog voice, else the default. The slot is a plain + /// string anyone can `defaults write`, and an id the service doesn't know + /// refuses the whole read (`3012`), so it is never passed through. The + /// picker's binding reads through this too, so a stale value shows as the + /// voice that will actually speak. + public static func knownVoiceID(_ id: String?) -> String { + id.flatMap(ReadAloudVoice.voice(id:))?.id ?? defaultVoiceID } /// The stored speed, snapped to what the picker offers. The slot is a plain diff --git a/Sources/BlurtEngine/Config/SelectionAskStore.swift b/Sources/BlurtEngine/Config/SelectionAskStore.swift deleted file mode 100644 index ca075d5e..00000000 --- a/Sources/BlurtEngine/Config/SelectionAskStore.swift +++ /dev/null @@ -1,37 +0,0 @@ -import Foundation - -/// Persists read-aloud's "hold to ask" switch in `UserDefaults`. While it and -/// read-aloud (`SelectionSpeechStore`) are both on, holding the trigger over -/// selected text records a spoken request instead of reading the text, and the -/// answer is read aloud (`SelectionSpeechRouting`, `SelectionSpeaker.answer`). -/// A tap over a selection still reads it. -/// -/// **Off by default**, like skip-jargon (`ReadAloudStyleStore`) and for the same -/// reason: asking sends the selection and the spoken request to the LLM -/// Gateway, a second service, so it is its own opt-in rather than riding -/// read-aloud's. The Settings window's Advanced pane writes the slot through -/// `@AppStorage`, so like `EnhancedTranscriptsStore` this store is read-only. -public struct SelectionAskStore { - /// UserDefaults key holding the switch. Public so SwiftUI views can observe - /// it directly (e.g. `@AppStorage`) and re-render on change. - public static var defaultsKey: String { DefaultsKey.selectionAsk.key } - - /// The value an unset key reads as. Public so the Settings toggle's - /// `@AppStorage` default comes from here and can't disagree with the router. - public static let defaultValue = false - - private let defaults: UserDefaults - - public init() { - self.init(defaults: .standard) - } - - init(defaults: UserDefaults) { - self.defaults = defaults - } - - /// Unset reads as `defaultValue`. - public var isEnabled: Bool { - defaults.object(forKey: Self.defaultsKey) as? Bool ?? Self.defaultValue - } -} diff --git a/Sources/BlurtEngine/Config/SelectionSpeechStore.swift b/Sources/BlurtEngine/Config/SelectionSpeechStore.swift deleted file mode 100644 index 274dea65..00000000 --- a/Sources/BlurtEngine/Config/SelectionSpeechStore.swift +++ /dev/null @@ -1,31 +0,0 @@ -import Foundation - -/// Persists the experimental "read selection aloud" switch in `UserDefaults`. -/// Off by default; the Settings window's Advanced pane flips it. While on, a -/// trigger press over selected text reads that text aloud through AssemblyAI's -/// streaming TTS (`SelectionSpeaker`) instead of starting a dictation — a press -/// with nothing selected still dictates as usual. -/// -/// Opt-in, for the reason `AssemblyAISpeechSynthesizer` gives. Same shape as -/// `DeveloperModeStore`. -public struct SelectionSpeechStore { - /// UserDefaults key holding the switch. Public so SwiftUI views can observe - /// it directly (e.g. `@AppStorage`) and re-render on change. - public static var defaultsKey: String { DefaultsKey.selectionSpeech.key } - private let defaults: UserDefaults - - public init() { - self.init(defaults: .standard) - } - - init(defaults: UserDefaults) { - self.defaults = defaults - } - - /// `bool(forKey:)` returns false for a missing key, so unset means off. - /// Read-only for the same reason as `DeveloperModeStore.isEnabled`: the - /// Settings toggle writes the slot through `@AppStorage`. - public var isEnabled: Bool { - defaults.bool(forKey: Self.defaultsKey) - } -} diff --git a/Sources/BlurtEngine/Config/StyleProfileStore.swift b/Sources/BlurtEngine/Config/StyleProfileStore.swift index 90576463..0fb98abe 100644 --- a/Sources/BlurtEngine/Config/StyleProfileStore.swift +++ b/Sources/BlurtEngine/Config/StyleProfileStore.swift @@ -55,7 +55,7 @@ public struct StyleProfile: Codable, Identifiable, Hashable, Sendable { /// documented exception for a value that is *encoded* on write (as with /// `TriggerKeyStore`'s keycode and `SoundPackStore`'s id): the JSON shape is the /// store's business, so the views write through here and merely *observe* the -/// raw slots with `@AppStorage` — see `HotkeyStepView`, where binding the raw +/// raw slots with `@AppStorage` — see `KeysSettingsTab`, where binding the raw /// slot directly left the setter test-only and a change to the encoding would /// have kept `swift test` green while the UI wrote the old form. /// diff --git a/Sources/BlurtEngine/Hotkey/DictationKeyRouter.swift b/Sources/BlurtEngine/Hotkey/DictationKeyRouter.swift index ee9dc5e4..65d60105 100644 --- a/Sources/BlurtEngine/Hotkey/DictationKeyRouter.swift +++ b/Sources/BlurtEngine/Hotkey/DictationKeyRouter.swift @@ -28,6 +28,10 @@ public struct DictationKeyRouter: Sendable { case flagsChanged(keyCode: Int, triggerFlagIsOn: Bool) /// A `keyDown` for `keyCode`. case keyDown(keyCode: Int) + /// A mouse button went down. A modifier held through a click is a + /// modifier-click (⌘-click a link, ⌥-click in Finder), not a lone press, so + /// it cancels the press exactly as another key would. + case pointerDown /// A `flagsChanged` delivery from its keycode and the event's raw /// `CGEventFlags`: down when the flags hold *that* key's device bit — so @@ -38,8 +42,9 @@ public struct DictationKeyRouter: Sendable { } } - /// The virtual keycodes of the bound trigger modifiers (`TriggerKey.keyCode`). - private(set) var triggerKeyCodes: Set + /// Each watched trigger's virtual keycode (`TriggerKey.keyCode`) and the + /// activation mode its presses resolve under. + private(set) var activations: [Int: TriggerActivation] /// The trigger that owns the gate — the one whose press started the current /// dictation — or nil while the gate is idle and no trigger is held. The host @@ -51,12 +56,10 @@ public struct DictationKeyRouter: Sendable { /// deliveries with an unchanged bit don't re-fire the gate. private var modifierIsDown = false - public init( - triggerKeyCodes: Set, holdThreshold: Duration = .seconds(1), - activation: TriggerActivation = .tapOrHold - ) { - self.triggerKeyCodes = triggerKeyCodes - self.gate = DictationKeyGate(holdThreshold: holdThreshold, activation: activation) + /// Each key with its own mode (keycode → activation). + public init(activations: [Int: TriggerActivation], holdThreshold: Duration = .seconds(1)) { + self.activations = activations + self.gate = DictationKeyGate(holdThreshold: holdThreshold, activation: activations.values.first ?? .tapOrHold) } /// Feeds one event through the relevance/edge filters into the gate and @@ -64,10 +67,16 @@ public struct DictationKeyRouter: Sendable { public mutating func handle(_ event: Event, at now: Duration) -> DictationKeyGate.Action { switch event { case .flagsChanged(let keyCode, let triggerFlagIsOn): - guard triggerKeyCodes.contains(keyCode), (activeKeyCode ?? keyCode) == keyCode else { return .none } + guard activations[keyCode] != nil, (activeKeyCode ?? keyCode) == keyCode else { return .none } if triggerFlagIsOn, !modifierIsDown { modifierIsDown = true activeKeyCode = keyCode + // A press from idle resolves under its own key's mode. The gate's mode + // is `let` (its determinism contract), so a different one is a fresh gate; + // an idle gate holds nothing to lose. + if gate.isIdle, let activation = activations[keyCode], activation != gate.activation { + gate = DictationKeyGate(holdThreshold: gate.holdThreshold, activation: activation) + } return gate.modifierDown(at: now) } if !triggerFlagIsOn, modifierIsDown { @@ -78,26 +87,29 @@ public struct DictationKeyRouter: Sendable { } return .none case .keyDown(let keyCode): - return triggerKeyCodes.contains(keyCode) ? .none : gate.otherKeyDown() + return activations[keyCode] != nil ? .none : gate.otherKeyDown() + case .pointerDown: + return gate.otherKeyDown() } } - /// Rebinds the trigger — keys and activation mode, the two halves of the - /// binding Settings can change — and resets when the change touches what the - /// gate is tracking: the active key unbound (its up-event can no longer match) - /// or the mode changed (it would resolve under the old one). The gate is - /// rebuilt then because its `activation` is `let`, part of its determinism - /// contract. Turning some *other* key on or off in Key Styles keeps the state, - /// so a recording latched while Settings is open survives it. Returns whether - /// a reset discarded a live recording (see `reset()`). + /// Rebinds the triggers — which keys, and each one's mode, the two halves of + /// the binding Settings can change — and resets only when the change touches + /// what the gate is tracking: the active key unbound (its up-event can no + /// longer match) or its mode changed (it would resolve under the old one). + /// Any other key's change keeps the state, so a recording latched while + /// Settings is open survives it; that key's next press picks up its mode. + /// Returns whether a reset discarded a live recording (see `reset()`). @discardableResult - public mutating func rebind(triggerKeyCodes: Set, activation: TriggerActivation) -> Bool { - self.triggerKeyCodes = triggerKeyCodes - let activeKeyStillBound = activeKeyCode.map(triggerKeyCodes.contains) ?? true - guard !activeKeyStillBound || activation != gate.activation else { return false } - let discardedRecording = reset() - gate = DictationKeyGate(holdThreshold: gate.holdThreshold, activation: activation) - return discardedRecording + public mutating func rebind(activations: [Int: TriggerActivation]) -> Bool { + self.activations = activations + guard let activeKeyCode else { return false } + // An unbound active key resets even with the gate idle (a combo cancelled + // its press, the key still down): its up-event can no longer match, so it + // would hold `activeKeyCode` and lock every other key out. + guard let activation = activations[activeKeyCode] else { return reset() } + guard !gate.isIdle, activation != gate.activation else { return false } + return reset() } /// Recovery after the host's event tap was disabled (by timeout, or by user input @@ -115,7 +127,7 @@ public struct DictationKeyRouter: Sendable { /// the dropped events, so the gate is reset. /// /// Returns whether that reset discarded a live recording the caller must cancel - /// upstream, matching `reset()` and `rebind(triggerKeyCodes:activation:)`. This lived in the + /// upstream, matching `reset()` and `rebind(activations:)`. This lived in the /// shell as a bare `if`, where nothing could test it — the app target has no test /// target and a `CGEventTap` can't be driven from XCUITest — while carrying the /// worst failure of the three decisions here: a session left in `.recording` with diff --git a/Sources/BlurtEngine/Hotkey/SelectionSpeechRouter.swift b/Sources/BlurtEngine/Hotkey/SelectionSpeechRouter.swift new file mode 100644 index 00000000..d94499e9 --- /dev/null +++ b/Sources/BlurtEngine/Hotkey/SelectionSpeechRouter.swift @@ -0,0 +1,204 @@ +import Foundation + +#if os(macOS) + + /// Sits between the trigger keys and the dictation session. A press of a key + /// set to read aloud reads the selected text; a press of a key set to ask + /// records a spoken request about it and reads the answer; a dictating key's + /// commands go straight to `DictationSession.submit` (see `TriggerKeyBindings` + /// for which key does what). + /// + /// The decisions are the engine's `SelectionSpeechRouting`. This class carries + /// out the actions it returns: the selection read, the hold timer, the speech + /// task, and the gate resets. Only read and ask presses read the selection, so + /// dictation never waits on an Accessibility round trip. + /// + /// Main-actor because its callers are: the key tap's callbacks, the session's + /// phase observer and Settings all run on the main thread, and isolation lets + /// the compiler prove single-threaded access to the routing state. + @MainActor + public final class SelectionSpeechRouter { + private static let logger = HostIdentity.current.logger("SelectionSpeech") + + private let session: DictationSession + private let speaker: any ReadAloudSpeaker + private let readSelection: @Sendable () async -> String? + private let defaults: UserDefaults + private var routing = SelectionSpeechRouting() + private var speech: Task? + /// The pending `holdElapsed`, restarted by each press that may ask. + private var holdTimer: Task? + /// A voice sample playing from Settings. Apart from `speech`, so it never + /// touches the routing state a key press owns. + private var preview: Task? + /// The key tap's `syncAfterTerminalPhase`: a dictation ended without a key event. + public var syncGateAfterSession: () -> Void = {} + /// The key tap's `resetGate`: a read ended without a key event. + public var resetGate: () -> Void = {} + /// A read or answer began (the pill appears, the start cue plays) and ended + /// (the pill goes, the stop cue plays): once each per read, whether it ran + /// out, was stopped, or failed. + public var speechStarted: () -> Void = {} + public var speechEnded: () -> Void = {} + /// The loudness of the voice playing, ~20 times a second: the pill's meter. + public var speechLevel: (Float) -> Void = { _ in } + /// A read, answer or voice sample failed (it's already logged): the app + /// beeps, since the pill just goes away. + public var speechFailed: (any Error) -> Void = { _ in } + /// Counts reads, so a stopped read's late end can't take down a newer read's pill. + private var speechGeneration = 0 + /// The read the pill is showing, or nil once it ended: a level still on its + /// way from a read that's over must not land on the pill. + private var activeGeneration: Int? + + public convenience init(session: DictationSession) { + self.init(session: session, speaker: SelectionSpeaker(), readSelection: SelectionSpeaker.focusedSelection) + } + + /// The seams: what speaks, how the selection is read, and where the + /// settings each press reads come from. Public so a host can run the whole + /// read-aloud path offline (Blurt's UI tests do). + public init( + session: DictationSession, speaker: any ReadAloudSpeaker, + readSelection: @escaping @Sendable () async -> String?, defaults: UserDefaults = .standard + ) { + self.session = session + self.speaker = speaker + self.readSelection = readSelection + self.defaults = defaults + } + + /// A trigger key went down. What it does and whether it can be held are read + /// per press, so a change in the Keys tab applies to the next one. + public func press(key: TriggerKey) { + let bindings = TriggerKeyBindings(defaults: defaults) + let action = bindings.action(for: key) + let holds = bindings.activation(for: key) != .tap + perform(routing.press(key: key, action: action, holds: holds)) + } + + /// Speaks a short sample in the chosen voice at the chosen speed, as picking + /// a cue sound plays it. A newer pick cuts off the one still playing. + public func previewSample() { + let style = ReadAloudStyleStore(defaults: defaults).style + let name = ReadAloudVoice.voice(id: style.voiceID)?.name ?? "your voice" + let sample = style.verbatim + preview?.cancel() + preview = Task { [weak self, speaker] in + do { + let line = "Hi, I'm \(name). This is how Blurt reads your selection." + try await speaker.speak(line, style: sample, onLevel: nil) + } catch is CancellationError { + } catch { + Self.logger.error("voice preview failed: \(error.localizedDescription, privacy: .public)") + self?.speechFailed(error) + } + } + } + + /// The gate's other commands: `.release`, `.cancel`, `.cancelRecording`. + public func submit(_ command: DictationSession.Command) { + perform(routing.submit(command)) + } + + /// `DictationKeyTap`'s latch: the press was a tap. + public func keyLatched() { + perform(routing.keyLatched()) + } + + public func sessionPhaseChanged(_ phase: PipelinePhase) { + perform(routing.sessionPhaseChanged(phase)) + } + + private func perform(_ actions: [SelectionSpeechRouting.Action]) { + for action in actions { + switch action { + case .forward(let command): session.submit(command) + case .readSelection: + Task { [weak self] in + let selection = await self?.readSelection() + guard let self else { return } + self.perform(self.routing.selectionResolved(selection)) + } + case .startHoldTimer: startHoldTimer() + case .speak(let text): + // Read per press, like the switch itself, so a Settings change applies + // to the next read. + let style = ReadAloudStyleStore(defaults: defaults).style + run { [speaker] onLevel in try await speaker.speak(text, style: style, onLevel: onLevel) } + case .answer(let request, let selection): + let style = ReadAloudStyleStore(defaults: defaults).style + run { [speaker] onLevel in + try await speaker.answer(request, about: selection, style: style, onLevel: onLevel) + } + case .stopSpeech: + speech?.cancel() + speech = nil + // Ended now, not when the cancelled task gets round to it: a + // dictation the same press starts must not have its pill taken down + // by this read's late end. + endSpeech() + case .syncGateAfterSession: syncGateAfterSession() + case .resetGate: resetGate() + } + } + } + + /// One timer at a time: a newer press cancels the older one's, so a timer + /// left over from a quick tap can't mark the next press as a hold. + private func startHoldTimer() { + holdTimer?.cancel() + holdTimer = Task { [weak self] in + try? await Task.sleep(for: SelectionSpeechRouting.holdDelay) + guard !Task.isCancelled, let self else { return } + self.holdTimer = nil + self.perform(self.routing.holdElapsed()) + } + } + + /// Whether a read or answer is playing, wearing the pill — what + /// `PipelinePhase.overlayState(whileSpeaking:)` asks. + public var isSpeaking: Bool { activeGeneration != nil } + + /// Reports the read the pill is showing as over, once: the meter to 0, then + /// `speechEnded`. + private func endSpeech() { + guard activeGeneration != nil else { return } + activeGeneration = nil + speechLevel(0) + speechEnded() + } + + /// Runs one read or answer as the current speech task, handing it the meter + /// callback, and reports its start and end. + private func run( + _ work: @escaping @Sendable (_ onLevel: @escaping @Sendable (Float) -> Void) async throws -> Void + ) { + speechGeneration += 1 + let generation = speechGeneration + activeGeneration = generation + speechStarted() + let onLevel: @Sendable (Float) -> Void = { [weak self] level in + Task { @MainActor in + guard let self, self.activeGeneration == generation else { return } + self.speechLevel(level) + } + } + speech = Task { [weak self] in + do { + try await work(onLevel) + } catch is CancellationError { + } catch { + Self.logger.error("read-aloud failed: \(error.localizedDescription, privacy: .public)") + self?.speechFailed(error) + } + if let self, self.activeGeneration == generation { self.endSpeech() } + // A read this router stopped was cancelled, and a newer one may already + // be playing, so only a read that ended on its own reports back. + guard !Task.isCancelled, let self else { return } + self.speech = nil + self.perform(self.routing.speechFinished()) + } + } + } +#endif diff --git a/Sources/BlurtEngine/Hotkey/SelectionSpeechRouting.swift b/Sources/BlurtEngine/Hotkey/SelectionSpeechRouting.swift index 59295d51..c483c0b1 100644 --- a/Sources/BlurtEngine/Hotkey/SelectionSpeechRouting.swift +++ b/Sources/BlurtEngine/Hotkey/SelectionSpeechRouting.swift @@ -1,43 +1,42 @@ -/// Decides where each trigger command goes while the experimental read-aloud -/// switch is on: to the dictation session, to reading the selection aloud, or -/// to asking about the selection. It is a pure state machine. The app's -/// `SelectionSpeechRouter` carries out the actions it returns (the AX read, the -/// hold timer, the speech task, the gate resets), the same split as -/// `DictationKeyRouter` under `DictationKeyTap`. +/// Decides what each trigger command does, by the action of the key that +/// pressed (`TriggerKeyAction`): a dictating key's commands go straight to the +/// dictation session; a read-aloud key reads the selection aloud; an ask key +/// records a spoken request about the selection and reads the answer. It is a +/// pure state machine. The app's `SelectionSpeechRouter` carries out the +/// actions it returns (the AX read, the hold timer, the speech task, the gate +/// resets), the same split as `DictationKeyRouter` under `DictationKeyTap`. /// -/// **Deciding** between dictating and speaking needs an Accessibility read of -/// the selection, so a press first waits in `.deciding`. Commands arriving in the -/// meantime (a quick release, a ⌘C combo's cancel) are held and replayed in order -/// once the read returns. That keeps the session's FIFO order either way. +/// **A dictating key never waits on the selection.** Only a read or ask press +/// reads it, so dictation's recording starts at the press as it always has. /// -/// **Speech follows the gate's tap/hold semantics, like dictation.** A tap -/// latches the gate, so speech runs until the next tap. A hold speaks until the -/// release. A combo that cancels the press cancels the speech, because right-⌘ + C -/// over a selection is a copy, not a request to hear it. Any key command ends a -/// read that is playing. +/// **Read and ask follow their key's tap/hold mode, like dictation**, with one +/// difference: nothing starts at the key-down. A modifier going down over a +/// selection is as often the start of ⌘C as a request to hear it, so the read +/// (or the mic, for an ask) waits for the press to prove itself: a tap (the +/// gate's `.latch`), or the key still down `holdDelay` later in a mode that +/// holds. A combo or a click before then (`.cancel`) did nothing. After that: +/// - A tap's read frees the gate and plays until it ends or any key is +/// pressed: the dictation key then dictates. A hold's plays until the key is +/// let go. +/// - A tapped ask records until the next tap, as a dictation does; a held ask +/// until the key is let go. Then the transcript comes back as `.handedBack` +/// and the answer is read. /// -/// **With asking on (`SelectionAskStore`), a hold over a selection asks about -/// it instead.** A tap still reads: the gate's `.latch` is the moment a tap is -/// known (`keyLatched`). A key still down `askHoldDelay` after the press is a -/// hold, and only then does the mic open, through the session's hand-back press, -/// so a tap never pays for a recording. (Opening the mic is not free: on AirPods -/// it switches the link to its headset mode, which the read would then play -/// through.) Letting go ends the request, whether the gate saw that as a hold's -/// stop or a short hold's latch. The transcript comes back as `.handedBack`, and -/// the answer is read aloud. A request with nothing said reads the selection -/// instead, which is also how a hold-only trigger still reads at all. +/// The Accessibility read is in flight while those signals land, so a press +/// first waits in `.deciding`, noting them, and acts once the read returns. +/// With nothing selected the press does nothing. /// /// **This owns every "the gate is stale" decision.** A dictation that ends /// without a key event leaves the gate latched, and so does a read that ends on /// its own. Each asks for a different reset: `.syncGateAfterSession` goes through /// the tap's recovery path, while `.resetGate` just clears it, because no /// recording is involved. -public struct SelectionSpeechRouting: Sendable { - public enum Action: Sendable, Equatable { +struct SelectionSpeechRouting: Sendable { + enum Action: Sendable, Equatable { case forward(DictationSession.Command) /// Start the Accessibility read. Its result comes back through `selectionResolved`. case readSelection - /// Call `holdElapsed` after `askHoldDelay`, unless a newer press restarts it. + /// Call `holdElapsed` after `holdDelay`, unless a newer press restarts it. case startHoldTimer case speak(String) /// Answer the spoken `request` about `selection`, aloud. @@ -49,23 +48,37 @@ public struct SelectionSpeechRouting: Sendable { case resetGate } - /// How long the key must stay down over a selection to count as a hold. Well - /// under the gate's 1 s tap/hold threshold, because the user waits this long - /// (plus the mic bring-up) for the start chime before speaking; a deliberate - /// tap is over long before it. - public static let askHoldDelay = Duration.milliseconds(350) + /// How long a read or ask key must stay down, in a mode that holds, before + /// the press counts. Well under the gate's 1 s tap/hold threshold, because an + /// asker waits this long (plus the mic bring-up) for the start chime before + /// speaking; a deliberate tap is over long before it. + static let holdDelay = Duration.milliseconds(350) + + private enum Intent: Sendable, Equatable { + case read, ask + } + + /// How a read or ask press proved itself: a tap (the gate latched), or a + /// hold (still down at `holdDelay`). + private enum Trigger: Sendable, Equatable { + case tap, hold + } private enum Mode: Sendable, Equatable { case passthrough - /// The selection read is in flight. For a press that may ask, the tap and - /// hold signals that land meanwhile are kept too. - case deciding(pending: [DictationSession.Command], asks: Bool, latched: Bool, held: Bool) - /// A selection is in hand and the key is still down: a tap reads it, a hold asks. - case selected(String) + /// The selection read is in flight. `trigger`: how the press already + /// proved itself, if it has. `ended`: the key let go or a combo cancelled + /// it; the gate is idle. + case deciding(Intent, trigger: Trigger?, ended: Bool) + /// A selection is in hand, and the key is down but hasn't proved itself yet. + case selected(Intent, String) /// Recording the spoken request about the selection. Not `started` while /// an earlier dictation is still finishing: the hand-back press waits for /// it, since the session refuses a press until it is back at rest. - case asking(String, started: Bool) + /// `byHold`: the key is held for the request, so letting it go — even + /// before the gate's own hold threshold — ends it; a tapped request runs + /// until the next tap, like a dictation. + case asking(String, started: Bool, byHold: Bool) /// The request was let go of and is being transcribed (or, not `started`, /// was let go of before the mic ever opened). case transcribing(String, started: Bool) @@ -76,48 +89,71 @@ public struct SelectionSpeechRouting: Sendable { /// Whether the session's last phase was in flight. A terminal phase is then /// that dictation's end, never a hand-back press that hasn't started. private var sessionBusy = false - /// The trigger key of the press a read is deciding over, so the dictation it - /// becomes — the press itself with no selection, or a request about one — is - /// made with that key's style. - private var decidingKey: TriggerKey? - public init() {} + init() {} - public mutating func submit( - _ command: DictationSession.Command, readAloudEnabled: Bool, askEnabled: Bool = false + /// A trigger key went down from idle. `holds`: its mode lets a press be a + /// hold (anything but tap-only), so a read or ask starts the hold timer. + mutating func press( + key: TriggerKey, action: TriggerKeyAction, holds: Bool ) -> [Action] { switch mode { case .passthrough: - guard case .press(let key) = command, readAloudEnabled else { return [.forward(command)] } - decidingKey = key - mode = .deciding(pending: [], asks: askEnabled, latched: false, held: false) - return askEnabled ? [.readSelection, .startHoldTimer] : [.readSelection] - case .deciding(var pending, let asks, let latched, let held): - pending.append(command) - mode = .deciding(pending: pending, asks: asks, latched: latched, held: held) + switch action { + // Off only reaches here when Settings turned the key off mid-press: + // dictate, as it was bound to. + case .dictate, .off: return [.forward(.press(key: key))] + case .readAloud, .ask: + mode = .deciding(action == .ask ? .ask : .read, trigger: nil, ended: false) + return holds ? [.readSelection, .startHoldTimer] : [.readSelection] + } + case .speaking: + // A press only arrives here when the gate was idle under the speech: an + // answer, or a tapped read, which frees the gate as it starts. Any key + // stops it. A dictating key then dictates, as pressed; any other key's + // press armed the gate, and its key-up would latch something nobody + // started, swallowing the next press, so clear it instead. + mode = .passthrough + switch action { + case .dictate, .off: return [.stopSpeech, .forward(.press(key: key))] + case .readAloud, .ask: return [.stopSpeech, .resetGate] + } + case .transcribing(_, let started): + // A press while the request is transcribing drops it, for the same reason. + mode = .passthrough + return started ? [.forward(.cancel), .resetGate] : [.resetGate] + case .deciding, .selected, .asking: + // The gate belongs to the key down now; another press can't start. + return [] + } + } + + /// A gate command other than a press: `.release` (the gate's stop), + /// `.cancel` (a combo), or `.cancelRecording` (a recovery reset). + mutating func submit(_ command: DictationSession.Command) -> [Action] { + switch mode { + case .passthrough: + return [.forward(command)] + case .deciding(let intent, let trigger, _): + if command == .cancelRecording { + mode = .passthrough + return [.forward(command)] + } + mode = .deciding(intent, trigger: trigger, ended: true) return [] case .selected: - // Let go before it was a hold (a hold-only trigger's quick release), or a - // combo over the selection. Neither asked for anything. + // Let go, or a combo, before the press proved itself: nothing asked for. mode = .passthrough return command == .cancelRecording ? [.forward(command)] : [] - case .asking(let selection, let started): + case .asking(let selection, let started, _): return asking(selection, started: started, command) case .transcribing(_, let started): return transcribing(started: started, command) case .speaking: mode = .passthrough - switch command { // A recovery reset still reaches the session, where it can only cancel a // live recording, and there isn't one. - case .cancelRecording: return [.stopSpeech, .forward(command)] - // A press only arrives here when the gate was idle under the speech: an - // answer, or a read after a request with nothing said. The press armed - // the gate, and its key-up would latch a recording nobody started, - // swallowing the next press. Clear it now instead. - case .press, .pressHandingBack: return [.stopSpeech, .resetGate] - case .release, .cancel: return [.stopSpeech] - } + return command == .cancelRecording ? [.stopSpeech, .forward(command)] : [.stopSpeech] } } @@ -134,97 +170,101 @@ public struct SelectionSpeechRouting: Sendable { // a cancel would throw away. A recovery reset still goes through. return started || command == .cancelRecording ? [.forward(command)] : [] case .press, .pressHandingBack: - // The key is down for the whole request, so the gate can't press again. return [] } } private mutating func transcribing(started: Bool, _ command: DictationSession.Command) -> [Action] { switch command { - case .press, .pressHandingBack: - // A press while the request is transcribing drops it. The gate is armed - // for this press, and its key-up would latch a recording nobody started. - mode = .passthrough - return started ? [.forward(.cancel), .resetGate] : [.resetGate] case .cancel: mode = .passthrough return started ? [.forward(.cancel)] : [] case .release, .cancelRecording: // Nothing is recording, so the session treats either as a no-op. return [.forward(command)] + case .press, .pressHandingBack: + return [] } } - /// The gate latched: the key came up quickly enough to be a tap. - public mutating func keyLatched() -> [Action] { + /// The gate latched: the key came up quickly enough to be a tap (or, in + /// tap-only mode, came up at all). + mutating func keyLatched() -> [Action] { switch mode { - case .deciding(let pending, let asks, _, let held): - mode = .deciding(pending: pending, asks: asks, latched: true, held: held) + case .deciding(let intent, let trigger, let ended): + // Held past the delay, then let go before the gate's own threshold: a + // short hold. A held ask ends there, having asked nothing yet; a read + // keeps going, as a tapped one does. + if trigger == .hold, intent == .ask { + mode = .deciding(intent, trigger: trigger, ended: true) + } else { + mode = .deciding(intent, trigger: .tap, ended: ended) + } return [] - case .selected(let selection): - mode = .speaking - return [.speak(selection)] - case .asking(let selection, let started): - // Let go before the gate's own hold threshold, so it latched. For a - // request, letting go is the end of it, and the latch holds no recording - // the user knows about. + case .selected(let intent, let selection): + return self.trigger(intent, selection, by: .tap) + case .asking(let selection, let started, byHold: true): + // Letting go of a held request ends it, before the gate's hold + // threshold too; the latch holds no recording the user knows about. mode = .transcribing(selection, started: started) return started ? [.forward(.release), .resetGate] : [.resetGate] - case .passthrough, .transcribing, .speaking: + case .speaking: + // A held read let go before the gate's threshold keeps playing, latched. + // Free the gate, as a tapped read does, so the other keys still work. + return [.resetGate] + case .passthrough, .asking, .transcribing: return [] } } - /// `askHoldDelay` passed since the press. Meaningful only while the key may + /// `holdDelay` passed since the press. Meaningful only while the key may /// still be down over a selection. - public mutating func holdElapsed() -> [Action] { + mutating func holdElapsed() -> [Action] { switch mode { - case .deciding(let pending, true, let latched, _): - mode = .deciding(pending: pending, asks: true, latched: latched, held: true) + case .deciding(let intent, nil, false): + mode = .deciding(intent, trigger: .hold, ended: false) return [] - case .selected(let selection): - return beginAsking(selection) + case .selected(let intent, let selection): + return trigger(intent, selection, by: .hold) case .passthrough, .deciding, .asking, .transcribing, .speaking: return [] } } - public mutating func selectionResolved(_ selection: String?) -> [Action] { - guard case .deciding(let pending, let asks, let latched, let held) = mode else { return [] } - guard let selection else { + mutating func selectionResolved(_ selection: String?) -> [Action] { + guard case .deciding(let intent, let trigger, let ended) = mode else { return [] } + guard let selection, !ended else { mode = .passthrough - return ([.press(key: decidingKey)] + pending).map(Action.forward) + // Nothing selected, or the press already over: nothing to do. A press + // still holding the gate (down, or latched by a tap) would otherwise + // swallow the key's next press, so clear it. + return ended ? [] : [.resetGate] } - // The press was already over before the read returned: a combo cancelled - // it, or a hold was released. Neither one asked to hear anything. When a - // later press is among them it still holds the gate, armed or latched, and - // nothing will run for it, so clear the gate or the next press is swallowed. - guard pending.isEmpty else { - mode = .passthrough - guard case .press? = pending.last else { return [] } - return [.resetGate] - } - // A tap, or a press that can't ask, reads now. - if !asks || latched { - mode = .speaking - return [.speak(selection)] - } - if held { return beginAsking(selection) } - mode = .selected(selection) + if let trigger { return self.trigger(intent, selection, by: trigger) } + mode = .selected(intent, selection) return [] } - /// Opens the mic for a request, through the session's hand-back press — or, - /// while an earlier dictation is still finishing, once it has. - private mutating func beginAsking(_ selection: String) -> [Action] { - mode = .asking(selection, started: !sessionBusy) - return sessionBusy ? [] : [.forward(.pressHandingBack(key: decidingKey))] + private mutating func trigger(_ intent: Intent, _ selection: String, by trigger: Trigger) -> [Action] { + switch intent { + case .read: + mode = .speaking + // A tapped read frees the gate at once: a read can run for many + // minutes, and the read key holding the gate all that time would leave + // every other key — dictation's among them — dead until it ended. A + // held read keeps it, so letting go stops the read. + return trigger == .tap ? [.speak(selection), .resetGate] : [.speak(selection)] + case .ask: + mode = .asking(selection, started: !sessionBusy, byHold: trigger == .hold) + return sessionBusy ? [] : [.forward(.pressHandingBack(key: nil))] + } } /// The read ended on its own, or failed. The caller must not report a read it /// stopped itself: a key stop has already moved on, and a newer read may be - /// playing. - public mutating func speechFinished() -> [Action] { + /// playing. The read's key may still hold the gate (latched by a tap), so + /// clear it, or its next press would only stop a read that's over. + mutating func speechFinished() -> [Action] { guard mode == .speaking else { return [] } mode = .passthrough return [.resetGate] @@ -237,20 +277,20 @@ public struct SelectionSpeechRouting: Sendable { /// transcript (`.handedBack`), nothing said (`.idle`), or a failure — and /// only once its own press has started; before that, a terminal phase is the /// earlier dictation's end, which lets the request start. - public mutating func sessionPhaseChanged(_ phase: PipelinePhase) -> [Action] { + mutating func sessionPhaseChanged(_ phase: PipelinePhase) -> [Action] { sessionBusy = !phase.isTerminal guard phase.isTerminal else { return [] } switch mode { case .passthrough: return [.syncGateAfterSession] - case .asking(let selection, started: false): - mode = .asking(selection, started: true) - return [.forward(.pressHandingBack(key: decidingKey))] - case .transcribing(let selection, started: false): - // Let go before the mic ever opened: a request with nothing said. - mode = .speaking - return [.speak(selection)] - case .asking(let selection, started: true): + case .asking(let selection, started: false, let byHold): + mode = .asking(selection, started: true, byHold: byHold) + return [.forward(.pressHandingBack(key: nil))] + case .transcribing(_, started: false): + // Let go before the mic ever opened: nothing was asked. + mode = .passthrough + return [] + case .asking(let selection, started: true, _): return requestEnded(phase, selection: selection, stillDown: true) case .transcribing(let selection, started: true): return requestEnded(phase, selection: selection, stillDown: false) @@ -262,16 +302,13 @@ public struct SelectionSpeechRouting: Sendable { private mutating func requestEnded(_ phase: PipelinePhase, selection: String, stillDown: Bool) -> [Action] { if case .handedBack(let request) = phase { mode = .speaking - // Handed back with the key still down means the session's recording cap - // released it. Clear the gate, or the eventual key-up would stop the answer. + // Handed back with the key still down (or latched) means the session's + // recording cap released it. Clear the gate, or the eventual key-up + // would stop the answer. return [.answer(request: request, selection: selection)] + (stillDown ? [.resetGate] : []) } - if phase == .idle, !stillDown { - mode = .speaking - return [.speak(selection)] - } - // The request failed or was cancelled: a dictation ending without a key - // event, like any other. + // Nothing said, or the request failed or was cancelled: a dictation ending + // without a key event, like any other. mode = .passthrough return [.syncGateAfterSession] } diff --git a/Sources/BlurtEngine/Hotkey/TriggerActivation.swift b/Sources/BlurtEngine/Hotkey/TriggerActivation.swift index 81758521..bdcc49c2 100644 --- a/Sources/BlurtEngine/Hotkey/TriggerActivation.swift +++ b/Sources/BlurtEngine/Hotkey/TriggerActivation.swift @@ -32,17 +32,21 @@ public enum TriggerActivation: String, CaseIterable, Sendable { } } - /// The Shortcut section's footer sentence: how to dictate under this mode. - /// Lives here rather than at the SwiftUI call site so the wording stays - /// beside the behavior it describes (and inside the tested target). - public var guidance: String { + /// How to use a key under this mode, for what it does: the ready screen's + /// key rows ("Tap to start and stop. Hold to talk."). Lives here rather than + /// at the SwiftUI call site so the wording stays beside the behavior it + /// describes (and inside the tested target). + public func hint(for action: TriggerKeyAction) -> String { + let verb = + switch action { + case .readAloud: "listen" + case .ask: "ask" + case .dictate, .off: "talk" + } switch self { - case .tapOrHold: - return "Tap to start and tap again to stop, or hold the key and release to dictate." - case .tap: - return "Tap to start and tap again to stop dictating." - case .hold: - return "Hold the key while you speak and release to stop dictating." + case .tapOrHold: return "Tap to start and stop. Hold to \(verb)." + case .tap: return "Tap to start and stop." + case .hold: return "Hold to \(verb). Release to stop." } } @@ -60,10 +64,6 @@ public enum TriggerActivation: String, CaseIterable, Sendable { /// to finish." — only the gestures this mode actually stops on. public var finishHint: String { "\(stopGesture) to finish." } - /// How to stop something the trigger started that isn't a dictation, today - /// a read-aloud (see `SelectionSpeechRouting`): "Tap again or release to stop." - public var stopHint: String { "\(stopGesture) to stop." } - private var stopGesture: String { switch self { case .tapOrHold: "Tap again or release" diff --git a/Sources/BlurtEngine/Hotkey/TriggerActivationStore.swift b/Sources/BlurtEngine/Hotkey/TriggerActivationStore.swift index 33b107b7..55b5265f 100644 --- a/Sources/BlurtEngine/Hotkey/TriggerActivationStore.swift +++ b/Sources/BlurtEngine/Hotkey/TriggerActivationStore.swift @@ -1,27 +1,10 @@ -import Foundation - -/// Persists the chosen `TriggerActivation` as its raw value in `UserDefaults`. -/// Defaults to tap-or-hold (the shipped behavior) when unset or when the stored -/// value isn't one of the known modes. +/// The shared tap/hold mode's slot: the one setting there was before each key +/// had its own (`TriggerKeyActivationStore`), and still the mode of any key +/// without one. Read through `TriggerKeyBindings`, which decodes it with +/// `TriggerActivation.fromPersisted` (unset or unknown is tap-or-hold); no +/// screen writes it any more, but an existing install's value is honoured. public struct TriggerActivationStore { - /// UserDefaults key holding the activation mode. Public so SwiftUI views can + /// UserDefaults key holding the shared mode. Public so SwiftUI views can /// observe it directly (e.g. `@AppStorage`) and re-render on change. public static var defaultsKey: String { DefaultsKey.triggerActivation.key } - private let defaults: UserDefaults - - public init(defaults: UserDefaults = .standard) { - self.defaults = defaults - } - - public var activation: TriggerActivation { - get { - // Unset reads as nil → "", which isn't a known raw value, so - // `fromPersisted`'s tap-or-hold fallback covers both "never set" and - // "unknown value". - TriggerActivation.fromPersisted(defaults.string(forKey: Self.defaultsKey) ?? "") - } - nonmutating set { - defaults.set(newValue.rawValue, forKey: Self.defaultsKey) - } - } } diff --git a/Sources/BlurtEngine/Hotkey/TriggerKey.swift b/Sources/BlurtEngine/Hotkey/TriggerKey.swift index 8bfd5e9b..e1e61d01 100644 --- a/Sources/BlurtEngine/Hotkey/TriggerKey.swift +++ b/Sources/BlurtEngine/Hotkey/TriggerKey.swift @@ -7,6 +7,11 @@ public enum TriggerKey: Int, CaseIterable, Sendable, Hashable { case rightCommand = 54 case rightOption = 61 case function = 63 + // The left-hand keys: off by default, except left ⌥, which reads the selection + // aloud (`TriggerKeyBindings.defaultChoice`). Any key can take any action. + case leftCommand = 55 + case leftOption = 58 + case leftControl = 59 public var keyCode: Int { rawValue } @@ -35,6 +40,9 @@ public enum TriggerKey: Int, CaseIterable, Sendable, Hashable { case .rightCommand: return 0x10 // NX_DEVICERCMDKEYMASK case .rightOption: return 0x40 // NX_DEVICERALTKEYMASK case .function: return 0x80_0000 // kCGEventFlagMaskSecondaryFn + case .leftCommand: return 0x08 // NX_DEVICELCMDKEYMASK + case .leftOption: return 0x20 // NX_DEVICELALTKEYMASK + case .leftControl: return 0x01 // NX_DEVICELCTLKEYMASK } } @@ -44,6 +52,9 @@ public enum TriggerKey: Int, CaseIterable, Sendable, Hashable { case .rightCommand: return "right ⌘" case .rightOption: return "right ⌥" case .function: return "fn" + case .leftCommand: return "left ⌘" + case .leftOption: return "left ⌥" + case .leftControl: return "left ⌃" } } @@ -55,6 +66,9 @@ public enum TriggerKey: Int, CaseIterable, Sendable, Hashable { case .rightCommand: return "Right Command (⌘)" case .rightOption: return "Right Option (⌥)" case .function: return "Function (fn)" + case .leftCommand: return "Left Command (⌘)" + case .leftOption: return "Left Option (⌥)" + case .leftControl: return "Left Control (⌃)" } } } diff --git a/Sources/BlurtEngine/Hotkey/TriggerKeyActivationStore.swift b/Sources/BlurtEngine/Hotkey/TriggerKeyActivationStore.swift new file mode 100644 index 00000000..cbf56ce9 --- /dev/null +++ b/Sources/BlurtEngine/Hotkey/TriggerKeyActivationStore.swift @@ -0,0 +1,25 @@ +import Foundation + +/// Persists whether each trigger key is tapped, held, or either — the Keys +/// tab's second pop-up — as keycode → `TriggerActivation` raw value +/// (`TriggerKeyMap`), the bound key's included. Read through +/// `TriggerKeyBindings`: a key with no entry follows `TriggerActivationStore`'s +/// shared mode (the one setting there was before keys had their own), so an +/// existing install reads unchanged, and moving the binding moves nothing. +public struct TriggerKeyActivationStore { + /// Public so SwiftUI views can observe it directly (e.g. `@AppStorage`). + public static var defaultsKey: String { DefaultsKey.triggerKeyActivations.key } + + private let defaults: UserDefaults + + public init(defaults: UserDefaults = .standard) { + self.defaults = defaults + } + + /// Sets `key`'s mode. + public func setActivation(_ activation: TriggerActivation, for key: TriggerKey) { + var entries = TriggerKeyMap.decode(defaults.string(forKey: Self.defaultsKey) ?? "") + entries[key] = activation.rawValue + TriggerKeyMap.write(entries, to: Self.defaultsKey, in: defaults) + } +} diff --git a/Sources/BlurtEngine/Hotkey/TriggerKeyBindings.swift b/Sources/BlurtEngine/Hotkey/TriggerKeyBindings.swift new file mode 100644 index 00000000..ab917d8a --- /dev/null +++ b/Sources/BlurtEngine/Hotkey/TriggerKeyBindings.swift @@ -0,0 +1,152 @@ +import Foundation + +/// What a press of one trigger key does (`TriggerKeyBindings.action(for:)`). +public enum TriggerKeyAction: Sendable, Hashable { + case off + /// Dictates, with the style the key's choice names. + case dictate + /// Reads the selected text aloud (`SelectionSpeechRouting`). + case readAloud + /// Records a spoken request about the selected text and reads the answer aloud. + case ask +} + +/// What one trigger key is set to in the Keys tab. Persisted as a string +/// (`raw`, `init(raw:)`) in `TriggerKeyStyleStore`'s map; everywhere else it's +/// this type, so a choice is matched by the compiler rather than by string. +public enum TriggerKeyChoice: Hashable, Sendable { + case off + /// Dictates with `style`: a profile's `uuidString`, + /// `StyleProfileStore.defaultStyleID` for base styling, or + /// `mainWindowStyle` for whatever the main window has selected. + case dictate(style: String) + case readAloud + case ask + + /// The style id for "use the main window's selection". Not a UUID, and not + /// `StyleProfileStore.defaultStyleID`, so it can collide with neither. + public static let mainWindowStyle = "current" + /// Dictating with the main window's selection: the bound key's default. + public static let dictate = TriggerKeyChoice.dictate(style: mainWindowStyle) + + public var action: TriggerKeyAction { + switch self { + case .off: .off + case .dictate: .dictate + case .readAloud: .readAloud + case .ask: .ask + } + } + + /// The stored form. Any string that isn't one of the action markers is a + /// style id — the encoding from before read and ask existed. + init(raw: String) { + switch raw { + case "": self = .off + case "read-aloud": self = .readAloud + case "ask": self = .ask + default: self = .dictate(style: raw) + } + } + + var raw: String { + switch self { + case .off: "" + case .dictate(let style): style + case .readAloud: "read-aloud" + case .ask: "ask" + } + } +} + +/// One read of everything the trigger keys are set to: the bound key +/// (`TriggerKeyStore`), each key's choice (`TriggerKeyStyleStore`) and each +/// key's tap/hold mode (`TriggerKeyActivationStore`, with +/// `TriggerActivationStore` as the mode of a key that has none). Every reader — +/// the tap, the read-aloud router, the press's style, and each view — asks +/// this, so the rules are written once: the bound key always dictates; with no +/// entry, the bound key dictates with the main window's style, left ⌥ reads +/// aloud, and every other key is off; a key with no mode of its own reads +/// aloud on a tap or hold, or else follows the shared mode. +public struct TriggerKeyBindings: Sendable, Equatable { + /// The key the ready screen and menu bar name, which always dictates. + public let bound: TriggerKey + private let choices: [TriggerKey: TriggerKeyChoice] + private let modes: [TriggerKey: TriggerActivation] + private let sharedMode: TriggerActivation + + public init(defaults: UserDefaults = .standard) { + self.init( + boundKeyCode: defaults.integer(forKey: TriggerKeyStore.defaultsKey), + choicesRaw: defaults.string(forKey: TriggerKeyStyleStore.defaultsKey) ?? "", + modesRaw: defaults.string(forKey: TriggerKeyActivationStore.defaultsKey) ?? "", + sharedModeRaw: defaults.string(forKey: TriggerActivationStore.defaultsKey) ?? "") + } + + /// From the raw slots a view has already observed through `@AppStorage`. + public init(boundKeyCode: Int, choicesRaw: String, modesRaw: String, sharedModeRaw: String) { + bound = TriggerKey.fromPersisted(boundKeyCode) + choices = TriggerKeyMap.decode(choicesRaw).mapValues(TriggerKeyChoice.init(raw:)) + modes = TriggerKeyMap.decode(modesRaw).compactMapValues(TriggerActivation.init(rawValue:)) + sharedMode = TriggerActivation.fromPersisted(sharedModeRaw) + } + + /// What `key` does with no choice of its own. Left ⌥ rather than left ⌘ + /// reads aloud: ⌘ is down for every shortcut and ⌘-click, so a lone press of + /// it is far more often the start of something else. + public static func defaultChoice(for key: TriggerKey, bound: TriggerKey) -> TriggerKeyChoice { + if key == bound { return .dictate } + return key == .leftOption ? .readAloud : .off + } + + /// `key`'s choice. The bound key always dictates, whatever its entry says. + public func choice(for key: TriggerKey) -> TriggerKeyChoice { + let choice = choices[key] ?? Self.defaultChoice(for: key, bound: bound) + return key == bound && choice.action != .dictate ? .dictate : choice + } + + /// `key`'s choice as its pop-up shows it: a style since deleted shows as the + /// main window's, which is what a press of that key then dictates with (see + /// `StyleProfileStore.profile(for:)`). + public func choice(for key: TriggerKey, profiles: [StyleProfile]) -> TriggerKeyChoice { + guard case .dictate(let style) = choice(for: key), !StyleProfileStore.isStyle(style, in: profiles) else { + return choice(for: key) + } + return .dictate + } + + /// What a press of `key` does. + public func action(for key: TriggerKey) -> TriggerKeyAction { choice(for: key).action } + + /// `key`'s tap/hold mode. + public func activation(for key: TriggerKey) -> TriggerActivation { + if let mode = modes[key] { return mode } + return key != bound && action(for: key) == .readAloud ? .tapOrHold : sharedMode + } + + /// The keys the tap watches: every key that does something. + public var enabledKeys: Set { Set(TriggerKey.allCases.filter { action(for: $0) != .off }) } + + /// The keys that dictate, in `TriggerKey` order — the bound key among them. + public var dictatingKeys: [TriggerKey] { TriggerKey.allCases.filter { action(for: $0) == .dictate } } + + /// Whether `key` is the only key that dictates, so it can't be set to + /// anything else (`TriggerKeyStyleStore.setChoice` refuses it). + public func isOnlyDictatingKey(_ key: TriggerKey) -> Bool { dictatingKeys == [key] } + + /// The keys besides the bound one that do something, in `TriggerKey` order: + /// the ready screen's extra rows. + public var extraKeys: [TriggerKey] { + TriggerKey.allCases.filter { $0 != bound && action(for: $0) != .off } + } + + /// Whether some key reads aloud or asks — read-aloud's settings do something + /// — and whether some key asks, which sends what's said to the LLM Gateway. + public var readsOrAsks: Bool { TriggerKey.allCases.contains { [.readAloud, .ask].contains(action(for: $0)) } } + public var asks: Bool { TriggerKey.allCases.contains { action(for: $0) == .ask } } + + /// Every key the tap watches, with its mode — the router's binding. + public var activations: [TriggerKey: TriggerActivation] { + Dictionary(uniqueKeysWithValues: enabledKeys.map { ($0, activation(for: $0)) }) + } +} diff --git a/Sources/BlurtEngine/Hotkey/TriggerKeyMap.swift b/Sources/BlurtEngine/Hotkey/TriggerKeyMap.swift new file mode 100644 index 00000000..159940f2 --- /dev/null +++ b/Sources/BlurtEngine/Hotkey/TriggerKeyMap.swift @@ -0,0 +1,28 @@ +import Foundation + +/// The on-disk form of a per-key setting — one JSON object, keycode → string — +/// shared by `TriggerKeyStyleStore` (actions and styles) and +/// `TriggerKeyActivationStore` (tap/hold modes), so the encoding and its one +/// safety rule are written once. +enum TriggerKeyMap { + /// The map behind a raw slot value. Blank or undecodable reads as empty, and + /// an entry for a keycode outside `TriggerKey` is dropped, so a hand-edited + /// value can't bind a new key. + static func decode(_ raw: String) -> [TriggerKey: String] { + guard let decoded = try? JSONDecoder().decode([String: String].self, from: Data(raw.utf8)) else { + return [:] + } + var map: [TriggerKey: String] = [:] + for (code, value) in decoded { + if let key = Int(code).flatMap(TriggerKey.init(rawValue:)) { map[key] = value } + } + return map + } + + /// Writes `map` to `slot` in `defaults`. A string map always encodes. + static func write(_ map: [TriggerKey: String], to slot: String, in defaults: UserDefaults) { + let raw = Dictionary(uniqueKeysWithValues: map.map { (String($0.key.rawValue), $0.value) }) + guard let data = try? JSONEncoder().encode(raw), let json = String(data: data, encoding: .utf8) else { return } + defaults.set(json, forKey: slot) + } +} diff --git a/Sources/BlurtEngine/Hotkey/TriggerKeyStore.swift b/Sources/BlurtEngine/Hotkey/TriggerKeyStore.swift index de4aee61..43322b06 100644 --- a/Sources/BlurtEngine/Hotkey/TriggerKeyStore.swift +++ b/Sources/BlurtEngine/Hotkey/TriggerKeyStore.swift @@ -20,17 +20,7 @@ public struct TriggerKeyStore { TriggerKey.fromPersisted(defaults.integer(forKey: Self.defaultsKey)) } nonmutating set { - let old = triggerKey defaults.set(newValue.rawValue, forKey: Self.defaultsKey) - // The key moved away from stops dictating unless the user had made it an - // extra key before binding it. Its Key Styles row offered no Off while it - // was bound, so a style picked for it then says nothing about keeping it - // as an extra — left behind, it would silently stay one (and, for fn, - // keep `GlobeKeyOverride` switching 🌐 off). The style is stashed, not - // dropped, and comes back if the key is bound again. - if old != newValue { - TriggerKeyStyleStore(defaults: defaults).rebound(from: old, to: newValue) - } } } diff --git a/Sources/BlurtEngine/Hotkey/TriggerKeyStyleStore.swift b/Sources/BlurtEngine/Hotkey/TriggerKeyStyleStore.swift index c7b8a63e..247cdb9c 100644 --- a/Sources/BlurtEngine/Hotkey/TriggerKeyStyleStore.swift +++ b/Sources/BlurtEngine/Hotkey/TriggerKeyStyleStore.swift @@ -1,31 +1,16 @@ import Foundation -/// Persists which style each trigger key dictates with — and so which keys -/// dictate at all: the bound key (`TriggerKeyStore`) always does, and any other -/// `TriggerKey` does once the user gives it a style in the Advanced pane's Key -/// Styles sheet. Rebinding swaps the two keys' entries with a stash -/// (`rebound(from:to:)`), so a former bound key never lingers as an extra -/// trigger, and nothing the user picked is lost by moving away and back. -/// Every key is still a lone modifier from the same curated set, watched by the -/// same listen-only tap; this only widens which of them the router matches. +/// Persists what each trigger key is set to (`TriggerKeyChoice`) — dictate +/// with a style, read the selection aloud, ask about it, or nothing — as set in +/// the Keys tab. Read through `TriggerKeyBindings`, which owns the defaults and +/// the rule that the bound key always dictates; this is the one write. /// -/// Stored as one JSON object, keycode → style id, under its own key so an -/// install that never touches it reads exactly as before: no entries, one key, -/// the main window's style. A style id is a profile's `uuidString`, -/// `StyleProfileStore.defaultStyleID` for base styling, or `currentStyleID` for -/// "whatever the main window has selected". A press never writes the main -/// window's selection — see `PressStyle`. +/// Stored as one JSON object, keycode → choice (`TriggerKeyMap`), under its own +/// key so an install that never touches it reads as the defaults. The name and +/// raw key predate read and ask, when every entry was a style id. public struct TriggerKeyStyleStore { /// Public so SwiftUI views can observe it directly (e.g. `@AppStorage`). public static var defaultsKey: String { DefaultsKey.triggerKeyStyles.key } - /// The stash `rebound(from:to:)` keeps; same encoding as `defaultsKey`. - public static var stashDefaultsKey: String { DefaultsKey.triggerKeyStyleStash.key } - - /// The style id for "use the main window's selection". Not a UUID, and not - /// `StyleProfileStore.defaultStyleID`, so it can collide with neither. - public static let currentStyleID = "current" - /// A Key Styles pop-up's Off: the key doesn't dictate. - public static let offChoice = "" private let defaults: UserDefaults @@ -33,100 +18,23 @@ public struct TriggerKeyStyleStore { self.defaults = defaults } - /// Each key's chosen style id; a key with no entry is off, unless it is the - /// bound key, which dictates with the current style. - public var styles: [TriggerKey: String] { - read(Self.defaultsKey) - } - - /// Gives `key` the style `styleID`, or turns it off for `offChoice` — the Key Styles - /// sheet's write, through here because the value is encoded. A choice equal - /// to the key's default (`defaultChoice`) is stored as no entry, so the bound - /// key set back to the main window's style leaves nothing behind. Returns - /// whether that changed which keys dictate, the one case the tap has to - /// rebind for: a style-only change is read at the next press, and a rebind - /// would discard a recording in progress. - public func setStyle(_ styleID: String, for key: TriggerKey) -> Bool { - let bound = TriggerKeyStore(defaults: defaults).triggerKey - var styles = styles - let wasEnabled = Self.enabledKeys(bound: bound, styles: styles) - let isDefault = styleID == Self.offChoice || styleID == Self.defaultChoice(for: key, bound: bound) - styles[key] = isDefault ? nil : styleID - guard write(styles, to: Self.defaultsKey) else { return false } - return Self.enabledKeys(bound: bound, styles: styles) != wasEnabled - } - - /// Moves the entries when the bound key changes from `old` to `new` - /// (`TriggerKeyStore.triggerKey` calls it). A key's entry means something - /// different in each role — its style while bound, whether and how it dictates - /// as an extra otherwise — so each key keeps the role it's leaving in the - /// stash and takes back the one it's entering: - /// - `old` returns to whatever it was as an extra before it was bound (off, - /// unless the user had set it up as one), and its bound style is stashed. - /// - `new` dictates with the style it had when last bound, else with its - /// style as an extra, and that extra entry is stashed for when it's unbound. - func rebound(from old: TriggerKey, to new: TriggerKey) { - var styles = styles - var stash = read(Self.stashDefaultsKey) - (styles[old], stash[old]) = (stash[old], styles[old]) - let extra = styles[new] - styles[new] = stash[new] ?? extra - stash[new] = extra - write(styles, to: Self.defaultsKey) - write(stash, to: Self.stashDefaultsKey) - } - - private func read(_ key: String) -> [TriggerKey: String] { - Self.styles(decoding: defaults.string(forKey: key) ?? "") - } - + /// Sets `key`'s choice, keeping the one rule a write can break: the bound key + /// always dictates. Taking it off dictation first moves the binding to the + /// next dictating key; with none, nothing is written and this returns false. + /// A choice equal to the key's default is stored as no entry, so a key set + /// back to its default leaves nothing behind. @discardableResult - private func write(_ styles: [TriggerKey: String], to key: String) -> Bool { - let raw = Dictionary(uniqueKeysWithValues: styles.map { (String($0.key.rawValue), $0.value) }) - guard let data = try? JSONEncoder().encode(raw), let json = String(data: data, encoding: .utf8) - else { return false } - defaults.set(json, forKey: key) - return true - } - - /// The mapping behind a raw slot value a view has already observed. Blank or - /// undecodable reads as no mappings, and an entry for a keycode outside - /// `TriggerKey` is dropped, so a hand-edited value can't bind a new key. - public static func styles(decoding raw: String) -> [TriggerKey: String] { - guard let decoded = try? JSONDecoder().decode([String: String].self, from: Data(raw.utf8)) else { - return [:] - } - var styles: [TriggerKey: String] = [:] - for (code, id) in decoded { - if let key = Int(code).flatMap(TriggerKey.init(rawValue:)) { styles[key] = id } + public func setChoice(_ choice: TriggerKeyChoice, for key: TriggerKey) -> Bool { + let bindings = TriggerKeyBindings(defaults: defaults) + var bound = bindings.bound + if key == bound, choice.action != .dictate { + guard let next = bindings.dictatingKeys.first(where: { $0 != key }) else { return false } + TriggerKeyStore(defaults: defaults).triggerKey = next + bound = next } - return styles - } - - /// The keys that dictate: the bound key plus every key with a style. - public var enabledKeys: Set { - Self.enabledKeys(bound: TriggerKeyStore(defaults: defaults).triggerKey, styles: styles) - } - - /// `enabledKeys` over values a view has already observed. - public static func enabledKeys(bound: TriggerKey, styles: [TriggerKey: String]) -> Set { - Set(styles.keys).union([bound]) - } - - /// What `key`'s Key Styles pop-up shows: `offChoice`, `currentStyleID`, or a style - /// id that still exists. A style since deleted shows as the current style, - /// which is what a press of that key then dictates with (see - /// `StyleProfileStore.profile(for:)`). - public static func choice( - for key: TriggerKey, bound: TriggerKey, styles: [TriggerKey: String], profiles: [StyleProfile] - ) -> String { - guard let id = styles[key] else { return defaultChoice(for: key, bound: bound) } - return StyleProfileStore.isStyle(id, in: profiles) ? id : currentStyleID - } - - /// What `key`'s row shows with no entry: the bound key dictates with the main - /// window's style, every other key is off. - public static func defaultChoice(for key: TriggerKey, bound: TriggerKey) -> String { - key == bound ? currentStyleID : offChoice + var entries = TriggerKeyMap.decode(defaults.string(forKey: Self.defaultsKey) ?? "") + entries[key] = choice == TriggerKeyBindings.defaultChoice(for: key, bound: bound) ? nil : choice.raw + TriggerKeyMap.write(entries, to: Self.defaultsKey, in: defaults) + return true } } diff --git a/Sources/BlurtEngine/Pipeline/CueReprimeSchedule.swift b/Sources/BlurtEngine/Pipeline/CueReprimeSchedule.swift new file mode 100644 index 00000000..71e05796 --- /dev/null +++ b/Sources/BlurtEngine/Pipeline/CueReprimeSchedule.swift @@ -0,0 +1,44 @@ +/// When each pair of cue chimes reloads after the audio output route changed +/// under their pre-rolled players — the decision `CueSoundPlayer` carries out, +/// here where `swift test` covers it, as `RecordingCueGate` is for when a cue +/// plays. +/// +/// Dictation's pair waits for the next press's `.connecting` (the route flip a +/// press causes lands then, one phase before the start chime it must protect) +/// or a terminal phase. Read-aloud's pair reloads straight away when nothing is +/// capturing — reads produce no pipeline phases, so waiting for one would leave +/// someone who only reads aloud with cues primed against the old route — and +/// otherwise at the end of the capture, so it never competes with a bring-up. +public struct CueReprimeSchedule: Sendable, Equatable { + /// Which pairs to reload now. + public struct Reloads: Sendable, Equatable { + public let dictation: Bool + public let readAloud: Bool + + public static let none = Reloads(dictation: false, readAloud: false) + } + + private var dictationPending = false + private var readAloudPending = false + private var capturing = false + + public init() {} + + /// The output route changed. + public mutating func routeChanged() -> Reloads { + dictationPending = true + guard capturing else { return Reloads(dictation: false, readAloud: true) } + readAloudPending = true + return .none + } + + /// The session moved to `phase`. + public mutating func phase(_ phase: PipelinePhase) -> Reloads { + capturing = phase.isCapturing + let dictation = dictationPending && (phase == .connecting || phase.isTerminal) + let readAloud = readAloudPending && phase.isTerminal + if dictation { dictationPending = false } + if readAloud { readAloudPending = false } + return Reloads(dictation: dictation, readAloud: readAloud) + } +} diff --git a/Sources/BlurtEngine/Pipeline/OverlayUIState.swift b/Sources/BlurtEngine/Pipeline/OverlayUIState.swift index d508e1f2..3b0e12ce 100644 --- a/Sources/BlurtEngine/Pipeline/OverlayUIState.swift +++ b/Sources/BlurtEngine/Pipeline/OverlayUIState.swift @@ -29,6 +29,10 @@ public enum OverlayUIState: Equatable, Sendable { case connecting case recording case processing + /// Reading the selection (or an answer about it) aloud. Not a pipeline phase: + /// the app shows it for as long as a read plays, with the recording pill's + /// orb and meter, the meter following the voice rather than the mic. + case speaking /// A dictation attempt failed. The shell shows this as a brief red flash on /// the pill before settling back to `.idle`; `message` is the human-readable /// reason (e.g. "AssemblyAI error 401…"), surfaced via the pill's hover @@ -60,6 +64,7 @@ public enum OverlayUIState: Equatable, Sendable { case .idle: "Blurt." case .connecting: "Connecting to the microphone." case .recording: "Recording." + case .speaking: "Reading aloud." case .processing: "Processing." case .error(let message, _): message case .pasted: "Your dictation was pasted." @@ -80,7 +85,7 @@ public enum OverlayUIState: Equatable, Sendable { // A sentence, not a word: long enough to read the device name. case .error(_, _?): 3.0 case .error, .noTarget: 1.6 - case .idle, .connecting, .recording, .processing: nil + case .idle, .connecting, .recording, .processing, .speaking: nil } } @@ -134,3 +139,22 @@ extension PipelinePhase { } } } + +extension PipelinePhase { + /// The pill for this phase while a read may be playing, or nil to leave the + /// pill as it is. A read wears the pill until it ends (`.speaking`); only a + /// new capture takes it over — a press stops the read first — so an earlier + /// dictation finishing mid-read can neither cover nor hide it. + public func overlayState(whileSpeaking speaking: Bool) -> OverlayUIState? { + speaking && !isCapturing ? nil : overlayState + } + + /// The pill once a read ends, this being the session's last phase. A steady + /// state still under way (a capture, a transcription) shows again; a notice + /// (Pasted, Copied, an error) belonged to a dictation that ended before the + /// read did, so the pill settles to idle rather than replaying it — and + /// VoiceOver re-announcing it. + public var overlayStateAfterSpeech: OverlayUIState { + overlayState.noticeDwellSeconds == nil ? overlayState : .idle + } +} diff --git a/Sources/BlurtEngine/README.md b/Sources/BlurtEngine/README.md index a9736b59..07dc1086 100644 --- a/Sources/BlurtEngine/README.md +++ b/Sources/BlurtEngine/README.md @@ -257,13 +257,14 @@ Two string helpers carry the "usable text" rule shared by focus capture, the con The engine ships the _decision logic_ for a lone-modifier trigger; the host supplies the event source (in Blurt, a `CGEventTap` — see `App/Blurt/Blurt/Hotkey/DictationKeyTap.swift` for the reference wiring). -- **`TriggerKey`** — the curated lone modifiers usable as a trigger (right ⌘, right ⌥, `fn`), with keycodes, display labels, and the device-modifier masks the event source needs. -- **`GlobeKeyOverride`** — while `fn` is among the bound keys — the dictation key, or one turned on in Key Styles — sets macOS's "Press 🌐 key to" (`com.apple.HIToolbox` `AppleFnUsageType`) to Do Nothing so the press doesn't also fire the system's emoji/input-source/dictation action; once no bound key is `fn` the replaced value is restored. The record sits in a suite both builds share (`sharedSuite`), and if either domain can't be opened `sync` does nothing. The write only lands at the next log-in, so the Shortcut footer warns that `fn` may also open the emoji picker or Apple Dictation until then. `TriggerKeyStore.migrateStaleFunctionBinding()` rewrites a keycode 63 saved before `fn` was removed (#146) to right ⌘, once, so restoring `fn` doesn't rebind anyone behind their back. The listen-only tap can't swallow the press, so this is the only way to keep it from double-firing. +- **`TriggerKey`** — the curated lone modifiers usable as a trigger (right ⌘, right ⌥, `fn`, left ⌘, left ⌥, left ⌃), with keycodes, display labels, and the device-modifier masks the event source needs. +- **`TriggerKeyBindings`** — one read of what every key does (`TriggerKeyChoice`: off, dictate with a style, read aloud, ask) and how it's pressed: the bound key always dictates, and the defaults are the bound key dictating, left ⌥ reading aloud, the rest off. `TriggerKeyStyleStore.setChoice` and `TriggerKeyActivationStore.setActivation` are the writes. +- **`GlobeKeyOverride`** — while `fn` does anything — dictates, reads aloud, or asks, as set in the Keys tab — sets macOS's "Press 🌐 key to" (`com.apple.HIToolbox` `AppleFnUsageType`) to Do Nothing so the press doesn't also fire the system's emoji/input-source/dictation action; once `fn` is off the replaced value is restored. The record sits in a suite both builds share (`sharedSuite`), and if either domain can't be opened `sync` does nothing. The write only lands at the next log-in, so the Keys tab's footer warns that `fn` may also open the emoji picker or Apple Dictation until then. `TriggerKeyStore.migrateStaleFunctionBinding()` rewrites a keycode 63 saved before `fn` was removed (#146) to right ⌘, once, so restoring `fn` doesn't rebind anyone behind their back. The listen-only tap can't swallow the press, so this is the only way to keep it from double-firing. - **`TriggerKeyStore`** — persists the chosen key in `UserDefaults` (`BlurtTriggerKeyCode`), defaulting to right ⌘. - **`DictationKeyGate`** — a pure, clock-free state machine that turns `modifierDown(at:)` / `modifierUp(at:)` / `otherKeyDown()` into `.start` / `.stop` / `.cancel` / `.none`. Recording starts the instant the modifier goes down; on key-up, a release held ≥ `holdThreshold` (default 1 s) is push-to-talk (stop), a shorter release latches tap-to-toggle (next tap stops). A modifier+key combo from idle cancels the fresh capture; over a latched recording it passes through as a normal shortcut. Callers pass monotonic timestamps, so every decision is deterministic and unit-tested (`DictationKeyGateTests`, `HotkeyRaceTests`). -- **`DictationKeyRouter`** — the recommended layer over the gate: reduce each raw event to `.flagsChanged(keyCode:triggerFlagIsOn:)` / `.keyDown(keyCode:)` and `handle(_:at:)` applies the filters every event source needs — only the bound keycodes' flag changes count (the first one down owns the gate until it's idle again, readable as `activeKeyCode`), and only genuine down/up _edges_ reach the gate (`flagsChanged` deliveries re-report the bit whether or not it changed, so a repeat must not double-start a dictation). `reset()` / `rebind(triggerKeyCodes:activation:)` clear state that can no longer be trusted (dropped events, a rebound trigger) and return whether they discarded a live recording. Unit-tested (`DictationKeyRouterTests`). +- **`DictationKeyRouter`** — the recommended layer over the gate: reduce each raw event to `.flagsChanged(keyCode:triggerFlagIsOn:)` / `.keyDown(keyCode:)` / `.pointerDown` (a mouse button: a modifier held through a click is a modifier-click, which cancels the press like a combo) and `handle(_:at:)` applies the filters every event source needs — only the bound keycodes' flag changes count (the first one down owns the gate until it's idle again, readable as `activeKeyCode`), and only genuine down/up _edges_ reach the gate (`flagsChanged` deliveries re-report the bit whether or not it changed, so a repeat must not double-start a dictation). `init(activations:)` binds each keycode to its own mode (a press from idle resolves under its key's), and `reset()` / `rebind(activations:)` clear state that can no longer be trusted (dropped events, a rebound trigger) and return whether they discarded a live recording. Unit-tested (`DictationKeyRouterTests`). -Map the router's actions onto the session with `submit`: `.start` → `submit(.press(key:))` (the key that pressed, for its style; nil for the main window's), `.stop` → `submit(.release)`, `.cancel` → `submit(.cancel)` — event-tap callbacks can't `await`, and `submit` preserves their emit order where per-callback `Task` spawning wouldn't. If your event source can lose key-ups (a disabled tap, a rebind), call the router's `reset()`/`rebind(triggerKeyCodes:activation:)` and recover a discarded recording with `submit(.cancelRecording)`. +Map the router's actions onto the session with `submit`: `.start` → `submit(.press(key:))` (the key that pressed, for its style; nil for the main window's), `.stop` → `submit(.release)`, `.cancel` → `submit(.cancel)` — event-tap callbacks can't `await`, and `submit` preserves their emit order where per-callback `Task` spawning wouldn't. If your event source can lose key-ups (a disabled tap, a rebind), call the router's `reset()`/`rebind(activations:)` and recover a discarded recording with `submit(.cancelRecording)`. ## Testing your integration diff --git a/Sources/BlurtEngine/TTS/AssemblyAISpeechSynthesizer.swift b/Sources/BlurtEngine/TTS/AssemblyAISpeechSynthesizer.swift index 871eca96..5753f828 100644 --- a/Sources/BlurtEngine/TTS/AssemblyAISpeechSynthesizer.swift +++ b/Sources/BlurtEngine/TTS/AssemblyAISpeechSynthesizer.swift @@ -1,66 +1,61 @@ import Foundation import Synchronization -/// Streams speech for a piece of text from AssemblyAI's streaming TTS, as raw -/// PCM16 little-endian mono at `sampleRate`. Powers the experimental -/// read-selection-aloud press (`SelectionSpeaker`, `SelectionSpeechStore`). +/// Speaks one piece of text through AssemblyAI's Streaming TTS HTTP API +/// (`POST /v1/speech`), as raw PCM16 little-endian mono at `sampleRate`. +/// Powers the read-aloud and ask keys (`SelectionSpeaker`, +/// `SelectionSpeechRouting`). /// -/// **This endpoint is undocumented.** AssemblyAI's public docs say it offers no -/// standalone TTS, so everything here comes from the AssemblyAI agent runtime's -/// adapter (`aai-runtime/src/providers/tts/assemblyai.ts`), which read it back -/// from the live service. That is why the feature sits behind an experimental -/// switch instead of following the dictation API's docs-are-the-contract rule. -/// (It is also the one file `check-invariants.sh` lets hold a WebSocket.) +/// The API's docs are the contract (https://www.assemblyai.com/docs/tts/api-reference/http-api/create-speech), +/// as they are for the dictation API: send only what they describe. /// -/// The protocol: -/// - **Connect** to `endpoint` with `voice` and `sample_rate` query params. Authenticate -/// with the **raw** API key in `Authorization` — not `Bearer `. A bad key -/// still completes the upgrade, then comes back in-band as an `Error` frame. -/// - **Send** `{"type":"Generate","text":…}` followed by `{"type":"Flush"}` for -/// each segment. `Generate` alone only buffers text: no audio is produced -/// until a `Flush`. See `SpeechSegmenter` for where the cuts go. -/// - **Receive** `Audio` frames (base64 PCM16), then one `FlushDone` for each -/// flush. The text has been spoken once every flush is acknowledged. -/// `WordBoundaries` frames are timing data, not acknowledgements, and a -/// `Begin` frame may or may not arrive, so don't wait for it. -/// - **Close** with `{"type":"Terminate"}`. +/// One request per piece, its audio the response body, streamed while it is +/// synthesized: the server splits a long text into sentences itself and the +/// first ones' audio arrives long before the last. `SelectionSpeaker` cuts a +/// long selection into pieces (`ListeningChunks`) and asks for the next one only +/// as the listening catches up, so a long article starts quickly and is +/// synthesized — and billed — only as far as it is heard. HTTP rather than the +/// WebSocket API for that reason: a request per piece needs no session to keep +/// alive while reading waits on the listener, and no session lifetime caps it. +/// +/// - **Request**: JSON `text`, `voice_id`, `sample_rate`, `encoding`; raw +/// audio and streaming are the defaults. Authenticate with the raw API key in +/// `Authorization`, as the dashboard shows it. +/// - **Response**: the status and headers wait for the first audio, so every +/// failure before it is an ordinary HTTP error with a JSON body (`code`, +/// `message`). Each write holds whole samples, but the network can split one, +/// so an odd byte is carried into the next chunk. struct AssemblyAISpeechSynthesizer: Sendable { static let sampleRate = 24_000 - static let voice = "jane" - static let endpoint = URL(staticString: "wss://streaming-tts.assemblyai.com/v1/ws/") + static let encoding = "pcm_s16le" + static let endpoint = URL(staticString: "https://tts.us.assemblyai.com/v1/speech") + /// How long the response may go quiet before the request is abandoned. A + /// stalled server would otherwise leave the press in read-aloud mode, + /// silent, until the next tap ended it. Real audio lands within a second or + /// two. + static let timeout: TimeInterval = 15 private let apiKeyProvider: @Sendable () -> String? - private let connect: @Sendable (URLRequest) -> any SpeechSocket - /// How long the socket may go silent before the read is abandoned. A stalled - /// server, or a flush that is never acknowledged, would otherwise leave the - /// press in read-aloud mode, silent, until the next tap ended it. Real frames - /// land within about a second of each flush. - private let receiveTimeout: Duration + private let transport: any SpeechTransport - /// `connect` is the socket seam: production opens a `URLSessionWebSocketTask`, - /// and the tests script the service's side of the exchange. + /// `transport` is the network seam: production streams through + /// `URLSession`, and the tests play the service's side. init( apiKeyProvider: @escaping @Sendable () -> String? = { APIKeyStore.current }, - receiveTimeout: Duration = .seconds(10), - connect: @escaping @Sendable (URLRequest) -> any SpeechSocket = { request in - let socket = URLSession.shared.webSocketTask(with: request) - socket.resume() - return socket - } + transport: any SpeechTransport = URLSession.shared ) { self.apiKeyProvider = apiKeyProvider - self.receiveTimeout = receiveTimeout - self.connect = connect + self.transport = transport } - /// PCM chunks in playback order. The stream finishes once the service has - /// acknowledged every segment. Cancelling the consumer closes the socket. - func synthesize(_ text: String) -> AsyncThrowingStream { - let segments = SpeechSegmenter.segments(of: text) - return AsyncThrowingStream { continuation in + /// PCM chunks of `text` spoken by `voiceID`, in playback order, as they + /// arrive. Blank text finishes at once without a request. Cancelling the + /// consumer cancels the request. + func speech(for text: String, voiceID: String = ReadAloudVoice.defaultID) -> AsyncThrowingStream { + AsyncThrowingStream { continuation in let task = Task { do { - try await run(segments: segments) { continuation.yield($0) } + try await stream(text, voiceID: voiceID) { continuation.yield($0) } continuation.finish() } catch { continuation.finish(throwing: error) @@ -70,141 +65,85 @@ struct AssemblyAISpeechSynthesizer: Sendable { } } - static let url = endpoint.appending(queryItems: [ - URLQueryItem(name: "voice", value: voice), - URLQueryItem(name: "sample_rate", value: String(sampleRate)), - ]) - - private func run(segments: [String], onAudio: @escaping @Sendable (Data) -> Void) async throws { - guard !segments.isEmpty else { return } + private func stream(_ text: String, voiceID: String, onAudio: (Data) -> Void) async throws { + guard let text = text.trimmedNonEmpty() else { return } guard let apiKey = apiKeyProvider(), !apiKey.isEmpty else { throw BlurtError.apiKeyMissing } - var request = URLRequest(url: Self.url) - // Raw key, not `Bearer` — see the type doc. - request.setValue(apiKey, forHTTPHeaderField: "Authorization") - request.setUserAgent() - let socket = connect(request) - try await withTaskCancellationHandler { - // `exchange` closes the socket itself on every failure. - try await exchange(over: socket, segments: segments, onAudio: onAudio) - try? await socket.send(Self.terminate) - socket.close(normally: true) - } onCancel: { - socket.close(normally: false) - } - } - - private func exchange( - over socket: any SpeechSocket, segments: [String], onAudio: @escaping @Sendable (Data) -> Void - ) async throws { - let watchdog = Watchdog(timeout: receiveTimeout) - try await withThrowingTaskGroup(of: Void.self) { group in - // Sending runs beside the receive loop rather than ahead of it. A - // rejected key is reported in a frame, and the receive loop is what - // turns that frame into a readable error. A send failing first would - // only say the socket closed. - group.addTask { - for segment in segments { - // Each segment is billed synthesis. Once the read has failed or been - // stopped, don't queue any more. - try Task.checkCancellation() - try await socket.send(Self.command("Generate", text: segment)) - try await socket.send(Self.flush) - } - } - // One idle timer for the whole exchange. A closed socket is the only - // thing that ends a pending receive, so the timer closes it. - group.addTask { - try await watchdog.run { socket.close(normally: false) } + do { + let (response, body) = try await transport.stream( + try Self.request(for: text, voiceID: voiceID, apiKey: apiKey)) + guard response.statusCode == 200 else { + let reason = Self.failure(status: response.statusCode, body: try await Self.head(of: body)) + throw SpeechSynthesisError.server(reason) } - do { - var acknowledged = 0 - while acknowledged < segments.count { - let message = try await socket.receive() - watchdog.heard() - switch Self.frame(from: message) { - case .audio(let pcm): onAudio(pcm) - case .flushDone: acknowledged += 1 - case .error(let message): throw SpeechSynthesisError.server(message) - case .ignored: break - } + // At most the one byte of a sample the network split; whole chunks, + // the usual case, pass straight through without a copy. + var carry: UInt8? + for try await chunk in body { + var audio = chunk + if let byte = carry { + audio.insert(byte, at: audio.startIndex) + carry = nil } - } catch { - // Close *before* leaving the group. The group waits for the sender on - // the way out, and a socket send doesn't observe cancellation, so only - // a closed socket makes a pending send fail rather than finish. - socket.close(normally: false) - throw watchdog.expired ? SpeechSynthesisError.timedOut : error + if audio.count % 2 == 1 { carry = audio.removeLast() } + if !audio.isEmpty { onAudio(audio) } } - group.cancelAll() + } catch let error as URLError where error.code == .timedOut { + throw SpeechSynthesisError.timedOut + } catch let error as URLError where error.code == .cancelled { + // A stopped read cancels its request, which URLSession reports as an + // error. It's a cancellation, and must read as one: the router beeps on + // a failure, never on a stop. + throw CancellationError() } } // MARK: - Wire format - enum Frame: Equatable { - case audio(Data) - case flushDone - case error(String) - /// `Begin`, `Warning`, `WordBoundaries`, `Cancelled`, and anything newer. - case ignored - } - - private struct Command: Encodable { - let type: String - let text: String? + private struct Body: Encodable { + let text: String + let voiceID: String + let sampleRate: Int + let encoding: String + + enum CodingKeys: String, CodingKey { + case text + case voiceID = "voice_id" + case sampleRate = "sample_rate" + case encoding + } } - /// The fixed commands, encoded once rather than per segment. - private static let flush = command("Flush") - private static let terminate = command("Terminate") - - static func command(_ type: String, text: String? = nil) -> String { - let encoder = JSONEncoder() - encoder.outputFormatting = .sortedKeys - // Encoding two strings to UTF-8 JSON can't fail. - guard let data = try? encoder.encode(Command(type: type, text: text)) else { return "" } - return String(bytes: data, encoding: .utf8) ?? "" + static func request(for text: String, voiceID: String, apiKey: String) throws -> URLRequest { + try URLRequest.assemblyAIJSONPost( + to: endpoint, apiKey: apiKey, timeout: timeout, + body: Body(text: text, voiceID: voiceID, sampleRate: sampleRate, encoding: encoding)) } - static func frame(from message: URLSessionWebSocketTask.Message) -> Frame { - let data: Data - switch message { - case .string(let text): data = Data(text.utf8) - case .data(let bytes): data = bytes - @unknown default: return .ignored + /// The start of an error body: enough for any error the docs describe; a + /// proxy's page or a stalled stream past it isn't worth waiting on. + private static func head(of body: AsyncThrowingStream) async throws -> Data { + var data = Data() + for try await chunk in body { + data.append(chunk) + if data.count >= 65_536 { break } } - return frame(from: data) + return data } - /// Decodes one server frame leniently: anything that isn't a JSON object with - /// a known `type` is `.ignored`, not an error. `error_code` may be a number or - /// a string, depending on the server version. - static func frame(from data: Data) -> Frame { - guard - let object = (try? JSONSerialization.jsonObject(with: data)) as? [String: Any], - let type = object["type"] as? String - else { return .ignored } - switch type { - case "Audio": - guard let base64 = object["audio"] as? String, let pcm = Data(base64Encoded: base64), !pcm.isEmpty - else { return .ignored } - return .audio(pcm) - case "FlushDone": - return .flushDone - case "Error": - let reason = (object["error"] as? String)?.trimmedNonEmpty() ?? "unknown error" - if let code = object["error_code"] { return .error("(\(code)) \(reason)") } - return .error(reason) - default: - return .ignored - } + /// A failed request's reason: the error body's `code` and `message`, or the + /// bare status when the body isn't one (a proxy's page). + static func failure(status: Int, body: Data) -> String { + let object = (try? JSONSerialization.jsonObject(with: body)) as? [String: Any] + guard let message = (object?["message"] as? String)?.trimmedNonEmpty() else { return "HTTP \(status)" } + if let code = object?["code"] as? String { return "(\(code)) \(message)" } + return message } } enum SpeechSynthesisError: Error, Equatable, LocalizedError { - /// The socket went silent for longer than the synthesizer's receive timeout. + /// The response went quiet for longer than `AssemblyAISpeechSynthesizer.timeout`. case timedOut - /// An in-band `Error` frame, e.g. `(1008) Unauthorized: Invalid API key`. + /// A failed request, e.g. `(unauthorized) Invalid API key`. case server(String) var errorDescription: String? { @@ -215,51 +154,70 @@ enum SpeechSynthesisError: Error, Equatable, LocalizedError { } } -/// Idle timer for one exchange: `run` closes the socket once `timeout` passes -/// with no `heard()`, and records that it did so, so the receive loop can report -/// a timeout rather than the closed socket it causes. -private final class Watchdog: Sendable { - private struct State { - var lastHeard = ContinuousClock.now - var expired = false - } - - private let timeout: Duration - private let state = Mutex(State()) +/// The one network call the synthesizer makes: a request whose response body +/// is read as it arrives. `URLSession` is the production conformance; the tests +/// play the service. +protocol SpeechTransport: Sendable { + /// The response head, once it arrives, and its body as it streams in. + func stream(_ request: URLRequest) async throws -> (HTTPURLResponse, AsyncThrowingStream) +} - init(timeout: Duration) { self.timeout = timeout } +extension URLSession: SpeechTransport { + /// Hands the body on in the blocks URLSession delivers (`SpeechStreamDelegate`) + /// rather than a byte at a time: a long read is millions of bytes. + func stream(_ request: URLRequest) async throws -> (HTTPURLResponse, AsyncThrowingStream) { + let (body, feed) = AsyncThrowingStream.makeStream() + let task = dataTask(with: request) + feed.onTermination = { _ in task.cancel() } + let response = try await withTaskCancellationHandler { + try await withCheckedThrowingContinuation { (continuation: CheckedContinuation) in + task.delegate = SpeechStreamDelegate(response: continuation, body: feed) + task.resume() + } + } onCancel: { + task.cancel() + } + return (response, body) + } +} - var expired: Bool { state.withLock(\.expired) } +/// One streamed request's delegate: resumes `response` with the head once it +/// arrives (or the failure, if it never does), then feeds each block of the +/// body to `body` and finishes it with the request. +private final class SpeechStreamDelegate: NSObject, URLSessionDataDelegate, Sendable { + private let pending: Mutex?> + private let body: AsyncThrowingStream.Continuation - func heard() { state.withLock { $0.lastHeard = .now } } + init(response: CheckedContinuation, body: AsyncThrowingStream.Continuation) { + self.pending = Mutex(response) + self.body = body + } - func run(onExpiry close: () -> Void) async throws { - while true { - let deadline = state.withLock { $0.lastHeard } + timeout - guard ContinuousClock.now < deadline else { break } - try await Task.sleep(until: deadline) + func urlSession( + _ session: URLSession, dataTask: URLSessionDataTask, didReceive response: URLResponse + ) async -> URLSession.ResponseDisposition { + let continuation = pending.withLock { pending in + defer { pending = nil } + return pending } - state.withLock { $0.expired = true } - close() + if let http = response as? HTTPURLResponse { + continuation?.resume(returning: http) + return .allow + } + continuation?.resume(throwing: URLError(.badServerResponse)) + return .cancel } -} -/// The socket operations the synthesizer uses, so the tests can play the -/// service's side. `URLSessionWebSocketTask` is the production conformance. -protocol SpeechSocket: Sendable { - func send(_ text: String) async throws - func receive() async throws -> URLSessionWebSocketTask.Message - /// Closes the socket: normally after `Terminate`, abruptly on failure or - /// cancel. Synchronous, because a cancellation handler can't await. - func close(normally: Bool) -} - -extension URLSessionWebSocketTask: SpeechSocket { - func send(_ text: String) async throws { - try await send(.string(text)) + func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) { + body.yield(data) } - func close(normally: Bool) { - cancel(with: normally ? .normalClosure : .goingAway, reason: nil) + func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: (any Error)?) { + let continuation = pending.withLock { pending in + defer { pending = nil } + return pending + } + continuation?.resume(throwing: error ?? URLError(.badServerResponse)) + if let error { body.finish(throwing: error) } else { body.finish() } } } diff --git a/Sources/BlurtEngine/TTS/ListeningChunks.swift b/Sources/BlurtEngine/TTS/ListeningChunks.swift new file mode 100644 index 00000000..45b418c2 --- /dev/null +++ b/Sources/BlurtEngine/TTS/ListeningChunks.swift @@ -0,0 +1,43 @@ +import Foundation + +/// Cuts a selection into the pieces `SelectionSpeaker` reads one request at a +/// time (`AssemblyAISpeechSynthesizer`), rewriting each for listening first +/// when skip-jargon is on. Reading starts after the first piece — kept short +/// (`firstMaxCharacters`) so its rewrite and first audio come back quickly — +/// and the rest are asked for only as the listening reaches them. +/// +/// Whole paragraphs are packed into a chunk while they fit; a paragraph too +/// long for one is cut at sentence ends (`SpeechSegmenter`), so a chunk never +/// starts or ends mid-sentence and the rewrite sees whole thoughts. +enum ListeningChunks { + /// Big enough that a chunk is a few paragraphs (a minute or two of speech), + /// small enough that its rewrite comes back in a second or two. + static let maxCharacters = 1_500 + /// The first piece's limit: about a paragraph, the wait before the first word. + static let firstMaxCharacters = 400 + + static func chunks( + of text: String, maxCharacters: Int = maxCharacters, firstMaxCharacters: Int = firstMaxCharacters + ) -> [String] { + var chunks: [String] = [] + var current = "" + var limit: Int { chunks.isEmpty ? min(firstMaxCharacters, maxCharacters) : maxCharacters } + for paragraph in text.split(separator: "\n\n", omittingEmptySubsequences: true) { + let trimmed = paragraph.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { continue } + let pieces = + trimmed.count <= limit ? [trimmed] : SpeechSegmenter.segments(of: trimmed, maxCharacters: limit) + for (index, piece) in pieces.enumerated() { + let separator = current.isEmpty ? "" : (index == 0 ? "\n\n" : " ") + if current.count + separator.count + piece.count > limit, !current.isEmpty { + chunks.append(current) + current = piece + } else { + current += separator + piece + } + } + } + if !current.isEmpty { chunks.append(current) } + return chunks + } +} diff --git a/Sources/BlurtEngine/TTS/PCMLevel.swift b/Sources/BlurtEngine/TTS/PCMLevel.swift new file mode 100644 index 00000000..ec3a3674 --- /dev/null +++ b/Sources/BlurtEngine/TTS/PCMLevel.swift @@ -0,0 +1,26 @@ +import Foundation + +/// The loudness of PCM16 LE mono audio on the mic meter's 0...1 scale +/// (`MicCapture.linearLevel`), so the pill's meter reads a voice playing back +/// the way it reads one being recorded. +enum PCMLevel { + /// One level per `windowFrames`-frame window of `pcm`, the last window + /// possibly short. An odd trailing byte is ignored. + static func levels(of pcm: Data, windowFrames: Int) -> [Float] { + let frames = pcm.count / 2 + guard frames > 0, windowFrames > 0 else { return [] } + return pcm.withUnsafeBytes { raw in + let samples = raw.bindMemory(to: Int16.self) + return stride(from: 0, to: frames, by: windowFrames).map { start in + let end = min(start + windowFrames, frames) + var sum: Double = 0 + for index in start.. String { try await complete( Self.chat( - system: Self.answerInstruction, user: Self.answerPrompt(request, about: selection), + system: Self.answerInstruction, + user: Self.answerPrompt(request, about: String(selection.prefix(Self.answerSelectionCharacters))), maxTokens: Self.answerMaxTokens), timeout: Self.answerTimeout ).text @@ -162,16 +168,7 @@ struct ReadAloudLLM: Sendable { } static func request(for body: Body, apiKey: String, timeout: TimeInterval) throws -> URLRequest { - var request = URLRequest(url: endpoint, timeoutInterval: timeout) - request.httpMethod = "POST" - // The raw key, as every AssemblyAI request Blurt makes. - request.setValue(apiKey, forHTTPHeaderField: "Authorization") - request.setValue("application/json", forHTTPHeaderField: "Content-Type") - request.setUserAgent() - let encoder = JSONEncoder() - encoder.outputFormatting = .sortedKeys - request.httpBody = try encoder.encode(body) - return request + try URLRequest.assemblyAIJSONPost(to: endpoint, apiKey: apiKey, timeout: timeout, body: body) } private struct Completion: Decodable { diff --git a/Sources/BlurtEngine/TTS/ReadAloudVoice.swift b/Sources/BlurtEngine/TTS/ReadAloudVoice.swift new file mode 100644 index 00000000..15efb740 --- /dev/null +++ b/Sources/BlurtEngine/TTS/ReadAloudVoice.swift @@ -0,0 +1,61 @@ +/// A voice read-aloud can speak with: its `voice_id` in a TTS request and the +/// name the Settings picker shows. +/// +/// `catalog` is the English voices from the TTS docs' Voices page +/// (https://www.assemblyai.com/docs/tts/voices). English only, because a read +/// sends no `language`: without one a catalog voice speaks the language it was +/// recorded in, so a Spanish voice would read an English selection as Spanish. +/// The built-in voices are always loaded on the server; any other voice loads +/// when a session first opens with it, so its first read after a quiet spell +/// starts a little later. +public struct ReadAloudVoice: Sendable, Equatable, Identifiable { + public enum Group: String, Sendable, CaseIterable { + case builtIn = "Built-in" + case library = "Voice Library" + } + + /// The `voice_id` query parameter: an AssemblyAI library slug. + public let id: String + public let name: String + /// The accent the docs list for a library voice ("British"), or nil. + public let accent: String? + public let group: Group + + /// The picker's label: "Buford (British)", or just the name. + public var label: String { + accent.map { "\(name) (\($0))" } ?? name + } + + /// The service's own default, `aai-jane`, so an untouched install sounds as + /// a session with no `voice_id` would. + public static let defaultID = "aai-jane" + + public static let catalog: [ReadAloudVoice] = + [ + "Jane", "Alba", "Anna", "Charles", "Eve", "George", "Jean", "Mary", "Michael", "Paul", "Vera", + ].map { builtIn($0) } + + [ + ("Aaliyah", "American"), ("Abdiel", nil), ("Annabelle", "American"), ("April", "American"), + ("Audrey", "American"), ("Buford", "British"), ("Carrie", "British"), ("Cloyd", "British"), + ("Dorsey", nil), ("Ernest", "American"), ("Ewald", "American"), ("Gunnar", nil), + ("Hailey", "American"), ("Kyla", "American"), ("Lilla", "British"), ("Nancy", "British"), + ("Percival", "British"), ("Peter", "American"), ("Sara", "American"), ("Steven", "Australian"), + ("Titus", "American"), ("Troy", "Scottish"), ("Yasmin", "British"), + ].map { library($0.0, accent: $0.1) } + + /// The catalog entry for `id`, or nil for an id that isn't one. + public static func voice(id: String) -> ReadAloudVoice? { + catalog.first { $0.id == id } + } + + private static func builtIn(_ name: String) -> ReadAloudVoice { + ReadAloudVoice(id: slug(name), name: name, accent: nil, group: .builtIn) + } + + private static func library(_ name: String, accent: String?) -> ReadAloudVoice { + ReadAloudVoice(id: slug(name), name: name, accent: accent, group: .library) + } + + /// The docs' slugs are `aai-` and the lowercased name, for every voice listed. + private static func slug(_ name: String) -> String { "aai-" + name.lowercased() } +} diff --git a/Sources/BlurtEngine/TTS/SelectionSpeaker.swift b/Sources/BlurtEngine/TTS/SelectionSpeaker.swift index 9338af82..d9d161c7 100644 --- a/Sources/BlurtEngine/TTS/SelectionSpeaker.swift +++ b/Sources/BlurtEngine/TTS/SelectionSpeaker.swift @@ -1,16 +1,31 @@ #if os(macOS) + /// What reads text aloud for `SelectionSpeechRouter`: `SelectionSpeaker` in + /// production, a stand-in under UI testing (no network, no audio). Both calls + /// run until the speech has played; cancelling the task stops it. + public protocol ReadAloudSpeaker: Sendable { + func speak(_ text: String, style: ReadAloudStyle, onLevel: (@Sendable (Float) -> Void)?) async throws + func answer( + _ request: String, about selection: String, style: ReadAloudStyle, onLevel: (@Sendable (Float) -> Void)? + ) async throws + } + /// Reads text aloud with AssemblyAI's streaming TTS. This is the engine half of - /// the experimental read-selection-aloud press (`SelectionSpeechStore`); the app - /// decides when to call it. + /// the read-aloud and ask keys (`SelectionSpeechRouting`); the app decides when + /// to call it. /// /// `speak` runs until the last sample has played. To stop early, cancel the task - /// that is awaiting it: cancellation silences playback at once and closes the - /// socket. - public struct SelectionSpeaker: Sendable { - /// The most text one press will read. Enough for several paragraphs, while - /// keeping a whole-document select-all from queueing minutes of audio (and - /// the TTS usage that comes with it) behind a single tap. - public static let maxCharacters = 4_000 + /// that is awaiting it: cancellation silences playback at once and cancels the + /// request in flight. + struct SelectionSpeaker: ReadAloudSpeaker { + /// The most text one press will read: a long article (about 15,000 words). + /// Reading is paced by listening (`bufferTarget`), so a long selection costs + /// only as much TTS as is heard before it's stopped. + static let maxCharacters = 100_000 + + /// How far ahead of the listener the reading runs, in seconds: the next + /// piece is asked for once less than this is queued. Comfortably longer + /// than a piece's rewrite and first audio, so pieces join without a gap. + static let bufferTarget: Double = 30 private static let logger = HostIdentity.current.logger("SelectionSpeech") @@ -18,7 +33,7 @@ private let llm: ReadAloudLLM private let makeSink: @Sendable (_ rate: Double) -> any PCMSink - public init() { + init() { self.init(synthesizer: AssemblyAISpeechSynthesizer(), llm: ReadAloudLLM()) { rate in StreamingPCMPlayer(sampleRate: AssemblyAISpeechSynthesizer.sampleRate, rate: rate) } @@ -43,7 +58,7 @@ /// cooperative pool (`BlockingIPC.run`). When AX can't reach a /// focused element at all, it falls back to copying the selection /// (`SelectionCopy`). - public static func focusedSelection() async -> String? { + static func focusedSelection() async -> String? { let read = await BlockingIPC.run { FocusCapture.captureSelectedText(maxChars: maxCharacters) } return await resolve(read, copy: SelectionCopy()) } @@ -57,18 +72,49 @@ } } - /// Reads `text` in `style`: rewritten for listening first when it skips - /// jargon, then played at its rate. - public func speak(_ text: String, style: ReadAloudStyle = .standard) async throws { - let spoken = style.skipsJargon ? try await forListening(text) : text - // A stop that landed while the rewrite's answer was already on its way - // back: nothing should play, and no TTS socket should open. - try Task.checkCancellation() + /// Reads `text` in `style`, a piece at a time (`ListeningChunks`): each + /// rewritten for listening first when it skips jargon, then spoken in its + /// voice and played at its rate, the next asked for only as the listening + /// catches up (`bufferTarget`). `onLevel`, when + /// given, hears the loudness of what's playing about 20 times a second + /// while it plays, then a final 0 — the pill's meter. + func speak( + _ text: String, style: ReadAloudStyle = .standard, onLevel: (@Sendable (Float) -> Void)? = nil + ) async throws { + let pieces = ListeningChunks.chunks(of: text) + guard !pieces.isEmpty else { return } let player = makeSink(style.rate) + // At the mic meter's rate, and only on a change: the silence before the + // first audio and between words is a run of zeros the pill needn't hear. + let meter = onLevel.map { onLevel in + Task { + var last: Float? + while !Task.isCancelled { + let level = player.currentLevel + if level != last { onLevel(level) } + last = level + try? await Task.sleep(for: .seconds(MicCapture.meterIntervalSeconds)) + } + } + } + defer { + meter?.cancel() + onLevel?(0) + } try await withTaskCancellationHandler { do { - for try await chunk in synthesizer.synthesize(spoken) { - player.enqueue(chunk) + for piece in pieces { + try await waitForRoom(in: player) + let spoken = style.skipsJargon ? try await forListening(piece) : piece + // A stop that landed while the rewrite's answer was already on its + // way back: nothing more should play, and no request should go out. + try Task.checkCancellation() + for try await chunk in synthesizer.speech(for: spoken, voiceID: style.voiceID) { + player.enqueue(chunk) + } + // A stopped read's stream just ends — iteration doesn't throw on + // cancellation — so say it was stopped rather than drain as if done. + try Task.checkCancellation() } } catch { player.stop() @@ -80,14 +126,28 @@ } } - /// Answers a spoken `request` about `selection` and reads the answer aloud at - /// `style`'s rate. The answer is already written to be heard, so it skips the + /// Answers a spoken `request` about `selection` and reads the answer aloud in + /// `style`'s voice and rate. The answer is already written to be heard, so it skips the /// listening rewrite. No fallback: when the answer fails, there is nothing /// else to read. - public func answer(_ request: String, about selection: String, style: ReadAloudStyle) async throws { + func answer( + _ request: String, about selection: String, style: ReadAloudStyle, + onLevel: (@Sendable (Float) -> Void)? = nil + ) async throws { let reply = try await llm.answer(request, about: selection) try Task.checkCancellation() - try await speak(reply, style: ReadAloudStyle(rate: style.rate, skipsJargon: false)) + try await speak(reply, style: style.verbatim, onLevel: onLevel) + } + + /// Returns once less than `bufferTarget` seconds of audio are queued, so the + /// next piece is synthesized only as the listening reaches it. + private func waitForRoom(in player: any PCMSink) async throws { + // Sleep out the excess — it drains at listening speed — then look again: + // between a quarter second (should the clock stall) and a second, so a + // wait of minutes is a wake a second rather than four. + while player.bufferedSeconds > Self.bufferTarget { + try await Task.sleep(for: .seconds(min(1, max(0.25, player.bufferedSeconds - Self.bufferTarget)))) + } } /// `text` rewritten by `ReadAloudLLM`, or `text` itself when the rewrite diff --git a/Sources/BlurtEngine/TTS/SpeechSegmenter.swift b/Sources/BlurtEngine/TTS/SpeechSegmenter.swift index 6d0c744a..511e9da5 100644 --- a/Sources/BlurtEngine/TTS/SpeechSegmenter.swift +++ b/Sources/BlurtEngine/TTS/SpeechSegmenter.swift @@ -1,29 +1,20 @@ -/// Cuts text into the segments `AssemblyAISpeechSynthesizer` flushes one at a -/// time. +/// Cuts text at sentence ends — what `ListeningChunks` uses to split a +/// paragraph too long for one piece, so a piece never starts or ends +/// mid-sentence. /// -/// On AssemblyAI's streaming TTS, `Flush` is the only thing that starts -/// synthesis, and each flushed segment is synthesized as its own utterance, with -/// its own prosody and roughly 800 ms of padding. So where the cuts fall decides -/// both how soon the first audio arrives and how natural the speech sounds. The -/// rule is the one the AssemblyAI agent runtime measured +/// The rule is the one the AssemblyAI agent runtime measured /// (`aai-runtime/src/providers/tts/assemblyai-segment.ts`): /// /// - **Cut at sentence ends only.** Terminal punctuation plus any closing /// quotes or brackets, followed by whitespace or the end of the text. Never -/// cut at commas: a clause flushed alone gets a falling final intonation, and -/// per-clause flushing measured longer audio for no gain in latency. Line +/// at commas: a clause spoken alone gets a falling final intonation. Line /// breaks count as sentence ends too, because a heading or list item in a /// selection often has no punctuation. /// - **A segment needs at least two words.** One-word "sentences" are usually -/// abbreviations (`Dr.`, `e.g.`), and measured 25% longer audio when flushed -/// alone, so they join the sentence that follows. +/// abbreviations (`Dr.`, `e.g.`), so they join the sentence that follows. /// -/// The runtime also has a 40-character budget. It exists because an LLM -/// streams text in word by word, and without the budget time-to-first-audio -/// becomes the length of the first sentence. Blurt has the whole selection -/// before it sends anything, so it doesn't need that budget. `maxCharacters` -/// only splits a runaway sentence, such as a paragraph with no punctuation, so a -/// single flush can't take unreasonably long. +/// `maxCharacters` only splits a runaway sentence, such as a paragraph with no +/// punctuation. enum SpeechSegmenter { /// A sentence end: terminal punctuation plus closers, looking ahead for /// whitespace or the end so "3.5" and "v1.2" don't match. Curly closers are diff --git a/Sources/BlurtEngine/TTS/StreamingPCMPlayer.swift b/Sources/BlurtEngine/TTS/StreamingPCMPlayer.swift index 7e0c5e72..f0b0c6f3 100644 --- a/Sources/BlurtEngine/TTS/StreamingPCMPlayer.swift +++ b/Sources/BlurtEngine/TTS/StreamingPCMPlayer.swift @@ -26,9 +26,19 @@ import Synchronization /// which are only touched inside that lock (or, for the synchronizer clock read /// and `addRenderer` at init, through APIs AVFoundation documents as thread-safe). final class StreamingPCMPlayer: PCMSink, @unchecked Sendable { + /// One 50 ms window's loudness and where it sits on the media timeline. + private struct LevelWindow { + let start: Double + let end: Double + let level: Float + } + private struct State { var schedule: PCMSchedule var stopped = false + /// Each queued 50 ms window's loudness and where it sits on the media + /// timeline, oldest first, for `currentLevel`. + var levels: [LevelWindow] = [] } private let renderer = AVSampleBufferAudioRenderer() @@ -62,10 +72,19 @@ final class StreamingPCMPlayer: PCMSink, @unchecked Sendable { func enqueue(_ pcm: Data) { let frames = pcm.count / 2 guard frames > 0, let format else { return } + // Loudness per meter tick, measured before taking the lock the meter and + // the pacing both wait on; only the placement needs it. + let window = Int(Double(sampleRate) * MicCapture.meterIntervalSeconds) + let levels = PCMLevel.levels(of: pcm, windowFrames: window) state.withLock { state in guard !state.stopped else { return } let first = !state.schedule.started let at = state.schedule.place(frames: frames, now: CMTimeGetSeconds(synchronizer.currentTime())) + for (index, level) in levels.enumerated() { + let start = at + Double(index * window) / Double(sampleRate) + let end = min(start + Double(window) / Double(sampleRate), at + Double(frames) / Double(sampleRate)) + state.levels.append(LevelWindow(start: start, end: end, level: level)) + } let time = CMTime(seconds: at, preferredTimescale: sampleRate) guard let buffer = Self.sampleBuffer(pcm, frames: frames, format: format, at: time) else { return } renderer.enqueue(buffer) @@ -94,6 +113,11 @@ final class StreamingPCMPlayer: PCMSink, @unchecked Sendable { private static let drainSlack = Duration.seconds(2) + /// Seconds of audio queued ahead of what's playing, in listening time (the + /// rate applied), or 0 once stopped — how far ahead of the listener the + /// reading has got. + var bufferedSeconds: Double { (remaining() ?? 0) / rate } + /// Seconds of audio still ahead of the clock, or nil once stopped. private func remaining() -> Double? { state.withLock { state in @@ -101,6 +125,24 @@ final class StreamingPCMPlayer: PCMSink, @unchecked Sendable { } } + /// The loudness of what's playing right now, 0...1, or 0 between words, + /// before the first chunk, and once stopped. Reads the media clock, so it + /// follows the audio coming out of the speakers, not the audio arriving. + var currentLevel: Float { + state.withLock { state in + guard !state.stopped else { return 0 } + let now = CMTimeGetSeconds(synchronizer.currentTime()) + // Windows are in time order, so only the head can have expired. + if let live = state.levels.firstIndex(where: { $0.end > now }) { + state.levels.removeFirst(live) + } else { + state.levels.removeAll() + } + guard let window = state.levels.first, window.start <= now else { return 0 } + return window.level + } + } + /// Silences playback at once and discards anything queued. Idempotent. func stop() { state.withLock { state in @@ -108,6 +150,7 @@ final class StreamingPCMPlayer: PCMSink, @unchecked Sendable { state.stopped = true synchronizer.setRate(0, time: synchronizer.currentTime()) renderer.flush() + state.levels.removeAll() } } @@ -149,4 +192,13 @@ protocol PCMSink: Sendable { func drain() async /// Silences at once. Idempotent. func stop() + /// The loudness of what's playing now, 0...1, for the pill's meter. + var currentLevel: Float { get } + /// Seconds of audio queued ahead of what's playing, in listening time. + var bufferedSeconds: Double { get } +} + +extension PCMSink { + var currentLevel: Float { 0 } + var bufferedSeconds: Double { 0 } } diff --git a/Sources/BlurtEngine/UserAgent.swift b/Sources/BlurtEngine/UserAgent.swift index eeb78ee5..783409ed 100644 --- a/Sources/BlurtEngine/UserAgent.swift +++ b/Sources/BlurtEngine/UserAgent.swift @@ -145,4 +145,22 @@ extension URLRequest { mutating func setUserAgent() { setValue(UserAgent.current, forHTTPHeaderField: UserAgent.headerField) } + + /// A JSON `POST` of `body` to an AssemblyAI route: the raw API key in + /// `Authorization`, as every AssemblyAI request Blurt makes, the user agent, + /// and stably ordered keys. Read-aloud's two clients (`ReadAloudLLM`, + /// `AssemblyAISpeechSynthesizer`) both build theirs here. + static func assemblyAIJSONPost( + to url: URL, apiKey: String, timeout: TimeInterval, body: some Encodable + ) throws -> URLRequest { + var request = URLRequest(url: url, timeoutInterval: timeout) + request.httpMethod = "POST" + request.setValue(apiKey, forHTTPHeaderField: "Authorization") + request.setValue("application/json", forHTTPHeaderField: "Content-Type") + request.setUserAgent() + let encoder = JSONEncoder() + encoder.outputFormatting = .sortedKeys + request.httpBody = try encoder.encode(body) + return request + } } diff --git a/Tests/BlurtEngineTests/AssemblyAISpeechSynthesizerTests.swift b/Tests/BlurtEngineTests/AssemblyAISpeechSynthesizerTests.swift index 52d5f125..224ad3c0 100644 --- a/Tests/BlurtEngineTests/AssemblyAISpeechSynthesizerTests.swift +++ b/Tests/BlurtEngineTests/AssemblyAISpeechSynthesizerTests.swift @@ -1,377 +1,80 @@ import Foundation -import Synchronization import Testing @testable import AssemblyAI @testable import BlurtEngine -@Suite("AssemblyAISpeechSynthesizer wire format") +@Suite("AssemblyAISpeechSynthesizer") struct AssemblyAISpeechSynthesizerTests { - private func frame(_ json: String) -> AssemblyAISpeechSynthesizer.Frame { - AssemblyAISpeechSynthesizer.frame(from: Data(json.utf8)) - } - - @Test("connects to the streaming host with voice and sample rate as query params") - func url() { - #expect( - AssemblyAISpeechSynthesizer.url.absoluteString - == "wss://streaming-tts.assemblyai.com/v1/ws/?voice=jane&sample_rate=24000") - } - - @Test("commands are the service's Generate / Flush / Terminate frames") - func commands() { - #expect(AssemblyAISpeechSynthesizer.command("Generate", text: "Hi.") == #"{"text":"Hi.","type":"Generate"}"#) - #expect(AssemblyAISpeechSynthesizer.command("Flush") == #"{"type":"Flush"}"#) - } - - @Test("Audio frames decode their base64 PCM") - func audio() { - let pcm = Data([0x01, 0x00, 0xFF, 0x7F]) - #expect(frame(#"{"type":"Audio","audio":"\#(pcm.base64EncodedString())"}"#) == .audio(pcm)) - } - - @Test("FlushDone is the per-segment acknowledgement") - func flushDone() { - #expect(frame(#"{"type":"FlushDone"}"#) == .flushDone) - } - - @Test("Error frames carry their code, numeric or string") - func errors() { - #expect( - frame(#"{"type":"Error","error_code":1008,"error":"Unauthorized: Invalid API key"}"#) - == .error("(1008) Unauthorized: Invalid API key")) - #expect(frame(#"{"type":"Error","error_code":"x","error":" "}"#) == .error("(x) unknown error")) - #expect(frame(#"{"type":"Error"}"#) == .error("unknown error")) - } - - @Test("everything else — Begin, WordBoundaries, junk — is ignored, not fatal") - func ignored() { - #expect(frame(#"{"type":"Begin"}"#) == .ignored) - #expect(frame(#"{"type":"WordBoundaries","words":[]}"#) == .ignored) - #expect(frame(#"{"type":"Audio"}"#) == .ignored) - #expect(frame("not json") == .ignored) - #expect(frame("[1]") == .ignored) - } - - @Test("string and binary socket messages decode the same way") - func messages() { - #expect(AssemblyAISpeechSynthesizer.frame(from: .string(#"{"type":"FlushDone"}"#)) == .flushDone) - #expect(AssemblyAISpeechSynthesizer.frame(from: .data(Data(#"{"type":"FlushDone"}"#.utf8))) == .flushDone) - } - - @Test("a missing key fails before any socket opens") - func missingKey() async { - let synth = AssemblyAISpeechSynthesizer(apiKeyProvider: { nil }) - await #expect(throws: BlurtError.self) { - for try await _ in synth.synthesize("Hello there.") {} - } - } - - @Test("blank text finishes immediately with no audio") - func blankText() async throws { - let synth = AssemblyAISpeechSynthesizer(apiKeyProvider: { nil }) - var chunks = 0 - for try await _ in synth.synthesize(" ") { chunks += 1 } - #expect(chunks == 0) - } -} - -@Suite("AssemblyAISpeechSynthesizer exchange") -struct AssemblyAISpeechSynthesizerExchangeTests { - private let twoSentences = "The first sentence here. And the second one." - - private func synthesizer( - _ socket: FakeSpeechSocket, key: String? = "raw-key", request: ValueBox = ValueBox(nil) - ) -> AssemblyAISpeechSynthesizer { - AssemblyAISpeechSynthesizer( - apiKeyProvider: { key }, - connect: { sent in - request.value = sent - return socket - }) - } - - @Test("flushes each segment, streams audio in order, and terminates once every flush is acked") - func happyPath() async throws { - let socket = FakeSpeechSocket(respond: FakeSpeechSocket.service()) - let request = ValueBox(nil) + private func synthesizer(_ transport: FakeSpeechTransport, key: String? = "raw-key") -> AssemblyAISpeechSynthesizer { + AssemblyAISpeechSynthesizer(apiKeyProvider: { key }, transport: transport) + } + + private func body(_ request: URLRequest) throws -> [String: Any] { + let data = try #require(request.httpBody) + return try #require(try JSONSerialization.jsonObject(with: data) as? [String: Any]) + } + + @Test("posts the documented JSON to the US host's /v1/speech, with the raw key") + func request() throws { + let request = try AssemblyAISpeechSynthesizer.request(for: "Hi there.", voiceID: "aai-sara", apiKey: "raw-key") + #expect(request.url?.absoluteString == "https://tts.us.assemblyai.com/v1/speech") + #expect(request.httpMethod == "POST") + #expect(request.value(forHTTPHeaderField: "Authorization") == "raw-key") + #expect(request.value(forHTTPHeaderField: "Content-Type") == "application/json") + #expect(request.timeoutInterval == AssemblyAISpeechSynthesizer.timeout) + let fields = try body(request) + #expect(fields["text"] as? String == "Hi there.") + #expect(fields["voice_id"] as? String == "aai-sara") + #expect(fields["sample_rate"] as? Int == 24_000) + #expect(fields["encoding"] as? String == "pcm_s16le") + // Nothing undocumented, and not the retired bare `voice`. + #expect(Set(fields.keys) == ["text", "voice_id", "sample_rate", "encoding"]) + } + + @Test("streams the response body's audio in order") + func streams() async throws { + let transport = FakeSpeechTransport { _ in (200, [Data([1, 0, 2, 0]), Data([3, 0])]) } var chunks: [Data] = [] - for try await chunk in synthesizer(socket, request: request).synthesize(twoSentences) { + for try await chunk in synthesizer(transport).speech(for: "Read this.", voiceID: "aai-titus") { chunks.append(chunk) } - #expect(chunks == [Data([1, 0]), Data([2, 0])]) - #expect( - socket.sent == [ - #"{"text":"The first sentence here.","type":"Generate"}"#, #"{"type":"Flush"}"#, - #"{"text":"And the second one.","type":"Generate"}"#, #"{"type":"Flush"}"#, - #"{"type":"Terminate"}"#, - ]) - #expect(socket.closedNormally == true) - let sent = try #require(request.value) - // The raw key — `Bearer ` upgrades fine and is then refused in-band. - #expect(sent.value(forHTTPHeaderField: "Authorization") == "raw-key") - #expect(sent.url == AssemblyAISpeechSynthesizer.url) - } - - @Test("an in-band Error frame fails the stream and releases the socket") - func serverError() async { - let socket = FakeSpeechSocket { _ in - [#"{"type":"Error","error_code":1008,"error":"Unauthorized: Invalid API key"}"#] + #expect(chunks == [Data([1, 0, 2, 0]), Data([3, 0])]) + #expect(transport.spoken == ["Read this."]) + #expect(try body(#require(transport.requests.first))["voice_id"] as? String == "aai-titus") + } + + @Test("a sample split across network reads is carried over whole") + func carriesOddBytes() async throws { + let transport = FakeSpeechTransport { _ in (200, [Data([1]), Data([0, 2, 0, 3]), Data([0])]) } + var audio = Data() + for try await chunk in synthesizer(transport).speech(for: "Read this.") { + #expect(chunk.count % 2 == 0) + audio.append(chunk) } - await #expect(throws: SpeechSynthesisError.server("(1008) Unauthorized: Invalid API key")) { - for try await _ in synthesizer(socket).synthesize(twoSentences) {} - } - #expect(socket.closedNormally == false) + #expect(audio == Data([1, 0, 2, 0, 3, 0])) } - @Test("a connection dropped mid-exchange fails the stream instead of hanging") - func dropped() async { - let socket = FakeSpeechSocket(respond: { _ in [] }) - await #expect(throws: URLError.self) { - let stream = synthesizer(socket).synthesize(twoSentences) - socket.drop() - for try await _ in stream {} + @Test("a failed request reports the body's code and message, or the bare status") + func failures() async { + let refused = FakeSpeechTransport { _ in + (401, [Data(#"{"code":"unauthorized","message":"Invalid API key"}"#.utf8)]) } - #expect(socket.closedNormally == false) - } - - @Test("a server that goes silent times out and releases the socket") - func silentServer() async { - let socket = FakeSpeechSocket(respond: { _ in [] }) - let synth = AssemblyAISpeechSynthesizer( - apiKeyProvider: { "k" }, receiveTimeout: .milliseconds(50), connect: { _ in socket }) - await #expect(throws: SpeechSynthesisError.timedOut) { - for try await _ in synth.synthesize("Say something here.") {} - } - #expect(socket.closedNormally == false) - } - - @Test("Begin, WordBoundaries and empty audio don't count as audio or acks") - func noise() async throws { - let socket = FakeSpeechSocket { command in - guard command.contains(#""Flush""#) else { return [#"{"type":"Begin"}"#] } - return [ - #"{"type":"Audio","audio":""}"#, #"{"type":"WordBoundaries"}"#, - FakeSpeechSocket.audio([9, 9]), FakeSpeechSocket.flushDone, - ] - } - var chunks: [Data] = [] - for try await chunk in synthesizer(socket).synthesize("Just one sentence.") { chunks.append(chunk) } - #expect(chunks == [Data([9, 9])]) - } - - @Test("cancelling the consumer closes the socket abruptly") - func cancellation() async throws { - let socket = FakeSpeechSocket(respond: { _ in [] }) - let synth = synthesizer(socket) - let consumer = Task { - for try await _ in synth.synthesize(twoSentences) {} + await #expect(throws: SpeechSynthesisError.server("(unauthorized) Invalid API key")) { + for try await _ in synthesizer(refused).speech(for: "Read this.") {} } - while socket.sent.isEmpty { await Task.yield() } - consumer.cancel() - _ = await consumer.result - while socket.closedNormally == nil { await Task.yield() } - #expect(socket.closedNormally == false) + #expect(AssemblyAISpeechSynthesizer.failure(status: 502, body: Data("".utf8)) == "HTTP 502") } -} -@Suite("SelectionSpeaker") -struct SelectionSpeakerTests { - private func speaker( - _ socket: FakeSpeechSocket, sink: RecordingSink, - gateway: any HTTPTransport = FakeHTTPTransport.failing(with: URLError(.notConnectedToInternet)), - rate: ValueBox = ValueBox(nil) - ) -> SelectionSpeaker { - SelectionSpeaker( - synthesizer: AssemblyAISpeechSynthesizer(apiKeyProvider: { "k" }, connect: { _ in socket }), - llm: ReadAloudLLM(apiKeyProvider: { "k" }, transport: gateway), - makeSink: { played in - rate.value = played - return sink - }) - } - - /// The text of every `Generate` the speaker sent, in order. - private func generated(_ socket: FakeSpeechSocket) -> [String] { - socket.sent.compactMap { command in - let object = (try? JSONSerialization.jsonObject(with: Data(command.utf8))) as? [String: String] - return object?["type"] == "Generate" ? object?["text"] : nil - } - } - - private let fastSkipping = ReadAloudStyle(rate: 2, skipsJargon: true) - - @Test("plays every chunk in order, then drains") - func plays() async throws { - let sink = RecordingSink() - try await speaker(FakeSpeechSocket(respond: FakeSpeechSocket.service()), sink: sink) - .speak("The first sentence here. And the second one.") - #expect(sink.chunks == [Data([1, 0]), Data([2, 0])]) - #expect(sink.drained) - } - - @Test("a synthesis failure silences playback and surfaces the error") - func failure() async { - let sink = RecordingSink() - let socket = FakeSpeechSocket { _ in [#"{"type":"Error","error":"boom"}"#] } - await #expect(throws: SpeechSynthesisError.server("boom")) { - try await speaker(socket, sink: sink).speak("Say something here.") - } - #expect(sink.stopped) - #expect(!sink.drained) - } - - @Test("cancelling the speaking task stops playback") - func cancellation() async { - let sink = RecordingSink() - let socket = FakeSpeechSocket(respond: { _ in [] }) - let speaking = Task { try await speaker(socket, sink: sink).speak("Say something here.") } - while socket.sent.isEmpty { await Task.yield() } - speaking.cancel() - _ = await speaking.result - #expect(sink.stopped) - } - - @Test("the standard style reads the selection verbatim at natural pace, with no gateway call") - func standardStyle() async throws { - let sink = RecordingSink() - let socket = FakeSpeechSocket(respond: FakeSpeechSocket.service()) - let calls = Counter() - let gateway = FakeHTTPTransport { _ in - _ = calls.next() - return (200, completion("unused")) - } - let rate = ValueBox(nil) - try await speaker(socket, sink: sink, gateway: gateway, rate: rate).speak("Run `npm ci` first.") - #expect(generated(socket) == ["Run `npm ci` first."]) - #expect(rate.value == 1) - #expect(calls.value == 0) - } - - @Test("skipping jargon speaks the gateway's rewrite, at the style's rate") - func skippingRewrites() async throws { - let sink = RecordingSink() - let socket = FakeSpeechSocket(respond: FakeSpeechSocket.service()) - let gateway = FakeHTTPTransport { _ in (200, completion("Run the command first.")) } - let rate = ValueBox(nil) - try await speaker(socket, sink: sink, gateway: gateway, rate: rate) - .speak("Run `npm ci --prefer-offline` first.", style: fastSkipping) - #expect(generated(socket) == ["Run the command first."]) - #expect(rate.value == 2) - #expect(sink.drained) - } - - @Test("a failed rewrite falls back to reading the selection verbatim") - func rewriteFailureFallsBack() async throws { - let sink = RecordingSink() - let socket = FakeSpeechSocket(respond: FakeSpeechSocket.service()) - let gateway = FakeHTTPTransport { _ in (503, Data()) } - try await speaker(socket, sink: sink, gateway: gateway).speak("Email jo@example.com today.", style: fastSkipping) - #expect(generated(socket) == ["Email jo@example.com today."]) - } - - @Test("an answer is spoken as the gateway wrote it, at the style's rate") - func answers() async throws { - let sink = RecordingSink() - let socket = FakeSpeechSocket(respond: FakeSpeechSocket.service()) - let gateway = FakeHTTPTransport { _ in (200, completion("It reports a strong quarter.")) } - let rate = ValueBox(nil) - try await speaker(socket, sink: sink, gateway: gateway, rate: rate) - .answer("Summarize this.", about: "Revenue rose `12%` in Q3.", style: fastSkipping) - // One gateway call (the answer, not a listening rewrite of it), then speech. - #expect(generated(socket) == ["It reports a strong quarter."]) - #expect(rate.value == 2) - } - - @Test("a failed answer surfaces its error and opens no socket") - func answerFailure() async { - let sink = RecordingSink() - let socket = FakeSpeechSocket(respond: FakeSpeechSocket.service()) - let gateway = FakeHTTPTransport { _ in (503, Data()) } - await #expect(throws: ReadAloudLLMError.status(503)) { - try await speaker(socket, sink: sink, gateway: gateway).answer("Explain.", about: "x", style: .standard) - } - #expect(socket.sent.isEmpty) - } - - @Test("a stop during the rewrite plays nothing, rather than falling back") - func cancelledDuringRewrite() async { - let sink = RecordingSink() - let socket = FakeSpeechSocket(respond: FakeSpeechSocket.service()) - let gateway = HangingTransport() - let speaking = Task { - try await speaker(socket, sink: sink, gateway: gateway).speak("Say something here.", style: fastSkipping) + @Test("a missing key fails before any request, and blank text makes none") + func noRequest() async throws { + let transport = FakeSpeechTransport() + await #expect(throws: BlurtError.self) { + for try await _ in synthesizer(transport, key: nil).speech(for: "Hello there.") {} } - while !gateway.asked { await Task.yield() } - speaking.cancel() - await #expect(throws: CancellationError.self) { try await speaking.value } - #expect(socket.sent.isEmpty) - #expect(sink.chunks.isEmpty) - } - - @Test("a rewrite that answers after the stop still plays nothing and opens no socket") - func cancelledAsRewriteLands() async { - let sink = RecordingSink() - let connects = Counter() - let socket = FakeSpeechSocket(respond: FakeSpeechSocket.service()) - let gateway = LateAnsweringTransport() - let speaker = SelectionSpeaker( - synthesizer: AssemblyAISpeechSynthesizer( - apiKeyProvider: { "k" }, - connect: { _ in - _ = connects.next() - return socket - }), - llm: ReadAloudLLM(apiKeyProvider: { "k" }, transport: gateway), - makeSink: { _ in sink }) - let speaking = Task { try await speaker.speak("Say something here.", style: fastSkipping) } - while !gateway.asked { await Task.yield() } - speaking.cancel() - gateway.release() - await #expect(throws: CancellationError.self) { try await speaking.value } - #expect(connects.value == 0) - #expect(sink.chunks.isEmpty) - } -} - -/// A gateway that ignores cancellation: it waits for `release()`, then answers -/// 200 anyway, the way a response already in flight lands after a stop. -private final class LateAnsweringTransport: HTTPTransport { - private let state = Mutex((asked: false, released: false)) - var asked: Bool { state.withLock { $0.asked } } - func release() { state.withLock { $0.released = true } } - - func data(for request: URLRequest) async throws -> (Data, URLResponse) { - state.withLock { $0.asked = true } - while !state.withLock({ $0.released }) { await Task.yield() } - let url = try #require(request.url) - let response = try #require(HTTPURLResponse(url: url, statusCode: 200, httpVersion: nil, headerFields: nil)) - return (completion("Say something here."), response) - } - - func upload( - for request: URLRequest, streaming body: AsyncThrowingStream, - delegate: (any URLSessionTaskDelegate)? - ) async throws -> (Data, URLResponse) { - throw URLError(.unsupportedURL) - } -} - -/// A gateway that never answers, until the request's task is cancelled. -private final class HangingTransport: HTTPTransport { - private let called = Mutex(false) - var asked: Bool { called.withLock { $0 } } - - func data(for request: URLRequest) async throws -> (Data, URLResponse) { - called.withLock { $0 = true } - try await Task.sleep(for: .seconds(60)) - throw URLError(.timedOut) - } - - func upload( - for request: URLRequest, streaming body: AsyncThrowingStream, - delegate: (any URLSessionTaskDelegate)? - ) async throws -> (Data, URLResponse) { - throw URLError(.unsupportedURL) + var chunks = 0 + for try await _ in synthesizer(transport).speech(for: " ") { chunks += 1 } + #expect(chunks == 0) + #expect(transport.requests.isEmpty) } } diff --git a/Tests/BlurtEngineTests/DictationKeyRouterTests.swift b/Tests/BlurtEngineTests/DictationKeyRouterTests.swift index 40d1a639..768ad6e6 100644 --- a/Tests/BlurtEngineTests/DictationKeyRouterTests.swift +++ b/Tests/BlurtEngineTests/DictationKeyRouterTests.swift @@ -21,6 +21,16 @@ struct DictationKeyRouterTests { .flagsChanged(keyCode: keyCode, triggerFlagIsOn: false) } + @Test("a click while the trigger is down is a modifier-click: it cancels the press") + func clickCancelsPress() { + var router = DictationKeyRouter(triggerKeyCodes: [55]) + #expect(router.handle(.flagsChanged(keyCode: 55, triggerFlagIsOn: true), at: .zero) == .start) + #expect(router.handle(.pointerDown, at: .milliseconds(80)) == .cancel) + #expect(router.handle(.flagsChanged(keyCode: 55, triggerFlagIsOn: false), at: .milliseconds(150)) == .none) + // With nothing pressed, a click is nothing. + #expect(router.handle(.pointerDown, at: .seconds(1)) == .none) + } + @Test("a held press is start → stop") func holdIsStartStop() { var router = DictationKeyRouter(triggerKeyCodes: [trigger]) diff --git a/Tests/BlurtEngineTests/LongReadTests.swift b/Tests/BlurtEngineTests/LongReadTests.swift new file mode 100644 index 00000000..35911d44 --- /dev/null +++ b/Tests/BlurtEngineTests/LongReadTests.swift @@ -0,0 +1,122 @@ +import Foundation +import Synchronization +import Testing + +@testable import AssemblyAI +@testable import BlurtEngine + +/// Long articles: reading starts after the first sentence, text goes to the +/// service only as fast as it's listened to, and skip-jargon rewrites one chunk +/// at a time instead of the whole article up front. +@Suite("Long reads") +struct LongReadTests { + /// An article of `paragraphs` paragraphs, each a dozen sentences long. + private func article(paragraphs: Int) -> String { + (1...paragraphs).map { index in + (1...12).map { "Paragraph \(index) sentence \($0) runs on a little." }.joined(separator: " ") + } + .joined(separator: "\n\n") + } + + private func speaker( + _ transport: FakeSpeechTransport, sink: RecordingSink, + gateway: any HTTPTransport = FakeHTTPTransport.failing( + with: URLError(.notConnectedToInternet)) + ) -> SelectionSpeaker { + SelectionSpeaker( + synthesizer: AssemblyAISpeechSynthesizer(apiKeyProvider: { "k" }, transport: transport), + llm: ReadAloudLLM(apiKeyProvider: { "k" }, transport: gateway), + makeSink: { _ in sink }) + } + + @Test("selections up to 100,000 characters are read, so a long article isn't cut off") + func raisedCap() { + #expect(SelectionSpeaker.maxCharacters == 100_000) + } + + @Test("the next piece is asked for only once little audio is queued ahead") + func pacesByPlayback() async throws { + let transport = FakeSpeechTransport() + let sink = RecordingSink() + let text = article(paragraphs: 4) + let pieces = ListeningChunks.chunks(of: text) + #expect(pieces.count > 2) + // The first piece's audio leaves a minute waiting to be heard. + sink.bufferedAfterEnqueue = 60 + let reading = Task { try await speaker(transport, sink: sink).speak(text) } + try await Task.sleep(for: .milliseconds(600)) + #expect(transport.spoken == [pieces[0]], "a piece went out while a minute was still queued") + // The listener catches up. + sink.bufferedAfterEnqueue = nil + sink.bufferedSeconds = 0 + try await reading.value + #expect(transport.spoken == pieces) + } + + @Test("the first piece is short, so reading starts after a paragraph, not the article") + func shortFirstPiece() { + let pieces = ListeningChunks.chunks(of: article(paragraphs: 6)) + #expect(pieces[0].count <= ListeningChunks.firstMaxCharacters) + #expect(pieces.dropFirst().allSatisfy { $0.count <= ListeningChunks.maxCharacters }) + } + + @Test("skip-jargon rewrites a long article a piece at a time, and reading starts after the first") + func rewritesInPieces() async throws { + let text = article(paragraphs: 6) + let pieces = ListeningChunks.chunks(of: text) + let gateway = GatedRewrites() + let transport = FakeSpeechTransport() + let sink = RecordingSink() + let reading = Task { + try await speaker(transport, sink: sink, gateway: gateway) + .speak(text, style: ReadAloudStyle(rate: 1, skipsJargon: true)) + } + // Only the first piece's rewrite is answered, and it's already being read. + gateway.release(1) + while transport.spoken.isEmpty { await Task.yield() } + for _ in 0..<200 { await Task.yield() } + #expect(transport.spoken == ["Rewritten 1. The words read aloud."]) + #expect(gateway.asked <= 2, "the whole article went to the gateway before reading") + gateway.release(.max) + try await reading.value + #expect(transport.spoken.count == pieces.count) + #expect(gateway.asked == pieces.count) + } + + @Test("pieces break at paragraphs, each within its limit, losing no text") + func chunks() { + let paragraph = String(repeating: "Some words in a sentence. ", count: 20) + let text = Array(repeating: paragraph.trimmingCharacters(in: .whitespaces), count: 5).joined(separator: "\n\n") + let chunks = ListeningChunks.chunks(of: text, maxCharacters: 1_200, firstMaxCharacters: 1_200) + #expect(chunks.count > 1) + #expect(chunks.allSatisfy { $0.count <= 1_200 }) + #expect( + chunks.joined(separator: " ").split(whereSeparator: \.isWhitespace) + == text.split(whereSeparator: \.isWhitespace)) + #expect(ListeningChunks.chunks(of: "Short.") == ["Short."]) + #expect(ListeningChunks.chunks(of: " \n\n ").isEmpty) + } +} + +/// A gateway answering rewrite requests only as the test releases them, each +/// answer naming its request: "Rewritten 1.", "Rewritten 2.", … +private final class GatedRewrites: GatewayStub, Sendable { + private let state = Mutex((asked: 0, released: 0)) + + var asked: Int { state.withLock(\.asked) } + + func release(_ count: Int) { + state.withLock { $0.released = count == .max ? .max : $0.released + count } + } + + func data(for request: URLRequest) async throws -> (Data, URLResponse) { + let number = state.withLock { state -> Int in + state.asked += 1 + return state.asked + } + while state.withLock({ $0.released < number }) { + try await Task.sleep(for: .milliseconds(5)) + } + return try ok(request, completion("Rewritten \(number). The words read aloud.")) + } +} diff --git a/Tests/BlurtEngineTests/PCMLevelTests.swift b/Tests/BlurtEngineTests/PCMLevelTests.swift new file mode 100644 index 00000000..03ce87ab --- /dev/null +++ b/Tests/BlurtEngineTests/PCMLevelTests.swift @@ -0,0 +1,34 @@ +import Foundation +import Testing + +@testable import BlurtEngine + +@Suite("PCMLevel") +struct PCMLevelTests { + private func pcm(_ samples: [Int16]) -> Data { + samples.withUnsafeBufferPointer { Data(buffer: $0) } + } + + @Test("silence reads 0, full scale reads 1, and a quieter voice reads in between") + func scale() throws { + #expect(PCMLevel.levels(of: pcm(Array(repeating: 0, count: 100)), windowFrames: 100) == [0]) + #expect(PCMLevel.levels(of: pcm(Array(repeating: .max, count: 100)), windowFrames: 100) == [1]) + let quiet = try #require(PCMLevel.levels(of: pcm(Array(repeating: 1_000, count: 100)), windowFrames: 100).first) + #expect(quiet > 0 && quiet < 1) + } + + @Test("one level per window, the last one short, an odd byte ignored") + func windows() { + var data = pcm(Array(repeating: 0, count: 50) + Array(repeating: .max, count: 25)) + data.append(0x7F) + #expect(PCMLevel.levels(of: data, windowFrames: 50) == [0, 1]) + #expect(PCMLevel.levels(of: Data(), windowFrames: 50).isEmpty) + #expect(PCMLevel.levels(of: data, windowFrames: 0).isEmpty) + } + + @Test("a read wears its own pill state, steady and announced") + func speakingPill() { + #expect(OverlayUIState.speaking.accessibilityLabel == "Reading aloud.") + #expect(OverlayUIState.speaking.noticeDwellSeconds == nil) + } +} diff --git a/Tests/BlurtEngineTests/PersistedSettingsTests.swift b/Tests/BlurtEngineTests/PersistedSettingsTests.swift index 1a9cca1f..10889300 100644 --- a/Tests/BlurtEngineTests/PersistedSettingsTests.swift +++ b/Tests/BlurtEngineTests/PersistedSettingsTests.swift @@ -44,8 +44,8 @@ struct PersistedSettingsTests { @Test("the roster carries no stale or duplicate keys") func rosterHasNoStrays() { - // Exactly the fifteen known stores' keys (OverlayOriginStore and - // ReadAloudStyleStore contribute two each, StyleProfileStore three): a removed store + // Exactly the fourteen known stores' keys (OverlayOriginStore and SoundPackStore two, + // ReadAloudStyleStore and StyleProfileStore three each): a removed store // must leave the roster in the same change, and a key listed twice would hint // at a copy-paste slip. #expect(PersistedSettings.allDefaultsKeys.count == 20) @@ -64,8 +64,9 @@ struct PersistedSettingsTests { TriggerKeyStore.defaultsKey, TriggerActivationStore.defaultsKey, TriggerKeyStyleStore.defaultsKey, - TriggerKeyStyleStore.stashDefaultsKey, + TriggerKeyActivationStore.defaultsKey, SoundPackStore.defaultsKey, + SoundPackStore.readAloudDefaultsKey, KeyTermsStore.defaultsKey, DeveloperModeStore.defaultsKey, EnhancedTranscriptsStore.defaultsKey, @@ -77,10 +78,9 @@ struct PersistedSettingsTests { OverlayOriginStore.yDefaultsKey, MicDeviceStore.defaultsKey, TextShortcutStore.defaultsKey, - SelectionSpeechStore.defaultsKey, ReadAloudStyleStore.speedDefaultsKey, ReadAloudStyleStore.skipsJargonDefaultsKey, - SelectionAskStore.defaultsKey, + ReadAloudStyleStore.voiceDefaultsKey, ] #expect(storeKeys == Set(DefaultsKey.allCases.map(\.key))) // No two stores sharing a slot — the Set above would have quietly absorbed a diff --git a/Tests/BlurtEngineTests/ReadAloudLLMTests.swift b/Tests/BlurtEngineTests/ReadAloudLLMTests.swift index 3805a6d5..6a5e51a6 100644 --- a/Tests/BlurtEngineTests/ReadAloudLLMTests.swift +++ b/Tests/BlurtEngineTests/ReadAloudLLMTests.swift @@ -142,6 +142,23 @@ struct ReadAloudLLMTests { ]) } + @Test("an ask about a whole article sends only its opening stretch, within the model's context") + func answerClipsLongSelection() async throws { + let sent = ValueBox(nil) + let transport = FakeHTTPTransport { request in + sent.value = request + return (200, completion("A summary.")) + } + let article = String(repeating: "word ", count: 20_000) + _ = try await llm(transport).answer("Summarize this.", about: article) + let body = try #require(sent.value?.httpBody) + let object = try #require(try JSONSerialization.jsonObject(with: body) as? [String: Any]) + let messages = try #require(object["messages"] as? [[String: String]]) + let user = try #require(messages.last?["content"]) + #expect(user.contains(String(article.prefix(ReadAloudLLM.answerSelectionCharacters)) + "\n")) + #expect(user.count < ReadAloudLLM.answerSelectionCharacters + 200) + } + @Test("an answer cut off at max_tokens is still read, and isn't held to the rewrite's length rule") func answerTruncated() async throws { let long = String(repeating: "Here is a long answer. ", count: 20) diff --git a/Tests/BlurtEngineTests/ReadAloudPillAndCueTests.swift b/Tests/BlurtEngineTests/ReadAloudPillAndCueTests.swift new file mode 100644 index 00000000..92eadda4 --- /dev/null +++ b/Tests/BlurtEngineTests/ReadAloudPillAndCueTests.swift @@ -0,0 +1,87 @@ +import Foundation +import Testing + +@testable import BlurtEngine + +/// The pill while and after a read plays, and when each cue pair reloads after +/// the output route changes. +@Suite("Read-aloud pill and cues") +struct ReadAloudPillAndCueTests { + /// A phase, with the pill it shows while a read plays (nil: leave the read's + /// pill) and once a read ends. + private struct Row { + let phase: PipelinePhase + let whileSpeaking: OverlayUIState? + let afterSpeech: OverlayUIState + + init(_ phase: PipelinePhase, _ whileSpeaking: OverlayUIState?, _ afterSpeech: OverlayUIState) { + (self.phase, self.whileSpeaking, self.afterSpeech) = (phase, whileSpeaking, afterSpeech) + } + } + + private static let phases: [Row] = [ + Row(.idle, nil, .idle), + Row(.connecting, .connecting, .connecting), + Row(.recording, .recording, .recording), + Row(.transcribing, nil, .processing), + Row(.injecting, nil, .processing), + Row(.pasted, nil, .idle), + Row(.noTarget, nil, .idle), + Row(.cancelled, nil, .idle), + Row(.handedBack("Explain."), nil, .idle), + Row(.failed(.apiKeyMissing), nil, .idle), + ] + + @Test("while a read plays, only a new capture takes the pill; when it ends, no old notice replays") + func pillAroundARead() { + for row in Self.phases { + let (phase, whileSpeaking, afterSpeech) = (row.phase, row.whileSpeaking, row.afterSpeech) + #expect(phase.overlayState(whileSpeaking: true) == whileSpeaking, "\(phase) while speaking") + #expect(phase.overlayState(whileSpeaking: false) == phase.overlayState, "\(phase) with no read") + #expect(phase.overlayStateAfterSpeech == afterSpeech, "\(phase) after a read") + } + } + + @Test("read-aloud's cues reload at once when nothing is capturing") + func readCuesReloadWhenIdle() { + var schedule = CueReprimeSchedule() + #expect(schedule.routeChanged() == .init(dictation: false, readAloud: true)) + // Dictation's still waits for a press or a terminal phase. + #expect(schedule.phase(.connecting) == .init(dictation: true, readAloud: false)) + #expect(schedule.phase(.recording) == .none) + } + + @Test("mid-capture, read-aloud's cues wait for the capture's end; dictation's reload at the next press") + func readCuesWaitForTheCapture() { + var schedule = CueReprimeSchedule() + _ = schedule.phase(.connecting) + _ = schedule.phase(.recording) + #expect(schedule.routeChanged() == .none) + #expect(schedule.phase(.transcribing) == .none) + #expect(schedule.phase(.pasted) == .init(dictation: true, readAloud: true)) + #expect(schedule.phase(.idle) == .none) + } + + @Test("with no route change, nothing reloads") + func nothingPending() { + var schedule = CueReprimeSchedule() + for row in Self.phases { #expect(schedule.phase(row.phase) == .none) } + } + + @Test("the views' key decisions: extra rows, the only dictating key, whether read-aloud does anything") + func bindingsViewHelpers() { + let defaults = TriggerKeyBindings(boundKeyCode: 54, choicesRaw: "", modesRaw: "", sharedModeRaw: "") + #expect(defaults.extraKeys == [.leftOption]) + #expect(defaults.isOnlyDictatingKey(.rightCommand)) + #expect(defaults.readsOrAsks) + #expect(!defaults.asks) + let asking = TriggerKeyBindings( + boundKeyCode: 54, choicesRaw: #"{"58":"","61":"current","59":"ask"}"#, modesRaw: "", sharedModeRaw: "") + #expect(asking.extraKeys == [.rightOption, .leftControl]) + #expect(!asking.isOnlyDictatingKey(.rightCommand)) + #expect(asking.readsOrAsks) + #expect(asking.asks) + let silent = TriggerKeyBindings(boundKeyCode: 54, choicesRaw: #"{"58":""}"#, modesRaw: "", sharedModeRaw: "") + #expect(!silent.readsOrAsks) + } +} diff --git a/Tests/BlurtEngineTests/ReadAloudStyleStoreTests.swift b/Tests/BlurtEngineTests/ReadAloudStyleStoreTests.swift index 5b5b1c25..3c9ad890 100644 --- a/Tests/BlurtEngineTests/ReadAloudStyleStoreTests.swift +++ b/Tests/BlurtEngineTests/ReadAloudStyleStoreTests.swift @@ -5,11 +5,11 @@ import Testing @Suite("ReadAloudStyleStore") struct ReadAloudStyleStoreTests { - @Test("untouched: natural pace, the selection verbatim") - func defaultsToStandard() { - #expect(ReadAloudStyleStore(defaults: freshDefaults()).style == .standard) + @Test("untouched: the default voice at natural pace, skipping jargon") + func defaults() { + #expect(ReadAloudStyleStore(defaults: freshDefaults()).style == ReadAloudStyle(rate: 1, skipsJargon: true)) #expect(ReadAloudStyleStore.defaultSpeed == 1) - #expect(!ReadAloudStyleStore.defaultSkipsJargon) + #expect(ReadAloudStyleStore.defaultSkipsJargon) } @Test("speed and skipping are independent settings") @@ -17,10 +17,10 @@ struct ReadAloudStyleStoreTests { let defaults = freshDefaults() let store = ReadAloudStyleStore(defaults: defaults) defaults.set(1.5, forKey: ReadAloudStyleStore.speedDefaultsKey) - #expect(store.style == ReadAloudStyle(rate: 1.5, skipsJargon: false)) + #expect(store.style == ReadAloudStyle(rate: 1.5, skipsJargon: true)) defaults.removeObject(forKey: ReadAloudStyleStore.speedDefaultsKey) - defaults.set(true, forKey: ReadAloudStyleStore.skipsJargonDefaultsKey) - #expect(store.style == ReadAloudStyle(rate: 1, skipsJargon: true)) + defaults.set(false, forKey: ReadAloudStyleStore.skipsJargonDefaultsKey) + #expect(store.style == ReadAloudStyle(rate: 1, skipsJargon: false)) } @Test("a speed off the picker's list snaps to the nearest choice, and a non-number reads as the default") @@ -41,6 +41,36 @@ struct ReadAloudStyleStoreTests { #expect(store.style.rate == ReadAloudStyleStore.defaultSpeed) } + @Test("untouched, the voice is the service's own default, aai-jane") + func defaultVoice() { + #expect(ReadAloudStyleStore(defaults: freshDefaults()).style.voiceID == "aai-jane") + #expect(ReadAloudVoice.voice(id: ReadAloudStyleStore.defaultVoiceID)?.group == .builtIn) + } + + @Test("a chosen voice reads back; one the catalog doesn't list reads as the default") + func voice() { + let defaults = freshDefaults() + let store = ReadAloudStyleStore(defaults: defaults) + defaults.set("aai-yasmin", forKey: ReadAloudStyleStore.voiceDefaultsKey) + #expect(store.style.voiceID == "aai-yasmin") + // The retired bare name, and a voice the service would refuse with 3012. + for stale in ["jane", "aai-nonexistent", ""] { + defaults.set(stale, forKey: ReadAloudStyleStore.voiceDefaultsKey) + #expect(store.style.voiceID == ReadAloudStyleStore.defaultVoiceID) + } + } + + @Test("the catalog: unique aai- slugs, built-ins first, every voice labelled") + func catalog() { + let ids = ReadAloudVoice.catalog.map(\.id) + #expect(Set(ids).count == ids.count) + #expect(ids.allSatisfy { $0.hasPrefix("aai-") && $0 == $0.lowercased() }) + #expect(ReadAloudVoice.catalog.filter { $0.group == .builtIn }.count == 11) + #expect(ReadAloudVoice.catalog.first?.id == ReadAloudVoice.defaultID) + #expect(ReadAloudVoice.voice(id: "aai-steven")?.label == "Steven (Australian)") + #expect(ReadAloudVoice.voice(id: "aai-gunnar")?.label == "Gunnar") + } + @Test("the default speed is one the picker offers") func defaultIsAChoice() { #expect(ReadAloudStyleStore.speedChoices.contains(ReadAloudStyleStore.defaultSpeed)) diff --git a/Tests/BlurtEngineTests/SelectionAskStoreTests.swift b/Tests/BlurtEngineTests/SelectionAskStoreTests.swift deleted file mode 100644 index b92e29d4..00000000 --- a/Tests/BlurtEngineTests/SelectionAskStoreTests.swift +++ /dev/null @@ -1,23 +0,0 @@ -import Foundation -import Testing - -@testable import BlurtEngine - -@Suite("SelectionAskStore") -struct SelectionAskStoreTests { - @Test("defaults to off when unset — asking sends the selection to the LLM Gateway, so it is its own opt-in") - func defaultsToOff() { - #expect(!SelectionAskStore(defaults: freshDefaults()).isEnabled) - #expect(!SelectionAskStore.defaultValue) - } - - @Test("reads back the switch the Settings toggle writes") - func readsBackTheToggledSlot() { - let defaults = freshDefaults() - let store = SelectionAskStore(defaults: defaults) - defaults.set(false, forKey: SelectionAskStore.defaultsKey) - #expect(!store.isEnabled) - defaults.set(true, forKey: SelectionAskStore.defaultsKey) - #expect(store.isEnabled) - } -} diff --git a/Tests/BlurtEngineTests/SelectionSpeakerTests.swift b/Tests/BlurtEngineTests/SelectionSpeakerTests.swift new file mode 100644 index 00000000..78707e09 --- /dev/null +++ b/Tests/BlurtEngineTests/SelectionSpeakerTests.swift @@ -0,0 +1,248 @@ +import Foundation +import Synchronization +import Testing + +@testable import AssemblyAI +@testable import BlurtEngine + +@Suite("SelectionSpeaker") +struct SelectionSpeakerTests { + private func speaker( + _ transport: any SpeechTransport, sink: RecordingSink, + gateway: any HTTPTransport = FakeHTTPTransport.failing(with: URLError(.notConnectedToInternet)), + rate: ValueBox = ValueBox(nil) + ) -> SelectionSpeaker { + SelectionSpeaker( + synthesizer: AssemblyAISpeechSynthesizer(apiKeyProvider: { "k" }, transport: transport), + llm: ReadAloudLLM(apiKeyProvider: { "k" }, transport: gateway), + makeSink: { played in + rate.value = played + return sink + }) + } + + private let fastSkipping = ReadAloudStyle(rate: 2, skipsJargon: true) + + @Test("plays every chunk in order, then drains") + func plays() async throws { + let sink = RecordingSink() + let transport = FakeSpeechTransport { _ in (200, [Data([1, 0]), Data([2, 0])]) } + try await speaker(transport, sink: sink).speak("The first sentence here. And the second one.") + #expect(sink.chunks == [Data([1, 0]), Data([2, 0])]) + #expect(sink.drained) + } + + @Test("the meter hears the sink's level while it plays, then a final 0") + func meter() async throws { + let levels = ValueBox<[Float]>([]) + let sink = LevelSink(level: 0.6) + let speaker = SelectionSpeaker( + synthesizer: AssemblyAISpeechSynthesizer(apiKeyProvider: { "k" }, transport: FakeSpeechTransport()), + llm: ReadAloudLLM(apiKeyProvider: { "k" }, transport: FakeHTTPTransport.failing(with: URLError(.timedOut))), + makeSink: { _ in sink }) + try await speaker.speak("Say something here.") { level in levels.value.append(level) } + #expect(levels.value.last == 0) + #expect(levels.value.dropLast().allSatisfy { $0 == 0.6 }) + } + + @Test("a synthesis failure silences playback and surfaces the error") + func failure() async { + let sink = RecordingSink() + let transport = FakeSpeechTransport { _ in (500, [Data(#"{"message":"boom"}"#.utf8)]) } + await #expect(throws: SpeechSynthesisError.server("boom")) { + try await speaker(transport, sink: sink).speak("Say something here.") + } + #expect(sink.stopped) + #expect(!sink.drained) + } + + @Test("cancelling the speaking task stops playback") + func cancellation() async { + let sink = RecordingSink() + let transport = HangingSpeechTransport() + let speaking = Task { try await speaker(transport, sink: sink).speak("Say something here.") } + while !transport.asked { await Task.yield() } + speaking.cancel() + _ = await speaking.result + #expect(sink.stopped) + } + + @Test("the standard style reads the selection verbatim at natural pace, with no gateway call") + func standardStyle() async throws { + let sink = RecordingSink() + let transport = FakeSpeechTransport() + let calls = Counter() + let gateway = FakeHTTPTransport { _ in + _ = calls.next() + return (200, completion("unused")) + } + let rate = ValueBox(nil) + try await speaker(transport, sink: sink, gateway: gateway, rate: rate).speak("Run `npm ci` first.") + #expect(transport.spoken == ["Run `npm ci` first."]) + #expect(rate.value == 1) + #expect(calls.value == 0) + } + + @Test("skipping jargon speaks the gateway's rewrite, at the style's rate") + func skippingRewrites() async throws { + let sink = RecordingSink() + let transport = FakeSpeechTransport() + let gateway = FakeHTTPTransport { _ in (200, completion("Run the command first.")) } + let rate = ValueBox(nil) + try await speaker(transport, sink: sink, gateway: gateway, rate: rate) + .speak("Run `npm ci --prefer-offline` first.", style: fastSkipping) + #expect(transport.spoken == ["Run the command first."]) + #expect(rate.value == 2) + #expect(sink.drained) + } + + @Test("the style's voice names each request's voice_id, for reads and answers alike") + func voice() async throws { + let transport = FakeSpeechTransport() + let speaker = SelectionSpeaker( + synthesizer: AssemblyAISpeechSynthesizer(apiKeyProvider: { "k" }, transport: transport), + llm: ReadAloudLLM( + apiKeyProvider: { "k" }, transport: FakeHTTPTransport { _ in (200, completion("An answer.")) }), + makeSink: { _ in RecordingSink() }) + let style = ReadAloudStyle(rate: 1, skipsJargon: false, voiceID: "aai-buford") + try await speaker.speak("Read this.", style: style) + try await speaker.answer("Explain.", about: "x", style: style) + let voices = transport.requests.map { request in + (request.httpBody.flatMap { try? JSONSerialization.jsonObject(with: $0) } as? [String: Any])?["voice_id"] + as? String + } + #expect(voices == ["aai-buford", "aai-buford"]) + } + + @Test("a failed rewrite falls back to reading the selection verbatim") + func rewriteFailureFallsBack() async throws { + let sink = RecordingSink() + let transport = FakeSpeechTransport() + let gateway = FakeHTTPTransport { _ in (503, Data()) } + try await speaker(transport, sink: sink, gateway: gateway).speak("Email jo@example.com today.", style: fastSkipping) + #expect(transport.spoken == ["Email jo@example.com today."]) + } + + @Test("an answer is spoken as the gateway wrote it, at the style's rate") + func answers() async throws { + let sink = RecordingSink() + let transport = FakeSpeechTransport() + let gateway = FakeHTTPTransport { _ in (200, completion("It reports a strong quarter.")) } + let rate = ValueBox(nil) + try await speaker(transport, sink: sink, gateway: gateway, rate: rate) + .answer("Summarize this.", about: "Revenue rose `12%` in Q3.", style: fastSkipping) + // One gateway call (the answer, not a listening rewrite of it), then speech. + #expect(transport.spoken == ["It reports a strong quarter."]) + #expect(rate.value == 2) + } + + @Test("a failed answer surfaces its error and makes no TTS request") + func answerFailure() async { + let sink = RecordingSink() + let transport = FakeSpeechTransport() + let gateway = FakeHTTPTransport { _ in (503, Data()) } + await #expect(throws: ReadAloudLLMError.status(503)) { + try await speaker(transport, sink: sink, gateway: gateway).answer("Explain.", about: "x", style: .standard) + } + #expect(transport.requests.isEmpty) + } + + @Test("a stop during the rewrite plays nothing, rather than falling back") + func cancelledDuringRewrite() async { + let sink = RecordingSink() + let transport = FakeSpeechTransport() + let gateway = HangingTransport() + let speaking = Task { + try await speaker(transport, sink: sink, gateway: gateway).speak("Say something here.", style: fastSkipping) + } + while !gateway.asked { await Task.yield() } + speaking.cancel() + await #expect(throws: CancellationError.self) { try await speaking.value } + #expect(transport.requests.isEmpty) + #expect(sink.chunks.isEmpty) + } + + @Test("an ask stopped while the gateway answers plays nothing and makes no TTS request") + func cancelledAsAnswerLands() async { + await expectStoppedAsGatewayAnswers { speaker in + Task { try await speaker.answer("Explain.", about: "x", style: .standard) } + } + } + + @Test("a rewrite that answers after the stop still plays nothing and makes no TTS request") + func cancelledAsRewriteLands() async { + await expectStoppedAsGatewayAnswers { [fastSkipping] speaker in + Task { try await speaker.speak("Say something here.", style: fastSkipping) } + } + } + + /// Starts a read or answer whose gateway call answers only after the stop, + /// stops it mid-call, and checks it throws a cancellation and plays nothing. + private func expectStoppedAsGatewayAnswers(_ start: (SelectionSpeaker) -> Task) async { + let sink = RecordingSink() + let transport = FakeSpeechTransport() + let gateway = LateAnsweringTransport() + let speaker = SelectionSpeaker( + synthesizer: AssemblyAISpeechSynthesizer(apiKeyProvider: { "k" }, transport: transport), + llm: ReadAloudLLM(apiKeyProvider: { "k" }, transport: gateway), + makeSink: { _ in sink }) + let task = start(speaker) + while !gateway.asked { await Task.yield() } + task.cancel() + gateway.release() + await #expect(throws: CancellationError.self) { try await task.value } + #expect(transport.requests.isEmpty) + #expect(sink.chunks.isEmpty) + } +} + +/// A gateway that ignores cancellation: it waits for `release()`, then answers +/// 200 anyway, the way a response already in flight lands after a stop. +private final class LateAnsweringTransport: GatewayStub { + private let state = Mutex((asked: false, released: false)) + var asked: Bool { state.withLock { $0.asked } } + func release() { state.withLock { $0.released = true } } + + func data(for request: URLRequest) async throws -> (Data, URLResponse) { + state.withLock { $0.asked = true } + while !state.withLock({ $0.released }) { await Task.yield() } + return try ok(request, completion("Say something here.")) + } +} + +/// A gateway that never answers, until the request's task is cancelled. +private final class HangingTransport: GatewayStub { + private let called = Mutex(false) + var asked: Bool { called.withLock { $0 } } + + func data(for request: URLRequest) async throws -> (Data, URLResponse) { + called.withLock { $0 = true } + try await Task.sleep(for: .seconds(60)) + throw URLError(.timedOut) + } +} + +/// A sink that plays for a moment at a fixed loudness, so the meter has +/// something to hear before the read finishes. +private final class LevelSink: PCMSink { + let currentLevel: Float + init(level: Float) { currentLevel = level } + func enqueue(_ pcm: Data) {} + func drain() async { try? await Task.sleep(for: .milliseconds(120)) } + func stop() {} +} + +/// A TTS service that takes the request and never answers it, until the +/// request is cancelled: a read that's still waiting on its first audio. +private final class HangingSpeechTransport: SpeechTransport, Sendable { + private let state = Mutex(false) + + var asked: Bool { state.withLock { $0 } } + + func stream(_ request: URLRequest) async throws -> (HTTPURLResponse, AsyncThrowingStream) { + state.withLock { $0 = true } + while true { + try await Task.sleep(for: .milliseconds(5)) + } + } +} diff --git a/Tests/BlurtEngineTests/SelectionSpeechRouterLifecycleTests.swift b/Tests/BlurtEngineTests/SelectionSpeechRouterLifecycleTests.swift new file mode 100644 index 00000000..b69eee34 --- /dev/null +++ b/Tests/BlurtEngineTests/SelectionSpeechRouterLifecycleTests.swift @@ -0,0 +1,262 @@ +import Foundation +import Synchronization +import Testing + +@testable import BlurtEngine + +/// The router around a read that's stopped, replaced or styled — with a speaker +/// that plays until stopped — and its real hold timer. +@MainActor +@Suite("SelectionSpeechRouter lifecycle") +struct SelectionSpeechRouterLifecycleTests { + private let defaults = freshDefaults() + + /// A router over `speaker`, the selection read returning `selection()` each + /// time, recording every callback it makes. + private func router( + _ speaker: GatedSpeaker, selection: @escaping @Sendable () -> String? = { "Hello there." } + ) -> Rig { + let fixture = makeSession() + let router = SelectionSpeechRouter( + session: fixture.session, speaker: speaker, readSelection: selection, defaults: defaults) + let events = Events() + router.speechStarted = { events.started += 1 } + router.speechEnded = { events.ended += 1 } + router.speechLevel = { events.levels.append($0) } + router.speechFailed = { _ in events.failures += 1 } + router.resetGate = { events.resets += 1 } + return Rig(router: router, events: events, fixture: fixture) + } + + /// A router under test, what it reported, and the session behind it. + private struct Rig { + let router: SelectionSpeechRouter + let events: Events + let fixture: SessionFixture + } + + /// A tap of left ⌥ (reading aloud by default): press, then the gate's latch. + private func tapRead(_ router: SelectionSpeechRouter) { + router.press(key: .leftOption) + router.keyLatched() + } + + /// Starts a tapped read, waits for the speaker to be playing it, then stops it + /// with a second press of the read key. + private func readThenStop(_ router: SelectionSpeechRouter, _ speaker: GatedSpeaker) async throws { + tapRead(router) + try await waitUntil { speaker.calls.count == 1 } + router.press(key: .leftOption) + } + + // MARK: - E1: interrupted reads + + @Test("a stopped read ends once, at once, and its late unwinding changes nothing") + func stopEndsOnce() async throws { + let speaker = GatedSpeaker() + let rig = router(speaker) + let (router, events) = (rig.router, rig.events) + try await readThenStop(router, speaker) + #expect(events.ended == 1) + #expect(!router.isSpeaking) + try await waitUntil { speaker.cancellations == 1 } + try await Task.sleep(for: .milliseconds(50)) + #expect(events.ended == 1, "the cancelled task's end mustn't report again") + #expect(events.failures == 0, "a stop is not a failure") + } + + @Test("a stopped read's late end can't end the read that replaced it") + func lateEndSparesTheNewerRead() async throws { + let speaker = GatedSpeaker() + let rig = router(speaker) + let (router, events) = (rig.router, rig.events) + try await readThenStop(router, speaker) + tapRead(router) + try await waitUntil { speaker.calls.count == 2 && speaker.cancellations == 1 } + try await Task.sleep(for: .milliseconds(50)) + #expect(events.started == 2) + #expect(events.ended == 1) + #expect(router.isSpeaking) + } + + @Test("a level from a read that's over never reaches the pill") + func lateLevelsAreDropped() async throws { + let speaker = GatedSpeaker() + let rig = router(speaker) + let (router, events) = (rig.router, rig.events) + try await readThenStop(router, speaker) + let atEnd = events.levels + try await waitUntil { speaker.cancellations == 1 } + try await Task.sleep(for: .milliseconds(50)) + #expect(atEnd.last == 0, "the end zeroes the meter") + #expect(events.levels == atEnd, "the stopped speaker's late level was dropped") + } + + @Test("the dictation key stopping a read leaves the gate of the dictation it starts alone") + func stopKeepsTheNewDictationsGate() async throws { + let speaker = GatedSpeaker() + let rig = router(speaker) + let (router, events) = (rig.router, rig.events) + tapRead(router) + try await waitUntil { speaker.calls.count == 1 } + let resets = events.resets + router.press(key: .rightCommand) + try await waitUntil { speaker.cancellations == 1 } + try await Task.sleep(for: .milliseconds(50)) + #expect(events.resets == resets, "the stopped read must not reset the dictation's gate") + #expect(events.ended == 1) + } + + // MARK: - E4: styles and samples + + @Test("each press reads with the style set at that moment") + func styleChangesReachTheNextRead() async throws { + let speaker = GatedSpeaker() + let rig = router(speaker) + let router = rig.router + tapRead(router) + try await waitUntil { speaker.calls.count == 1 } + speaker.finishAll() + try await waitUntil { !router.isSpeaking } + defaults.set(2.0, forKey: ReadAloudStyleStore.speedDefaultsKey) + defaults.set("aai-buford", forKey: ReadAloudStyleStore.voiceDefaultsKey) + tapRead(router) + try await waitUntil { speaker.calls.count == 2 } + #expect(speaker.calls[0].style.rate == 1) + #expect(speaker.calls[1].style == ReadAloudStyle(rate: 2, skipsJargon: true, voiceID: "aai-buford")) + } + + @Test("an answer reads with the style set when it's asked") + func styleReachesTheAnswer() async throws { + let speaker = GatedSpeaker() + let rig = router(speaker) + let (router, fixture) = (rig.router, rig.fixture) + TriggerKeyStyleStore(defaults: defaults).setChoice(.ask, for: .leftControl) + defaults.set(1.5, forKey: ReadAloudStyleStore.speedDefaultsKey) + let phases = Task { + for await phase in await fixture.session.phaseStream() { router.sessionPhaseChanged(phase) } + } + defer { phases.cancel() } + router.press(key: .leftControl) + let deadline = ContinuousClock.now + .seconds(5) + while await fixture.mic.startCalls == 0, ContinuousClock.now < deadline { + try await Task.sleep(for: .milliseconds(5)) + } + router.submit(.release) + try await waitUntil { speaker.calls.count == 1 } + #expect(speaker.calls[0].kind == "answer: Hello world.") + #expect(speaker.calls[0].style.rate == 1.5) + } + + @Test("a newer sample cuts off the older one without a failure, and a sample shows no pill") + func samples() async throws { + let speaker = GatedSpeaker() + let rig = router(speaker) + let (router, events) = (rig.router, rig.events) + router.previewSample() + try await waitUntil { speaker.calls.count == 1 } + router.previewSample() + try await waitUntil { speaker.calls.count == 2 && speaker.cancellations == 1 } + try await Task.sleep(for: .milliseconds(50)) + #expect(events.failures == 0) + #expect(events.started == 0 && events.ended == 0) + } + + @Test("a failed sample is reported") + func failedSample() async throws { + let speaker = GatedSpeaker(fails: true) + let rig = router(speaker) + let (router, events) = (rig.router, rig.events) + router.previewSample() + try await waitUntil { events.failures == 1 } + } + + // MARK: - E5: the real hold timer + + @Test("a held read starts once the key has been down for the hold delay") + func heldReadStartsAfterTheDelay() async throws { + let speaker = GatedSpeaker() + let rig = router(speaker) + let (router, events) = (rig.router, rig.events) + let pressed = ContinuousClock.now + router.press(key: .leftOption) + try await waitUntil { events.started == 1 } + #expect(ContinuousClock.now - pressed >= .milliseconds(300)) + } + + @Test("a press's leftover timer can't mark the next press as a hold early") + func leftoverTimerIsCancelled() async throws { + let speaker = GatedSpeaker() + let reads = Counter() + // The first press finds nothing selected; the second finds text. + let rig = router(speaker, selection: { reads.next() == 1 ? nil : "Hello there." }) + let (router, events) = (rig.router, rig.events) + router.press(key: .leftOption) + try await Task.sleep(for: .milliseconds(200)) + let second = ContinuousClock.now + router.press(key: .leftOption) + try await waitUntil { events.started == 1 } + #expect(ContinuousClock.now - second >= .milliseconds(300), "the first press's timer fired for the second") + } +} + +/// What the router reported, in order. +@MainActor +private final class Events { + var started = 0 + var ended = 0 + var levels: [Float] = [] + var failures = 0 + var resets = 0 +} + +/// A speaker that records each call and "plays" until stopped or released. A +/// stopped one reports one more meter level as it unwinds — a tick already on +/// its way — before throwing. +final class GatedSpeaker: ReadAloudSpeaker, Sendable { + struct Call: Sendable { + let kind: String + let style: ReadAloudStyle + } + + private struct State { + var calls: [Call] = [] + var cancellations = 0 + var released = false + } + + private let state = Mutex(State()) + private let fails: Bool + + init(fails: Bool = false) { self.fails = fails } + + var calls: [Call] { state.withLock(\.calls) } + var cancellations: Int { state.withLock(\.cancellations) } + + /// Lets every read playing now, and any later, finish on its own. + func finishAll() { state.withLock { $0.released = true } } + + func speak(_ text: String, style: ReadAloudStyle, onLevel: (@Sendable (Float) -> Void)?) async throws { + try await play(Call(kind: "speak: \(text)", style: style), onLevel: onLevel) + } + + func answer( + _ request: String, about selection: String, style: ReadAloudStyle, onLevel: (@Sendable (Float) -> Void)? + ) async throws { + try await play(Call(kind: "answer: \(request)", style: style), onLevel: onLevel) + } + + private func play(_ call: Call, onLevel: (@Sendable (Float) -> Void)?) async throws { + state.withLock { $0.calls.append(call) } + if fails { throw URLError(.badServerResponse) } + while !state.withLock(\.released) { + do { + try await Task.sleep(for: .milliseconds(5)) + } catch { + state.withLock { $0.cancellations += 1 } + onLevel?(0.7) + throw error + } + } + } +} diff --git a/Tests/BlurtEngineTests/SelectionSpeechRouterTests.swift b/Tests/BlurtEngineTests/SelectionSpeechRouterTests.swift new file mode 100644 index 00000000..162032c9 --- /dev/null +++ b/Tests/BlurtEngineTests/SelectionSpeechRouterTests.swift @@ -0,0 +1,88 @@ +import Foundation +import Testing + +@testable import AssemblyAI +@testable import BlurtEngine + +/// The router that carries out `SelectionSpeechRouting`'s decisions: the +/// selection read, the speech task, and the start/end the pill and cues follow. +@MainActor +@Suite("SelectionSpeechRouter") +struct SelectionSpeechRouterTests { + private let defaults = freshDefaults() + + private func router( + _ transport: FakeSpeechTransport, selection: String?, rate: ValueBox = ValueBox(nil) + ) -> (SelectionSpeechRouter, SessionFixture) { + // Verbatim reads, so no gateway call shapes what's spoken. + defaults.set(false, forKey: ReadAloudStyleStore.skipsJargonDefaultsKey) + let fixture = makeSession() + let speaker = SelectionSpeaker( + synthesizer: AssemblyAISpeechSynthesizer(apiKeyProvider: { "k" }, transport: transport), + llm: ReadAloudLLM(apiKeyProvider: { "k" }, transport: FakeHTTPTransport.failing(with: URLError(.timedOut))), + makeSink: { played in + rate.value = played + return RecordingSink() + }) + let router = SelectionSpeechRouter( + session: fixture.session, speaker: speaker, readSelection: { selection }, defaults: defaults) + return (router, fixture) + } + + @Test("a tap of left ⌥ over a selection reads it, between one start and one end") + func tapReadsSelection() async throws { + let transport = FakeSpeechTransport() + let (router, _) = router(transport, selection: "Hello there.") + var events: [String] = [] + router.speechStarted = { events.append("start") } + router.speechEnded = { events.append("end") } + router.press(key: .leftOption) + router.keyLatched() + try await waitUntil { events.count == 2 } + #expect(events == ["start", "end"]) + #expect(transport.spoken == ["Hello there."]) + } + + @Test("with nothing selected, left ⌥ reads nothing and clears the gate") + func nothingSelected() async throws { + let transport = FakeSpeechTransport() + let (router, _) = router(transport, selection: nil) + var resets = 0 + var started = false + router.resetGate = { resets += 1 } + router.speechStarted = { started = true } + router.press(key: .leftOption) + try await waitUntil { resets == 1 } + #expect(!started) + #expect(transport.requests.isEmpty) + } + + @Test("a failed read reports the failure and still ends") + func failureReported() async throws { + let transport = FakeSpeechTransport { _ in (500, [Data(#"{"message":"boom"}"#.utf8)]) } + let (router, _) = router(transport, selection: "Hello there.") + var failures: [String] = [] + var ended = false + router.speechFailed = { failures.append(($0 as? SpeechSynthesisError)?.errorDescription ?? "") } + router.speechEnded = { ended = true } + router.press(key: .leftOption) + router.keyLatched() + try await waitUntil { ended } + #expect(failures == ["Text-to-speech failed: boom"]) + } + + @Test("a sample plays in the picked voice at the picked speed") + func sampleUsesPickedStyle() async throws { + let transport = FakeSpeechTransport() + let rate = ValueBox(nil) + let (router, _) = router(transport, selection: nil, rate: rate) + defaults.set(2.0, forKey: ReadAloudStyleStore.speedDefaultsKey) + defaults.set("aai-buford", forKey: ReadAloudStyleStore.voiceDefaultsKey) + router.previewSample() + try await waitUntil { !transport.requests.isEmpty } + #expect(rate.value == 2) + let body = transport.requests.first?.httpBody.flatMap { try? JSONSerialization.jsonObject(with: $0) } + #expect((body as? [String: Any])?["voice_id"] as? String == "aai-buford") + #expect(transport.spoken.first?.hasPrefix("Hi, I'm Buford.") == true) + } +} diff --git a/Tests/BlurtEngineTests/SelectionSpeechRoutingTests.swift b/Tests/BlurtEngineTests/SelectionSpeechRoutingTests.swift index d95f5c9c..d9fe273c 100644 --- a/Tests/BlurtEngineTests/SelectionSpeechRoutingTests.swift +++ b/Tests/BlurtEngineTests/SelectionSpeechRoutingTests.swift @@ -5,304 +5,313 @@ import Testing @Suite("SelectionSpeechRouting") struct SelectionSpeechRoutingTests { - @Test("with the switch off, every command goes straight to the session") - func switchedOff() { - var routing = SelectionSpeechRouting() - for command in [DictationSession.Command.press(), .press(key: .rightOption), .release, .cancel, .cancelRecording] { - #expect(routing.submit(command, readAloudEnabled: false) == [.forward(command)]) - } + /// A read-aloud press: left ⌥, tap-only unless `holds`. + private func read(_ routing: inout SelectionSpeechRouting, holds: Bool = false) -> [SelectionSpeechRouting.Action] { + routing.press(key: .leftOption, action: .readAloud, holds: holds) } - @Test("a press with nothing selected dictates, replaying what arrived during the read in order") - func noSelectionReplays() { + // MARK: - Dictation + + @Test("a dictating key's commands go straight to the session, with no selection read") + func dictationPassesThrough() { var routing = SelectionSpeechRouting() - #expect(routing.submit(.press(), readAloudEnabled: true) == [.readSelection]) - #expect(routing.submit(.release, readAloudEnabled: true).isEmpty) - #expect(routing.selectionResolved(nil) == [.forward(.press()), .forward(.release)]) - #expect(routing.submit(.press(), readAloudEnabled: true) == [.readSelection]) + #expect(routing.press(key: .rightOption, action: .dictate, holds: true) == [.forward(.press(key: .rightOption))]) + for command in [DictationSession.Command.release, .cancel, .cancelRecording] { + #expect(routing.submit(command) == [.forward(command)]) + } } - @Test("a press with nothing selected dictates with its own trigger key") - func noSelectionKeepsTheKey() { + @Test("a key turned off mid-press dictates, as it was bound to") + func offDictates() { var routing = SelectionSpeechRouting() - _ = routing.submit(.press(key: .rightOption), readAloudEnabled: true) - #expect(routing.selectionResolved(nil) == [.forward(.press(key: .rightOption))]) + #expect(routing.press(key: .function, action: .off, holds: false) == [.forward(.press(key: .function))]) } - @Test("a press over a selection speaks it, and the next key command stops it") - func speaksThenStops() { + @Test("a dictation's terminal phase syncs the gate only while commands go to the session") + func terminalPhaseSync() { var routing = SelectionSpeechRouting() - _ = routing.submit(.press(), readAloudEnabled: true) - #expect(routing.selectionResolved("Hello there.") == [.speak("Hello there.")]) - #expect(routing.submit(.release, readAloudEnabled: true) == [.stopSpeech]) - // The key stop already left the gate idle, so a late finish is nothing. - #expect(routing.speechFinished().isEmpty) + #expect(routing.sessionPhaseChanged(.idle) == [.syncGateAfterSession]) + _ = read(&routing) + #expect(routing.sessionPhaseChanged(.idle).isEmpty) + _ = routing.keyLatched() + _ = routing.selectionResolved("Hi.") + #expect(routing.sessionPhaseChanged(.idle).isEmpty) } - @Test("a combo cancel during the read drops the press — ⌘C over a selection is a copy") - func comboDuringRead() { + // MARK: - Read aloud + + @Test("a tap reads the selection once the tap lands, freeing the gate; the next press stops it") + func tapReads() { var routing = SelectionSpeechRouting() - _ = routing.submit(.press(), readAloudEnabled: true) - _ = routing.submit(.cancel, readAloudEnabled: true) + #expect(read(&routing) == [.readSelection]) #expect(routing.selectionResolved("Hello there.").isEmpty) - #expect(routing.submit(.press(), readAloudEnabled: false) == [.forward(.press())]) + #expect(routing.keyLatched() == [.speak("Hello there."), .resetGate]) + // The read key's next press stops it and clears the gate that press armed. + #expect(read(&routing) == [.stopSpeech, .resetGate]) + // That stop already moved on, so a late finish is nothing. + #expect(routing.speechFinished().isEmpty) } - @Test("a recovery reset while speaking stops the read and still reaches the session") - func recoveryWhileSpeaking() { + @Test("pressing the dictation key during a tapped read stops it and dictates") + func dictationKeyInterruptsRead() { var routing = SelectionSpeechRouting() - _ = routing.submit(.press(), readAloudEnabled: true) + _ = read(&routing) + _ = routing.keyLatched() _ = routing.selectionResolved("Hello there.") #expect( - routing.submit(.cancelRecording, readAloudEnabled: true) - == [.stopSpeech, .forward(.cancelRecording)]) + routing.press(key: .rightCommand, action: .dictate, holds: true) + == [.stopSpeech, .forward(.press(key: .rightCommand))]) + #expect(routing.submit(.release) == [.forward(.release)]) } - @Test("a read that ends on its own clears the gate silently") - func naturalEnd() { + @Test("a tap that lands before the selection read returns still reads it") + func tapBeforeRead() { var routing = SelectionSpeechRouting() - _ = routing.submit(.press(), readAloudEnabled: true) - _ = routing.selectionResolved("Hello there.") - #expect(routing.speechFinished() == [.resetGate]) - #expect(routing.submit(.press(), readAloudEnabled: false) == [.forward(.press())]) + _ = read(&routing) + #expect(routing.keyLatched().isEmpty) + #expect(routing.selectionResolved("Hello there.") == [.speak("Hello there."), .resetGate]) } - @Test("a dictation's terminal phase syncs the gate only while commands go to the session") - func terminalPhaseOwnership() { + @Test("⌘C over a selection reads nothing, before or after the selection read returns") + func comboReadsNothing() { var routing = SelectionSpeechRouting() - #expect(routing.sessionPhaseChanged(.pasted) == [.syncGateAfterSession]) - _ = routing.submit(.press(), readAloudEnabled: true) - #expect(routing.sessionPhaseChanged(.pasted).isEmpty) - _ = routing.selectionResolved("Hello there.") - // The earlier dictation finishing mid-read must not reset the read's gate. - #expect(routing.sessionPhaseChanged(.pasted).isEmpty) + _ = read(&routing) + #expect(routing.submit(.cancel).isEmpty) + #expect(routing.selectionResolved("Hello there.").isEmpty) + + _ = read(&routing) + #expect(routing.selectionResolved("Hello there.").isEmpty) + #expect(routing.submit(.cancel).isEmpty) + #expect(routing.keyLatched().isEmpty) + #expect(routing.press(key: .rightCommand, action: .dictate, holds: true) == [.forward(.press(key: .rightCommand))]) } - @Test("a press that lands during a slow read and is dropped clears the gate it latched") - func pressDroppedDuringSlowRead() { + @Test("a holdable read key reads once held past the delay, and stops when let go") + func holdReads() { var routing = SelectionSpeechRouting() - _ = routing.submit(.press(), readAloudEnabled: true) - // ⌘C over the selection, then a tap to read, all before an Electron app's - // read returns. The tap latched the gate. - _ = routing.submit(.cancel, readAloudEnabled: true) - _ = routing.submit(.press(), readAloudEnabled: true) - _ = routing.keyLatched() - // Dropped like any press that was over before the read returned, but the - // latch it left would swallow the next tap, so clear it. - #expect(routing.selectionResolved("Hello there.") == [.resetGate]) - // A held-then-released press leaves the gate idle, so there is nothing to clear. - _ = routing.submit(.press(), readAloudEnabled: true) - _ = routing.submit(.release, readAloudEnabled: true) - #expect(routing.selectionResolved("Hello there.").isEmpty) + #expect(read(&routing, holds: true) == [.readSelection, .startHoldTimer]) + _ = routing.selectionResolved("Hello there.") + #expect(routing.holdElapsed() == [.speak("Hello there.")]) + #expect(routing.submit(.release) == [.stopSpeech]) } - @Test("a stale selection result after the press resolved is ignored") - func staleResolution() { + @Test("a hold that outlasts a slow selection read reads as soon as the read returns") + func holdBeforeRead() { var routing = SelectionSpeechRouting() - #expect(routing.selectionResolved("Hello there.").isEmpty) + _ = read(&routing, holds: true) + #expect(routing.holdElapsed().isEmpty) + #expect(routing.selectionResolved("Hello there.") == [.speak("Hello there.")]) } -} -/// Asking about the selection: a hold over it records a spoken request, and the -/// answer is read aloud. A tap still reads the selection. -@Suite("SelectionSpeechRouting asking") -struct SelectionSpeechRoutingAskTests { - private let selection = "The quarterly report." - - /// A routing that has pressed over `selection` with asking on, and resolved it - /// while the key is still down. - private func selected(_ start: SelectionSpeechRouting = SelectionSpeechRouting()) -> SelectionSpeechRouting { - var routing = start - _ = routing.submit(.press(), readAloudEnabled: true, askEnabled: true) - _ = routing.selectionResolved(selection) - return routing + @Test("let go of before the hold delay in hold-only mode, a read key does nothing") + func quickReleaseUnderHold() { + var routing = SelectionSpeechRouting() + _ = read(&routing, holds: true) + _ = routing.selectionResolved("Hello there.") + #expect(routing.submit(.release).isEmpty) + #expect(routing.holdElapsed().isEmpty) } - /// A routing recording the spoken request. - private func asking() -> SelectionSpeechRouting { - var routing = selected() + @Test("a hold released before the gate's threshold keeps the read playing, and frees the gate") + func shortHoldLatches() { + var routing = SelectionSpeechRouting() + _ = read(&routing, holds: true) + _ = routing.selectionResolved("Hello there.") _ = routing.holdElapsed() - return routing + #expect(routing.keyLatched() == [.resetGate]) + #expect(read(&routing, holds: true) == [.stopSpeech, .resetGate]) } - @Test("a press that may ask starts the hold timer beside the selection read") - func startsTimer() { + @Test("with nothing selected a read key does nothing, and clears the gate it holds") + func nothingSelected() { var routing = SelectionSpeechRouting() - #expect(routing.submit(.press(), readAloudEnabled: true, askEnabled: true) == [.readSelection, .startHoldTimer]) + _ = read(&routing) + #expect(routing.selectionResolved(nil) == [.resetGate]) + // Already let go: the gate is idle, so there's nothing to clear. + _ = read(&routing) + _ = routing.submit(.release) + #expect(routing.selectionResolved(nil).isEmpty) } - @Test("a tap over a selection reads it, without ever opening the mic") - func tapReads() { - var routing = selected() - #expect(routing.keyLatched() == [.speak(selection)]) - // The read's key stop still works as before. - #expect(routing.submit(.release, readAloudEnabled: true, askEnabled: true) == [.stopSpeech]) + @Test("a tapped twice before the selection read returns reads nothing") + func doubleTapBeforeRead() { + var routing = SelectionSpeechRouting() + _ = read(&routing) + _ = routing.keyLatched() + _ = routing.submit(.release) + #expect(routing.selectionResolved("Hello there.").isEmpty) } - @Test("a tap that lands before the selection read returns still reads it") - func tapDuringRead() { + @Test("a read that ends on its own clears the gate silently") + func readEndsOnItsOwn() { var routing = SelectionSpeechRouting() - _ = routing.submit(.press(), readAloudEnabled: true, askEnabled: true) - #expect(routing.keyLatched().isEmpty) - #expect(routing.selectionResolved(selection) == [.speak(selection)]) + _ = read(&routing) + _ = routing.keyLatched() + _ = routing.selectionResolved("Hello there.") + #expect(routing.speechFinished() == [.resetGate]) + #expect(routing.submit(.release) == [.forward(.release)]) } - @Test("a hold over a selection opens the mic with the hand-back press") - func holdAsks() { - var routing = selected() - #expect(routing.holdElapsed() == [.forward(.pressHandingBack())]) + @Test("a recovery reset while speaking stops the read and still reaches the session") + func recoveryWhileSpeaking() { + var routing = SelectionSpeechRouting() + _ = read(&routing) + _ = routing.keyLatched() + _ = routing.selectionResolved("Hello there.") + #expect(routing.submit(.cancelRecording) == [.stopSpeech, .forward(.cancelRecording)]) } - @Test("the request about a selection is made with its trigger key's style") - func askKeepsTheKey() { + @Test("a stale selection result after the press resolved is ignored") + func staleSelection() { var routing = SelectionSpeechRouting() - _ = routing.submit(.press(key: .rightOption), readAloudEnabled: true, askEnabled: true) - #expect(routing.holdElapsed().isEmpty) - #expect(routing.selectionResolved(selection) == [.forward(.pressHandingBack(key: .rightOption))]) + _ = read(&routing) + _ = routing.selectionResolved(nil) + #expect(routing.selectionResolved("late").isEmpty) } +} - @Test("a hold that outlasts a slow selection read asks as soon as the read returns") - func holdDuringRead() { - var routing = SelectionSpeechRouting() - _ = routing.submit(.press(), readAloudEnabled: true, askEnabled: true) - #expect(routing.holdElapsed().isEmpty) - #expect(routing.selectionResolved(selection) == [.forward(.pressHandingBack())]) +@Suite("SelectionSpeechRouting · Ask") +struct SelectionSpeechRoutingAskTests { + /// An ask press: left ⌃, holdable unless told otherwise. + private func ask(_ routing: inout SelectionSpeechRouting, holds: Bool = true) -> [SelectionSpeechRouting.Action] { + routing.press(key: .leftControl, action: .ask, holds: holds) } - @Test("letting go after the gate's hold threshold releases the request") - func longHoldReleases() { - var routing = asking() - #expect(routing.submit(.release, readAloudEnabled: true, askEnabled: true) == [.forward(.release)]) + @Test("an ask key held past the delay opens the mic with the hand-back press") + func askOpensMic() { + var routing = SelectionSpeechRouting() + #expect(ask(&routing) == [.readSelection, .startHoldTimer]) + _ = routing.selectionResolved("Revenue rose.") + #expect(routing.holdElapsed() == [.forward(.pressHandingBack(key: nil))]) + #expect(routing.submit(.release) == [.forward(.release)]) } - @Test("letting go before the gate's hold threshold also ends the request, and clears the latch") - func shortHoldReleases() { - var routing = asking() - #expect(routing.keyLatched() == [.forward(.release), .resetGate]) + @Test("a tapped ask key records until the next tap, like a dictation") + func tappedAskLatches() { + var routing = SelectionSpeechRouting() + _ = ask(&routing, holds: false) + _ = routing.selectionResolved("Revenue rose.") + #expect(routing.keyLatched() == [.forward(.pressHandingBack(key: nil))]) + #expect(routing.submit(.release) == [.forward(.release)]) } @Test("the transcript handed back is answered aloud, about the selection") func answers() { - var routing = asking() - _ = routing.submit(.release, readAloudEnabled: true, askEnabled: true) + var routing = SelectionSpeechRouting() + _ = ask(&routing) + _ = routing.selectionResolved("Revenue rose.") + _ = routing.holdElapsed() + _ = routing.submit(.release) #expect( routing.sessionPhaseChanged(.handedBack("Summarize this.")) - == [.answer(request: "Summarize this.", selection: selection)]) - // The answer is speech like any read: a key command stops it, and a natural - // end clears the gate. + == [.answer(request: "Summarize this.", selection: "Revenue rose.")]) #expect(routing.speechFinished() == [.resetGate]) } - @Test("a tap that stops an answer clears the gate it armed, so the next press isn't swallowed") - func stopAnswerWithTap() { - var routing = asking() - _ = routing.submit(.release, readAloudEnabled: true, askEnabled: true) - _ = routing.sessionPhaseChanged(.handedBack("Summarize this.")) - // The gate was idle under the answer, so the stopping tap arrives as a press. - #expect(routing.submit(.press(), readAloudEnabled: true, askEnabled: true) == [.stopSpeech, .resetGate]) - // Its key-up latch then finds nothing to do, and the next press is a press. - #expect(routing.keyLatched().isEmpty) - #expect(routing.submit(.press(), readAloudEnabled: true, askEnabled: true) == [.readSelection, .startHoldTimer]) - } - - @Test("a request with nothing said reads the selection instead") - func nothingSaid() { - var routing = asking() - _ = routing.submit(.release, readAloudEnabled: true, askEnabled: true) - #expect(routing.sessionPhaseChanged(.idle) == [.speak(selection)]) + @Test("a press stops an answer: the dictation key then dictates, any other key's press is cleared") + func pressStopsAnswer() { + var routing = SelectionSpeechRouting() + _ = ask(&routing) + _ = routing.selectionResolved("Revenue rose.") + _ = routing.holdElapsed() + _ = routing.submit(.release) + _ = routing.sessionPhaseChanged(.handedBack("Explain.")) + #expect( + routing.press(key: .rightCommand, action: .dictate, holds: true) + == [.stopSpeech, .forward(.press(key: .rightCommand))]) + #expect(ask(&routing) == [.readSelection, .startHoldTimer]) } - @Test("a request that fails is a dictation ending without a key event") - func failure() { - var routing = asking() - _ = routing.submit(.release, readAloudEnabled: true, askEnabled: true) - let failed = PipelinePhase.failed(.sttFailed(underlying: URLError(.timedOut))) - #expect(routing.sessionPhaseChanged(failed) == [.syncGateAfterSession]) - // Back to passthrough: the next press dictates or reads as usual. - #expect(routing.submit(.press(), readAloudEnabled: false) == [.forward(.press())]) + @Test("a held ask let go before the gate's threshold ends the request, rather than latching") + func shortHeldAskEnds() { + var routing = SelectionSpeechRouting() + _ = ask(&routing) + _ = routing.selectionResolved("Revenue rose.") + _ = routing.holdElapsed() + #expect(routing.keyLatched() == [.forward(.release), .resetGate]) + #expect( + routing.sessionPhaseChanged(.handedBack("Summarize.")) + == [.answer(request: "Summarize.", selection: "Revenue rose.")]) } - @Test("a recording that fails while the key is still down syncs the gate") - func failureWhileHeld() { - var routing = asking() - let failed = PipelinePhase.failed(.audioCaptureFailed(underlying: URLError(.unknown))) - #expect(routing.sessionPhaseChanged(failed) == [.syncGateAfterSession]) + @Test("a held ask let go before the selection read returns asks nothing") + func shortHeldAskBeforeRead() { + var routing = SelectionSpeechRouting() + _ = ask(&routing) + _ = routing.holdElapsed() + _ = routing.keyLatched() + #expect(routing.selectionResolved("Revenue rose.").isEmpty) + } + + @Test("a request with nothing said, or a failed one, is a dictation ending without a key event") + func requestEndsQuietly() { + for phase in [PipelinePhase.idle, .failed(.apiKeyMissing)] { + var routing = SelectionSpeechRouting() + _ = ask(&routing) + _ = routing.selectionResolved("Revenue rose.") + _ = routing.holdElapsed() + _ = routing.submit(.release) + #expect(routing.sessionPhaseChanged(phase) == [.syncGateAfterSession]) + } } @Test("the recording cap handing back with the key still down answers and clears the gate") - func handedBackWhileHeld() { - var routing = asking() + func capHandsBack() { + var routing = SelectionSpeechRouting() + _ = ask(&routing) + _ = routing.selectionResolved("Revenue rose.") + _ = routing.holdElapsed() #expect( - routing.sessionPhaseChanged(.handedBack("What is this?")) - == [.answer(request: "What is this?", selection: selection), .resetGate]) + routing.sessionPhaseChanged(.handedBack("Explain.")) + == [.answer(request: "Explain.", selection: "Revenue rose."), .resetGate]) } @Test("a combo while asking cancels the recording, as it would a dictation") func comboWhileAsking() { - var routing = asking() - #expect(routing.submit(.cancel, readAloudEnabled: true, askEnabled: true) == [.forward(.cancel)]) + var routing = SelectionSpeechRouting() + _ = ask(&routing) + _ = routing.selectionResolved("Revenue rose.") + _ = routing.holdElapsed() + #expect(routing.submit(.cancel) == [.forward(.cancel)]) } @Test("pressing again while the request is transcribing drops it and clears the new press") func pressWhileTranscribing() { - var routing = asking() - _ = routing.submit(.release, readAloudEnabled: true, askEnabled: true) - #expect(routing.submit(.press(), readAloudEnabled: true, askEnabled: true) == [.forward(.cancel), .resetGate]) - // A transcript arriving after that is someone else's now, and ignored. - #expect(routing.sessionPhaseChanged(.cancelled) == [.syncGateAfterSession]) - } - - @Test("a hold while an earlier dictation finishes waits for it, then asks") - func holdWaitsForEarlierDictation() { var routing = SelectionSpeechRouting() - // The earlier dictation was let go of and is still being transcribed. - _ = routing.sessionPhaseChanged(.transcribing) - var held = selected(routing) - // The session would refuse a press now, so none is sent yet... - #expect(held.holdElapsed().isEmpty) - // ...and the earlier dictation's end is not the request's: the mic opens now. - #expect(held.sessionPhaseChanged(.pasted) == [.forward(.pressHandingBack())]) - #expect(held.sessionPhaseChanged(.recording).isEmpty) - #expect(held.submit(.release, readAloudEnabled: true, askEnabled: true) == [.forward(.release)]) - #expect( - held.sessionPhaseChanged(.handedBack("Summarize this.")) - == [.answer(request: "Summarize this.", selection: selection)]) + _ = ask(&routing) + _ = routing.selectionResolved("Revenue rose.") + _ = routing.holdElapsed() + _ = routing.submit(.release) + #expect(ask(&routing) == [.forward(.cancel), .resetGate]) } - @Test("a request let go of before the earlier dictation finished reads the selection, cancelling nothing") - func releaseBeforeEarlierDictationEnds() { + @Test("an ask while an earlier dictation finishes waits for it, then asks") + func askWaitsForEarlierDictation() { var routing = SelectionSpeechRouting() _ = routing.sessionPhaseChanged(.transcribing) - var held = selected(routing) - _ = held.holdElapsed() - // Nothing goes to the session: it is still busy with the earlier dictation. - #expect(held.submit(.release, readAloudEnabled: true, askEnabled: true).isEmpty) - #expect(held.sessionPhaseChanged(.pasted) == [.speak(selection)]) - } - - @Test("a quick release before the hold delay, under a hold-only trigger, does nothing") - func quickHoldOnlyRelease() { - var routing = selected() - #expect(routing.submit(.release, readAloudEnabled: true, askEnabled: true).isEmpty) + _ = ask(&routing) + _ = routing.selectionResolved("Revenue rose.") #expect(routing.holdElapsed().isEmpty) + #expect(routing.sessionPhaseChanged(.idle) == [.forward(.pressHandingBack(key: nil))]) } - @Test("a hold timer is ignored outside a press that may ask") - func staleTimer() { + @Test("an ask let go of before the earlier dictation finished asks nothing and cancels nothing") + func askReleasedBeforeStart() { var routing = SelectionSpeechRouting() - #expect(routing.holdElapsed().isEmpty) - _ = routing.submit(.press(), readAloudEnabled: true, askEnabled: false) - #expect(routing.holdElapsed().isEmpty) - // Asking off reads at once, as before asking existed. - #expect(routing.selectionResolved(selection) == [.speak(selection)]) + _ = routing.sessionPhaseChanged(.transcribing) + _ = ask(&routing) + _ = routing.selectionResolved("Revenue rose.") + _ = routing.holdElapsed() + #expect(routing.submit(.release).isEmpty) + #expect(routing.sessionPhaseChanged(.idle).isEmpty) + #expect(routing.sessionPhaseChanged(.idle) == [.syncGateAfterSession]) } - @Test("with nothing selected, a press that may ask dictates as usual") - func noSelection() { + @Test("with nothing selected an ask key does nothing") + func askNothingSelected() { var routing = SelectionSpeechRouting() - _ = routing.submit(.press(), readAloudEnabled: true, askEnabled: true) - _ = routing.keyLatched() - #expect(routing.selectionResolved(nil) == [.forward(.press())]) + _ = ask(&routing) + #expect(routing.selectionResolved(nil) == [.resetGate]) #expect(routing.holdElapsed().isEmpty) } } diff --git a/Tests/BlurtEngineTests/SelectionSpeechStoreTests.swift b/Tests/BlurtEngineTests/SelectionSpeechStoreTests.swift deleted file mode 100644 index 8f2e3674..00000000 --- a/Tests/BlurtEngineTests/SelectionSpeechStoreTests.swift +++ /dev/null @@ -1,22 +0,0 @@ -import Foundation -import Testing - -@testable import BlurtEngine - -@Suite("SelectionSpeechStore") -struct SelectionSpeechStoreTests { - @Test("defaults to off when unset — the feature is experimental opt-in") - func defaultsToOff() { - #expect(!SelectionSpeechStore(defaults: freshDefaults()).isEnabled) - } - - @Test("reads back the switch the Settings toggle writes") - func readsBackTheToggledSlot() { - let defaults = freshDefaults() - let store = SelectionSpeechStore(defaults: defaults) - defaults.set(true, forKey: SelectionSpeechStore.defaultsKey) - #expect(store.isEnabled) - defaults.set(false, forKey: SelectionSpeechStore.defaultsKey) - #expect(!store.isEnabled) - } -} diff --git a/Tests/BlurtEngineTests/SignInSessionTests.swift b/Tests/BlurtEngineTests/SignInSessionTests.swift index 293816d5..7c4422eb 100644 --- a/Tests/BlurtEngineTests/SignInSessionTests.swift +++ b/Tests/BlurtEngineTests/SignInSessionTests.swift @@ -187,10 +187,16 @@ struct OAuthConfigurationInfoTests { @Suite("Credential rows") struct CredentialRowsTests { - @Test("signed out with sign-in configured: account row first and filled, key row below") - func signedOutWithSignIn() { + @Test("a new install with sign-in configured: only the account row, filled — no pasting a key") + func newInstallSignsIn() { let rows = CredentialRows.resolve(isKeyStored: false, isSignInAvailable: true, status: .signedOut) - #expect(rows == CredentialRows(showsAccountRow: true, showsKeyRow: true, primary: .signIn)) + #expect(rows == CredentialRows(showsAccountRow: true, showsKeyRow: false, primary: .signIn)) + } + + @Test("a legacy install with a pasted key keeps its key row, beside the account row") + func legacyKeyKeepsItsRow() { + let rows = CredentialRows.resolve(isKeyStored: true, isSignInAvailable: true, status: .signedOut) + #expect(rows == CredentialRows(showsAccountRow: true, showsKeyRow: true, primary: nil)) } @Test("no sign-in configured: just the key row, Connect filled until a key is stored") diff --git a/Tests/BlurtEngineTests/SoundPackStoreTests.swift b/Tests/BlurtEngineTests/SoundPackStoreTests.swift index 03e79b9b..28b88caa 100644 --- a/Tests/BlurtEngineTests/SoundPackStoreTests.swift +++ b/Tests/BlurtEngineTests/SoundPackStoreTests.swift @@ -27,6 +27,18 @@ struct SoundPackStoreTests { #expect(SoundPackStore(catalog: Self.catalog, defaults: defaults).soundPack == Self.clav) } + @Test("read-aloud's cues are their own setting, with their own default") + func readAloudIsSeparate() { + let defaults = freshDefaults() + let readAloud = SoundPackStore( + catalog: Self.catalog.defaulting(to: "rom1a-19"), cue: .readAloud, defaults: defaults) + #expect(readAloud.soundPack == Self.clav) + readAloud.soundPack = .none + #expect(SoundPackStore(catalog: Self.catalog, defaults: defaults).soundPack == Self.brass) + #expect(readAloud.soundPack == .none) + #expect(Self.catalog.defaulting(to: "missing").fromPersisted(nil) == .none) + } + @Test("none is a storable, distinct value") func storesNone() { let defaults = freshDefaults() diff --git a/Tests/BlurtEngineTests/SpeechTransportTests.swift b/Tests/BlurtEngineTests/SpeechTransportTests.swift new file mode 100644 index 00000000..87b2f54d --- /dev/null +++ b/Tests/BlurtEngineTests/SpeechTransportTests.swift @@ -0,0 +1,180 @@ +import Foundation +import Synchronization +import Testing + +@testable import AssemblyAI +@testable import BlurtEngine + +/// The TTS client's failures through the fake transport, and the production +/// `URLSession` streaming path through a `URLProtocol` on a private session — +/// the one place a real cancellation's `URLError(.cancelled)` can show up. +@Suite("TTS transport and errors", .serialized, .timeLimit(.minutes(1))) +struct SpeechTransportTests { + private func synthesizer(_ transport: any SpeechTransport) -> AssemblyAISpeechSynthesizer { + AssemblyAISpeechSynthesizer(apiKeyProvider: { "k" }, transport: transport) + } + + private func collect(_ synth: AssemblyAISpeechSynthesizer, _ audio: ValueBox) async throws { + for try await chunk in synth.speech(for: "Read this.") { audio.value.append(chunk) } + } + + // MARK: - Error paths + + @Test("a request that times out, before or during the body, reads as 'stopped responding'") + func timeouts() async { + await #expect(throws: SpeechSynthesisError.timedOut) { + try await collect(synthesizer(FakeSpeechTransport(failing: URLError(.timedOut))), ValueBox(Data())) + } + let audio = ValueBox(Data()) + let stalled = FakeSpeechTransport(end: .fail(URLError(.timedOut))) { _ in (200, [Data([1, 0])]) } + await #expect(throws: SpeechSynthesisError.timedOut) { try await collect(synthesizer(stalled), audio) } + #expect(audio.value == Data([1, 0]), "the audio before the stall still plays") + } + + @Test("a connection dropped mid-body fails with its own error, after the audio that arrived") + func droppedMidBody() async { + let audio = ValueBox(Data()) + let dropped = FakeSpeechTransport(end: .fail(URLError(.networkConnectionLost))) { _ in (200, [Data([1, 0])]) } + await #expect(throws: URLError.self) { try await collect(synthesizer(dropped), audio) } + #expect(audio.value == Data([1, 0])) + } + + @Test("an error body is read only so far: a huge or stalled one still fails promptly") + func errorBodyCap() async { + let page = Data(repeating: 0x3C, count: 1_024) + let endless = FakeSpeechTransport(end: .hang) { _ in (502, Array(repeating: page, count: 70)) } + await #expect(throws: SpeechSynthesisError.server("HTTP 502")) { + try await collect(synthesizer(endless), ValueBox(Data())) + } + } + + @Test("an error body with a message but no code reports the message") + func messageWithoutCode() { + let body = Data(#"{"message":"Slow down"}"#.utf8) + #expect(AssemblyAISpeechSynthesizer.failure(status: 429, body: body) == "Slow down") + } + + // MARK: - The URLSession path + + private func session() -> URLSession { + let configuration = URLSessionConfiguration.ephemeral + configuration.protocolClasses = [ScriptedSpeechProtocol.self] + return URLSession(configuration: configuration) + } + + @Test("URLSession streams the body through to the audio") + func urlSessionStreams() async throws { + ScriptedSpeechProtocol.play(.respond(status: 200, chunks: [Data([1, 0, 2, 0]), Data([3, 0])], finishes: true)) + let audio = ValueBox(Data()) + try await collect(synthesizer(session()), audio) + #expect(audio.value == Data([1, 0, 2, 0, 3, 0])) + } + + @Test("URLSession: a refused request reports the body's reason") + func urlSessionRefused() async { + let body = Data(#"{"code":"unauthorized","message":"Invalid API key"}"#.utf8) + ScriptedSpeechProtocol.play(.respond(status: 401, chunks: [body], finishes: true)) + await #expect(throws: SpeechSynthesisError.server("(unauthorized) Invalid API key")) { + try await collect(synthesizer(session()), ValueBox(Data())) + } + } + + @Test("URLSession: a failure before the response throws rather than hanging") + func urlSessionFailsEarly() async { + ScriptedSpeechProtocol.play(.fail(URLError(.notConnectedToInternet))) + await #expect(throws: URLError.self) { try await collect(synthesizer(session()), ValueBox(Data())) } + } + + @Test("URLSession: a connection dropped mid-body fails after the audio that arrived") + func urlSessionDropsMidBody() async { + ScriptedSpeechProtocol.play(.respondThenFail(chunk: Data([1, 0]), URLError(.networkConnectionLost))) + let audio = ValueBox(Data()) + await #expect(throws: URLError.self) { try await collect(synthesizer(session()), audio) } + #expect(audio.value == Data([1, 0])) + } + + @Test("URLSession: stopping a read cancels its request and reads as a cancellation, not a failure") + func urlSessionCancellation() async throws { + ScriptedSpeechProtocol.play(.respond(status: 200, chunks: [Data([1, 0])], finishes: false)) + let speaker = SelectionSpeaker( + synthesizer: synthesizer(session()), + llm: ReadAloudLLM(apiKeyProvider: { "k" }, transport: FakeHTTPTransport.failing(with: URLError(.timedOut))), + makeSink: { _ in RecordingSink() }) + let reading = Task { try await speaker.speak("Read this.", style: .standard, onLevel: nil) } + while !ScriptedSpeechProtocol.started { await Task.yield() } + try await Task.sleep(for: .milliseconds(50)) + reading.cancel() + await #expect(throws: CancellationError.self) { try await reading.value } + let deadline = ContinuousClock.now + .seconds(5) + while !ScriptedSpeechProtocol.stopped, ContinuousClock.now < deadline { await Task.yield() } + #expect(ScriptedSpeechProtocol.stopped, "the request should be cancelled, not left synthesizing") + } +} + +/// A `URLProtocol` playing one scripted response, registered only on the +/// private sessions `SpeechTransportTests` builds (never globally). +final class ScriptedSpeechProtocol: URLProtocol, @unchecked Sendable { + enum Script: Sendable { + case respond(status: Int, chunks: [Data], finishes: Bool) + case respondThenFail(chunk: Data, URLError) + case fail(URLError) + } + + private struct State { + var script = Script.fail(URLError(.unknown)) + var started = false + var stopped = false + } + + private static let state = Mutex(State()) + + static func play(_ script: Script) { + state.withLock { $0 = State(script: script) } + } + + static var started: Bool { state.withLock(\.started) } + static var stopped: Bool { state.withLock(\.stopped) } + + override static func canInit(with request: URLRequest) -> Bool { true } + override static func canonicalRequest(for request: URLRequest) -> URLRequest { request } + + override func startLoading() { + let script = Self.state.withLock { state in + state.started = true + return state.script + } + let url = request.url ?? AssemblyAISpeechSynthesizer.endpoint + switch script { + case .respond(let status, let chunks, let finishes): + respond(status: status, url: url) + for chunk in chunks { client?.urlProtocol(self, didLoad: chunk) } + if finishes { client?.urlProtocolDidFinishLoading(self) } + case .respondThenFail(let chunk, let error): + respond(status: 200, url: url) + client?.urlProtocol(self, didLoad: chunk) + // A moment later, as a real connection drops after audio has flowed: + // failing in the same instant, URLSession reports only the failure. + DispatchQueue.global().asyncAfter(deadline: .now() + 0.2) { [self] in + client?.urlProtocol(self, didFailWithError: error) + } + case .fail(let error): + client?.urlProtocol(self, didFailWithError: error) + } + } + + override func stopLoading() { + Self.state.withLock { $0.stopped = true } + } + + /// With the content type the service sends: without one, URLSession sniffs + /// the body before reporting the response, holding it back until enough of + /// the body (or its end) arrives. + private func respond(status: Int, url: URL) { + let type = status == 200 ? "audio/pcm;rate=24000;channels=1" : "application/json" + guard + let response = HTTPURLResponse( + url: url, statusCode: status, httpVersion: "HTTP/1.1", headerFields: ["Content-Type": type]) + else { return } + client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed) + } +} diff --git a/Tests/BlurtEngineTests/StreamingPCMPlayerTests.swift b/Tests/BlurtEngineTests/StreamingPCMPlayerTests.swift new file mode 100644 index 00000000..8acfe427 --- /dev/null +++ b/Tests/BlurtEngineTests/StreamingPCMPlayerTests.swift @@ -0,0 +1,75 @@ +import Foundation +import Testing + +@testable import BlurtEngine + +/// The PCM player's bookkeeping, around real playback of silence: what it +/// reports queued and loud, how stopping and a bad rate are handled, and that a +/// drain always finishes. (The drain's bound — queued audio over the rate, plus +/// slack — is what keeps a read from sticking on "reading" when no output +/// device moves the clock; a frozen clock can't be staged here, so these check +/// the bound holds with one.) +@Suite("StreamingPCMPlayer", .timeLimit(.minutes(1))) +struct StreamingPCMPlayerTests { + /// `seconds` of silence at the TTS sample rate. + private func silence(seconds: Double) -> Data { + Data(count: Int(Double(AssemblyAISpeechSynthesizer.sampleRate) * seconds) * 2) + } + + @Test("a zero, negative or non-finite rate plays at 1 rather than trapping the drain") + func badRateFallsBack() async { + for rate in [0, -1, .nan, .infinity] { + let player = StreamingPCMPlayer(sampleRate: AssemblyAISpeechSynthesizer.sampleRate, rate: rate) + player.enqueue(silence(seconds: 0.05)) + let start = ContinuousClock.now + await player.drain() + #expect(ContinuousClock.now - start < .seconds(3), "rate \(rate)") + } + } + + @Test("a drain finishes within the audio queued plus its slack") + func drainIsBounded() async { + let player = StreamingPCMPlayer(sampleRate: AssemblyAISpeechSynthesizer.sampleRate, rate: 2) + player.enqueue(silence(seconds: 0.5)) + let start = ContinuousClock.now + await player.drain() + #expect(ContinuousClock.now - start < .seconds(0.25 + 2 + 0.5)) + #expect(player.bufferedSeconds == 0) + } + + @Test("queued audio is reported in listening time, and nothing once stopped") + func bufferedSeconds() { + let player = StreamingPCMPlayer(sampleRate: AssemblyAISpeechSynthesizer.sampleRate, rate: 2) + #expect(player.bufferedSeconds == 0) + player.enqueue(silence(seconds: 1)) + // A second of audio at 2× is half a second of listening, plus the + // schedule's start lead. + #expect(player.bufferedSeconds > 0.3 && player.bufferedSeconds <= 0.5 + PCMSchedule.lead) + player.stop() + #expect(player.bufferedSeconds == 0) + #expect(player.currentLevel == 0) + } + + @Test("stopping twice is safe, audio after a stop is ignored, and so is less than a sample") + func stopAndShortInput() async { + let player = StreamingPCMPlayer(sampleRate: AssemblyAISpeechSynthesizer.sampleRate) + player.enqueue(Data()) + player.enqueue(Data([1])) + #expect(player.bufferedSeconds == 0) + player.stop() + player.stop() + player.enqueue(silence(seconds: 1)) + #expect(player.bufferedSeconds == 0) + let start = ContinuousClock.now + await player.drain() + #expect(ContinuousClock.now - start < .seconds(1)) + } + + @Test("silence reads as no level") + func silenceHasNoLevel() { + let player = StreamingPCMPlayer(sampleRate: AssemblyAISpeechSynthesizer.sampleRate) + player.enqueue(silence(seconds: 0.5)) + #expect(player.currentLevel == 0) + player.stop() + } +} diff --git a/Tests/BlurtEngineTests/Stubs/DictationKeyRouter+UniformMode.swift b/Tests/BlurtEngineTests/Stubs/DictationKeyRouter+UniformMode.swift new file mode 100644 index 00000000..b29f5e90 --- /dev/null +++ b/Tests/BlurtEngineTests/Stubs/DictationKeyRouter+UniformMode.swift @@ -0,0 +1,22 @@ +@testable import BlurtEngine + +/// The router with every key under one mode — the shape the router suites were +/// written against, before each key had its own. Test-only: production always +/// binds per key (`DictationKeyRouter.init(activations:)`). +extension DictationKeyRouter { + init( + triggerKeyCodes: Set, holdThreshold: Duration = .seconds(1), activation: TriggerActivation = .tapOrHold + ) { + self.init( + activations: Dictionary(uniqueKeysWithValues: triggerKeyCodes.map { ($0, activation) }), + holdThreshold: holdThreshold) + } + + @discardableResult + mutating func rebind(triggerKeyCodes: Set, activation: TriggerActivation) -> Bool { + rebind(activations: Dictionary(uniqueKeysWithValues: triggerKeyCodes.map { ($0, activation) })) + } + + /// The keycodes the router watches. + var triggerKeyCodes: Set { Set(activations.keys) } +} diff --git a/Tests/BlurtEngineTests/Stubs/FakeHTTPTransport.swift b/Tests/BlurtEngineTests/Stubs/FakeHTTPTransport.swift index fca65623..58b2bea6 100644 --- a/Tests/BlurtEngineTests/Stubs/FakeHTTPTransport.swift +++ b/Tests/BlurtEngineTests/Stubs/FakeHTTPTransport.swift @@ -1,5 +1,6 @@ import Foundation import Synchronization +import Testing @testable import AssemblyAI @testable import BlurtEngine @@ -98,3 +99,24 @@ final class Counter: Sendable { count.withLock { $0 } } } + +/// An LLM Gateway stub: read-aloud only ever asks the gateway with +/// `data(for:)`, so a streaming upload is unreachable and needn't be restated +/// by each stub. +protocol GatewayStub: HTTPTransport {} + +extension GatewayStub { + func upload( + for request: URLRequest, streaming body: AsyncThrowingStream, + delegate: (any URLSessionTaskDelegate)? + ) async throws -> (Data, URLResponse) { + throw URLError(.unsupportedURL) + } + + /// A 200 answering `request` with `body`. + func ok(_ request: URLRequest, _ body: Data) throws -> (Data, URLResponse) { + let url = try #require(request.url) + let response = try #require(HTTPURLResponse(url: url, statusCode: 200, httpVersion: nil, headerFields: nil)) + return (body, response) + } +} diff --git a/Tests/BlurtEngineTests/Stubs/FakeSpeechSocket.swift b/Tests/BlurtEngineTests/Stubs/FakeSpeechSocket.swift deleted file mode 100644 index 82d0f058..00000000 --- a/Tests/BlurtEngineTests/Stubs/FakeSpeechSocket.swift +++ /dev/null @@ -1,91 +0,0 @@ -import Foundation -import Synchronization - -@testable import BlurtEngine - -/// Plays the TTS service's side of the socket. Each command the synthesizer -/// sends is passed to `respond`, and the frames it returns are queued for -/// `receive()`, the same way the live service answers a `Flush` with `Audio` -/// then `FlushDone`. A `receive()` with nothing queued suspends until more -/// arrives or the socket is closed, so a responder that returns nothing models a -/// server that never answers. -actor FakeSpeechSocket: SpeechSocket { - struct Record { - var sent: [String] = [] - /// nil while open; otherwise whether `close` was the normal (post-Terminate) kind. - var closedNormally: Bool? - } - - nonisolated let record = Mutex(Record()) - private let respond: @Sendable (String) -> [String] - private let feed: AsyncStream.Continuation - private var incoming: AsyncStream.Iterator - - init(respond: @escaping @Sendable (String) -> [String]) { - self.respond = respond - let (stream, feed) = AsyncStream.makeStream() - self.feed = feed - self.incoming = stream.makeAsyncIterator() - } - - func send(_ text: String) async throws { - guard record.withLock({ $0.closedNormally == nil }) else { throw URLError(.networkConnectionLost) } - record.withLock { $0.sent.append(text) } - for frame in respond(text) { feed.yield(frame) } - } - - func receive() async throws -> URLSessionWebSocketTask.Message { - var iterator = incoming - let next = await iterator.next(isolation: #isolation) - incoming = iterator - guard let next else { throw URLError(.networkConnectionLost) } - return .string(next) - } - - nonisolated func close(normally: Bool) { - record.withLock { if $0.closedNormally == nil { $0.closedNormally = normally } } - feed.finish() - } - - /// Drops the connection from the server's side. - nonisolated func drop() { feed.finish() } - - nonisolated var sent: [String] { record.withLock(\.sent) } - nonisolated var closedNormally: Bool? { record.withLock(\.closedNormally) } - - static func audio(_ bytes: [UInt8]) -> String { - #"{"type":"Audio","audio":"\#(Data(bytes).base64EncodedString())"}"# - } - - static let flushDone = #"{"type":"FlushDone"}"# - - /// The live service's answer to each command: nothing for `Generate` (it - /// only buffers), one audio chunk then an ack for `Flush`, numbered by flush. - static func service() -> @Sendable (String) -> [String] { - let flushes = Counter() - return { command in - guard command.contains(#""Flush""#) else { return [] } - let index = flushes.next() - return [audio([UInt8(index), 0]), flushDone] - } - } -} - -/// A `PCMSink` that records what it's asked to do instead of playing it. -final class RecordingSink: PCMSink { - struct Record { - var chunks: [Data] = [] - var drained = false - var stopped = false - } - - let record = Mutex(Record()) - - func enqueue(_ pcm: Data) { record.withLock { $0.chunks.append(pcm) } } - func drain() async { record.withLock { $0.drained = true } } - func stop() { record.withLock { $0.stopped = true } } - - var chunks: [Data] { record.withLock(\.chunks) } - var drained: Bool { record.withLock(\.drained) } - var stopped: Bool { record.withLock(\.stopped) } -} diff --git a/Tests/BlurtEngineTests/Stubs/FakeSpeechTransport.swift b/Tests/BlurtEngineTests/Stubs/FakeSpeechTransport.swift new file mode 100644 index 00000000..d0ab58af --- /dev/null +++ b/Tests/BlurtEngineTests/Stubs/FakeSpeechTransport.swift @@ -0,0 +1,113 @@ +import Foundation +import Synchronization + +@testable import BlurtEngine + +/// Plays the TTS service's side of `POST /v1/speech`. Each request is passed to +/// `respond`, which returns the status and the body's chunks, delivered as a +/// streamed response would be. Every request is recorded, so a test can read +/// back what was asked for. +final class FakeSpeechTransport: SpeechTransport, Sendable { + /// How a response body ends once its chunks are delivered. + enum End: Sendable { + case finish + case fail(any Error & Sendable) + /// Never finishes: a stalled server. + case hang + } + + private let respond: @Sendable (URLRequest) -> (Int, [Data]) + private let end: End + private let failure: (any Error & Sendable)? + private let recorded = Mutex<[URLRequest]>([]) + + init( + end: End = .finish, respond: @escaping @Sendable (URLRequest) -> (Int, [Data]) = FakeSpeechTransport.speaking() + ) { + self.respond = respond + self.end = end + self.failure = nil + } + + /// A transport whose request fails before any response, with `error`. + init(failing error: any Error & Sendable) { + self.respond = { _ in (0, []) } + self.end = .finish + self.failure = error + } + + var requests: [URLRequest] { recorded.withLock { $0 } } + + /// The `text` of every request, in order. + var spoken: [String] { + requests.compactMap { request in + let body = request.httpBody.flatMap { try? JSONSerialization.jsonObject(with: $0) } as? [String: Any] + return body?["text"] as? String + } + } + + func stream(_ request: URLRequest) async throws -> (HTTPURLResponse, AsyncThrowingStream) { + recorded.withLock { $0.append(request) } + if let failure { throw failure } + let (status, chunks) = respond(request) + let url = request.url ?? AssemblyAISpeechSynthesizer.endpoint + guard let response = HTTPURLResponse(url: url, statusCode: status, httpVersion: nil, headerFields: nil) else { + throw URLError(.badServerResponse) + } + let end = end + return ( + response, + AsyncThrowingStream { continuation in + for chunk in chunks { continuation.yield(chunk) } + switch end { + case .finish: continuation.finish() + case .fail(let error): continuation.finish(throwing: error) + case .hang: break + } + } + ) + } + + /// The live service's answer: 200 and one PCM chunk per request, numbered by + /// request, so a test can tell the pieces' audio apart. + static func speaking() -> @Sendable (URLRequest) -> (Int, [Data]) { + let count = Counter() + return { _ in (200, [Data([UInt8(count.next()), 0])]) } + } +} + +/// A `PCMSink` that records what it's asked to do instead of playing it. For +/// the pacing tests, `bufferedSeconds` can be set, and `bufferedAfterEnqueue` +/// makes each enqueue report that much audio queued, as a real player would. +final class RecordingSink: PCMSink { + struct Record { + var chunks: [Data] = [] + var drained = false + var stopped = false + var buffered: Double = 0 + var bufferedAfterEnqueue: Double? + } + + let record = Mutex(Record()) + + func enqueue(_ pcm: Data) { + record.withLock { record in + record.chunks.append(pcm) + if let queued = record.bufferedAfterEnqueue { record.buffered = queued } + } + } + func drain() async { record.withLock { $0.drained = true } } + func stop() { record.withLock { $0.stopped = true } } + + var chunks: [Data] { record.withLock(\.chunks) } + var drained: Bool { record.withLock(\.drained) } + var stopped: Bool { record.withLock(\.stopped) } + var bufferedSeconds: Double { + get { record.withLock(\.buffered) } + set { record.withLock { $0.buffered = newValue } } + } + var bufferedAfterEnqueue: Double? { + get { record.withLock(\.bufferedAfterEnqueue) } + set { record.withLock { $0.bufferedAfterEnqueue = newValue } } + } +} diff --git a/Tests/BlurtEngineTests/Stubs/WaitUntil.swift b/Tests/BlurtEngineTests/Stubs/WaitUntil.swift new file mode 100644 index 00000000..0358e55f --- /dev/null +++ b/Tests/BlurtEngineTests/Stubs/WaitUntil.swift @@ -0,0 +1,11 @@ +/// Polls `condition` every few milliseconds until it holds, or throws after +/// `timeout` — for state a test can't await directly (a router's callbacks, a +/// background task's effects). +@MainActor +func waitUntil(timeout: Duration = .seconds(5), _ condition: () -> Bool) async throws { + let deadline = ContinuousClock.now + timeout + while !condition() { + guard ContinuousClock.now < deadline else { throw CancellationError() } + try await Task.sleep(for: .milliseconds(5)) + } +} diff --git a/Tests/BlurtEngineTests/TriggerActivationTests.swift b/Tests/BlurtEngineTests/TriggerActivationTests.swift index 902dfef1..c7543fc8 100644 --- a/Tests/BlurtEngineTests/TriggerActivationTests.swift +++ b/Tests/BlurtEngineTests/TriggerActivationTests.swift @@ -8,27 +8,31 @@ import Testing /// act, in `DictationKeyGateTests`. @Suite("TriggerActivation") struct TriggerActivationTests { + /// The bound key's mode with no entry of its own: the shared slot. + private func sharedMode(_ defaults: UserDefaults) -> TriggerActivation { + TriggerKeyBindings(defaults: defaults).activation(for: .rightCommand) + } + @Test("defaults to tap-or-hold when unset") func defaultsToTapOrHold() { // The shipped behavior: an install that never touched the picker must keep // both gestures working. - #expect(TriggerActivationStore(defaults: freshDefaults()).activation == .tapOrHold) + #expect(sharedMode(freshDefaults()) == .tapOrHold) #expect(TriggerActivation.fromPersisted("") == .tapOrHold) } - @Test("persists and reads back a chosen mode") + @Test("an existing install's shared mode is still honoured") func roundTrips() { let defaults = freshDefaults() - let store = TriggerActivationStore(defaults: defaults) - store.activation = .hold - #expect(TriggerActivationStore(defaults: defaults).activation == .hold) + defaults.set(TriggerActivation.hold.rawValue, forKey: TriggerActivationStore.defaultsKey) + #expect(sharedMode(defaults) == .hold) } @Test("an unknown stored value falls back to the default") func unknownFallsBack() { let defaults = freshDefaults() defaults.set("DoubleTap", forKey: TriggerActivationStore.defaultsKey) - #expect(TriggerActivationStore(defaults: defaults).activation == .tapOrHold) + #expect(sharedMode(defaults) == .tapOrHold) } @Test("every mode names itself and explains itself, distinctly") @@ -37,14 +41,10 @@ struct TriggerActivationTests { // differently, or the picker offers two spellings of one behavior. #expect(Set(TriggerActivation.allCases.map(\.label)).count == TriggerActivation.allCases.count) #expect( - Set(TriggerActivation.allCases.map(\.guidance)).count == TriggerActivation.allCases.count) - #expect( - Set(TriggerActivation.allCases.map(\.stopHint)).count == TriggerActivation.allCases.count) - // The default's footer is the sentence that shipped before the picker - // existed — the unset experience must not reword itself. - #expect( - TriggerActivation.tapOrHold.guidance - == "Tap to start and tap again to stop, or hold the key and release to dictate.") + Set(TriggerActivation.allCases.map { $0.hint(for: .dictate) }).count == TriggerActivation.allCases.count) + #expect(TriggerActivation.tapOrHold.hint(for: .dictate) == "Tap to start and stop. Hold to talk.") + #expect(TriggerActivation.hold.hint(for: .readAloud) == "Hold to listen. Release to stop.") + #expect(TriggerActivation.tapOrHold.hint(for: .ask) == "Tap to start and stop. Hold to ask.") #expect( Set(TriggerActivation.allCases.map(\.finishHint)).count == TriggerActivation.allCases.count) } diff --git a/Tests/BlurtEngineTests/TriggerKeyStyleTests.swift b/Tests/BlurtEngineTests/TriggerKeyStyleTests.swift index a6e56c8d..1ec1c5fd 100644 --- a/Tests/BlurtEngineTests/TriggerKeyStyleTests.swift +++ b/Tests/BlurtEngineTests/TriggerKeyStyleTests.swift @@ -3,16 +3,19 @@ import Testing @testable import BlurtEngine -/// Per-key styles: which trigger keys dictate (`TriggerKeyStyleStore`), and the -/// style one dictation uses without moving the main window's selection -/// (`PressStyle` over `StyleProfileStore.profile(for:)`). -@Suite("TriggerKeyStyle") +/// What each trigger key does and how it's pressed (`TriggerKeyBindings`, over +/// `TriggerKeyStyleStore` and `TriggerKeyActivationStore`), and the style one +/// dictation uses without moving the main window's selection (`PressStyle` +/// over `StyleProfileStore.profile(for:)`). +@Suite("TriggerKeyBindings") struct TriggerKeyStyleTests { private let defaults = freshDefaults() private let casual = StyleProfile(name: "Casual", instructions: "keep it breezy") private let formal = StyleProfile(name: "Formal", instructions: "no contractions") - private var keyStyles: TriggerKeyStyleStore { TriggerKeyStyleStore(defaults: defaults) } + private var choices: TriggerKeyStyleStore { TriggerKeyStyleStore(defaults: defaults) } + private var modes: TriggerKeyActivationStore { TriggerKeyActivationStore(defaults: defaults) } + private var bindings: TriggerKeyBindings { TriggerKeyBindings(defaults: defaults) } /// A style store holding both profiles with Formal selected in the main window. private func formalSelected() -> StyleProfileStore { @@ -22,99 +25,143 @@ struct TriggerKeyStyleTests { return store } - @Test("an untouched install binds exactly the one key, with the current style") - func untouchedIsTodaysBehavior() { - #expect(keyStyles.styles.isEmpty) - #expect(keyStyles.enabledKeys == [.rightCommand]) - #expect(keyStyles.styles[.rightCommand] == nil) + @Test("an untouched install: the bound key dictates, left ⌥ reads aloud, the rest are off") + func untouchedDefaults() { + #expect(defaults.string(forKey: TriggerKeyStyleStore.defaultsKey) == nil) + #expect(bindings.enabledKeys == [.rightCommand, .leftOption]) + #expect(bindings.choice(for: .rightCommand) == .dictate) + #expect(bindings.action(for: .leftOption) == .readAloud) + // Left ⌘ is off: it's down for every shortcut and ⌘-click. + #expect(bindings.action(for: .leftCommand) == .off) TriggerKeyStore(defaults: defaults).triggerKey = .function - #expect(keyStyles.enabledKeys == [.function]) - } - - @Test("styles round-trip, and a key given one dictates beside the bound key") - func stylesRoundTripAndEnableKeys() { - #expect(keyStyles.setStyle(casual.id.uuidString, for: .rightOption)) - #expect(keyStyles.setStyle(TriggerKeyStyleStore.currentStyleID, for: .function)) - #expect(TriggerKeyStyleStore(defaults: defaults).styles[.rightOption] == casual.id.uuidString) - #expect(keyStyles.enabledKeys == [.rightCommand, .rightOption, .function]) - #expect(keyStyles.styles[.function] == TriggerKeyStyleStore.currentStyleID) - #expect(keyStyles.styles[.rightCommand] == nil) - } - - @Test("only turning a key on or off reports a rebind; a style change doesn't") - func setStyleReportsEnabledChanges() { - #expect(keyStyles.setStyle(TriggerKeyStyleStore.currentStyleID, for: .rightOption)) - #expect(!keyStyles.setStyle(casual.id.uuidString, for: .rightOption)) - #expect(keyStyles.setStyle("", for: .rightOption)) - #expect(!keyStyles.setStyle("", for: .rightOption)) - #expect(keyStyles.styles.isEmpty) - // The bound key dictates either way, so giving it a style rebinds nothing. - #expect(!keyStyles.setStyle(formal.id.uuidString, for: .rightCommand)) - } - - @Test("the bound key set back to the main window's style stores no entry") - func boundKeyDefaultStoresNothing() { - _ = keyStyles.setStyle(casual.id.uuidString, for: .rightCommand) - _ = keyStyles.setStyle(TriggerKeyStyleStore.currentStyleID, for: .rightCommand) - #expect(keyStyles.styles.isEmpty) - } - - @Test("rebinding stops the key moved away from dictating, and keeps the others") - func rebindClearsTheOldBoundKey() { - _ = keyStyles.setStyle(casual.id.uuidString, for: .rightCommand) - _ = keyStyles.setStyle(formal.id.uuidString, for: .function) - TriggerKeyStore(defaults: defaults).triggerKey = .rightOption - #expect(keyStyles.enabledKeys == [.rightOption, .function]) - #expect(keyStyles.styles == [.function: formal.id.uuidString]) - } - - @Test("a key bound away and back gets its style back") - func rebindBackRestoresTheStyle() { - let keys = TriggerKeyStore(defaults: defaults) - _ = keyStyles.setStyle(casual.id.uuidString, for: .rightCommand) - keys.triggerKey = .rightOption - #expect(keyStyles.enabledKeys == [.rightOption]) - keys.triggerKey = .rightCommand - #expect(keyStyles.enabledKeys == [.rightCommand]) - #expect(keyStyles.styles == [.rightCommand: casual.id.uuidString]) - } - - @Test("a key set up as an extra before it was bound is an extra again once unbound") - func extraKeySurvivesBeingBound() { - let keys = TriggerKeyStore(defaults: defaults) - _ = keyStyles.setStyle(casual.id.uuidString, for: .rightOption) - keys.triggerKey = .rightOption - // Bound, it keeps dictating with the style it had as an extra. - #expect(keyStyles.styles[.rightOption] == casual.id.uuidString) - _ = keyStyles.setStyle(formal.id.uuidString, for: .rightOption) - keys.triggerKey = .rightCommand - #expect(keyStyles.enabledKeys == [.rightCommand, .rightOption]) - #expect(keyStyles.styles[.rightOption] == casual.id.uuidString) - // And bound again, the style it was given while bound. - keys.triggerKey = .rightOption - #expect(keyStyles.styles[.rightOption] == formal.id.uuidString) + #expect(bindings.enabledKeys == [.function, .leftOption]) + } + + @Test("any key can read aloud or ask, and Off sticks where the default isn't Off") + func actions() { + choices.setChoice(.ask, for: .leftControl) + choices.setChoice(.readAloud, for: .rightOption) + choices.setChoice(.off, for: .leftOption) + #expect(bindings.action(for: .leftControl) == .ask) + #expect(bindings.action(for: .rightOption) == .readAloud) + #expect(bindings.action(for: .leftOption) == .off) + #expect(bindings.enabledKeys == [.rightCommand, .rightOption, .leftControl]) + // Back to their defaults: no entries left behind. + choices.setChoice(.readAloud, for: .leftOption) + choices.setChoice(.off, for: .rightOption) + choices.setChoice(.off, for: .leftControl) + #expect(defaults.string(forKey: TriggerKeyStyleStore.defaultsKey) == "{}") + } + + @Test("choices keep the stored form styles had before read and ask") + func rawForm() { + for choice in [TriggerKeyChoice.off, .readAloud, .ask, .dictate, .dictate(style: UUID().uuidString)] { + #expect(TriggerKeyChoice(raw: choice.raw) == choice) + } + #expect(TriggerKeyChoice(raw: "current") == .dictate) + } + + @Test("the bound key dictates whatever its entry says") + func boundKeyAlwaysDictates() { + let bindings = TriggerKeyBindings( + boundKeyCode: TriggerKey.rightCommand.keyCode, choicesRaw: #"{"54":"read-aloud"}"#, modesRaw: "", + sharedModeRaw: "") + #expect(bindings.choice(for: .rightCommand) == .dictate) + } + + @Test("taking the bound key off dictation moves the binding to the next dictating key") + func movingTheBinding() { + choices.setChoice(.dictate(style: casual.id.uuidString), for: .function) + #expect(choices.setChoice(.readAloud, for: .rightCommand)) + #expect(bindings.bound == .function) + #expect(bindings.action(for: .rightCommand) == .readAloud) + // The new bound key keeps the style it dictated with. + #expect(bindings.choice(for: .function) == .dictate(style: casual.id.uuidString)) + } + + @Test("the only dictating key can't be taken off dictation") + func lastDictatingKeyStays() { + #expect(!choices.setChoice(.off, for: .rightCommand)) + #expect(bindings.bound == .rightCommand) + #expect(bindings.dictatingKeys == [.rightCommand]) } @Test("a corrupt value or an unknown keycode binds nothing new") func corruptOrUnknownDecodesToNothing() { - #expect(TriggerKeyStyleStore.styles(decoding: "not json").isEmpty) - #expect(TriggerKeyStyleStore.styles(decoding: #"{"12":"current","x":"current"}"#).isEmpty) - #expect(TriggerKeyStyleStore.styles(decoding: #"{"61":"current"}"#) == [.rightOption: "current"]) + #expect(TriggerKeyMap.decode("not json").isEmpty) + #expect(TriggerKeyMap.decode(#"{"12":"current","x":"current"}"#).isEmpty) + #expect(TriggerKeyMap.decode(#"{"61":"current"}"#) == [.rightOption: "current"]) } - @Test("a row shows Off, the current style, or a style that still exists") + @Test("a row shows Off, the main window's style, a style that still exists, read or ask") func rowChoice() { - let profiles = [casual] - let styles: [TriggerKey: String] = [.rightOption: casual.id.uuidString, .function: formal.id.uuidString] - func choice(_ key: TriggerKey, _ styles: [TriggerKey: String]) -> String { - TriggerKeyStyleStore.choice(for: key, bound: .rightCommand, styles: styles, profiles: profiles) + func bindings(_ raw: String) -> TriggerKeyBindings { + TriggerKeyBindings(boundKeyCode: 54, choicesRaw: raw, modesRaw: "", sharedModeRaw: "") } - #expect(choice(.rightCommand, styles) == TriggerKeyStyleStore.currentStyleID) - #expect(choice(.rightOption, styles) == casual.id.uuidString) + let styled = bindings(#"{"61":"\#(casual.id.uuidString)","63":"\#(formal.id.uuidString)"}"#) + #expect(styled.choice(for: .rightCommand, profiles: [casual]) == .dictate) + #expect(styled.choice(for: .rightOption, profiles: [casual]) == .dictate(style: casual.id.uuidString)) // Formal was deleted: the row shows what a press now does. - #expect(choice(.function, styles) == TriggerKeyStyleStore.currentStyleID) - #expect(choice(.function, [:]) == "") - #expect(choice(.function, [.function: StyleProfileStore.defaultStyleID]) == StyleProfileStore.defaultStyleID) + #expect(styled.choice(for: .function, profiles: [casual]) == .dictate) + #expect(bindings("").choice(for: .function, profiles: []) == .off) + #expect(bindings("").choice(for: .leftOption, profiles: []) == .readAloud) + let base = bindings(#"{"63":"\#(StyleProfileStore.defaultStyleID)","59":"ask"}"#) + #expect(base.choice(for: .function, profiles: []) == .dictate(style: StyleProfileStore.defaultStyleID)) + #expect(base.choice(for: .leftControl, profiles: []) == .ask) + } + + @Test("modes: a key's own, else left ⌥'s read is tap or hold, else the shared mode") + func activations() { + defaults.set(TriggerActivation.hold.rawValue, forKey: TriggerActivationStore.defaultsKey) + choices.setChoice(.dictate(style: casual.id.uuidString), for: .rightOption) + #expect(bindings.activation(for: .rightCommand) == .hold) + #expect(bindings.activation(for: .leftOption) == .tapOrHold) + #expect(bindings.activation(for: .rightOption) == .hold) + modes.setActivation(.tap, for: .leftOption) + modes.setActivation(.tap, for: .rightCommand) + #expect(bindings.activation(for: .leftOption) == .tap) + #expect(bindings.activation(for: .rightCommand) == .tap) + // Right ⌥ has no mode of its own, so it follows the shared one. + #expect(bindings.activations == [.rightCommand: .tap, .rightOption: .hold, .leftOption: .tap]) + } + + @Test("each key keeps its own mode when the binding moves") + func modesStayWithTheirKeys() { + choices.setChoice(.dictate, for: .rightOption) + modes.setActivation(.tap, for: .rightOption) + modes.setActivation(.hold, for: .rightCommand) + choices.setChoice(.off, for: .rightCommand) + #expect(bindings.bound == .rightOption) + #expect(bindings.activation(for: .rightOption) == .tap) + } + + @Test("each press resolves under its own key's mode") + func routerAppliesEachKeysMode() { + let command = TriggerKey.rightCommand.keyCode + let left = TriggerKey.leftCommand.keyCode + var router = DictationKeyRouter(activations: [command: .hold, left: .tap]) + // A long press of the tap key still latches; a short press of the hold key stops. + #expect(router.handle(.flagsChanged(keyCode: left, triggerFlagIsOn: true), at: .zero) == .start) + #expect(router.handle(.flagsChanged(keyCode: left, triggerFlagIsOn: false), at: .seconds(3)) == .latch) + #expect(router.handle(.flagsChanged(keyCode: left, triggerFlagIsOn: true), at: .seconds(4)) == .none) + #expect(router.handle(.flagsChanged(keyCode: left, triggerFlagIsOn: false), at: .seconds(5)) == .stop) + #expect(router.handle(.flagsChanged(keyCode: command, triggerFlagIsOn: true), at: .seconds(6)) == .start) + #expect( + router.handle(.flagsChanged(keyCode: command, triggerFlagIsOn: false), at: .seconds(6) + .milliseconds(100)) + == .stop) + } + + @Test("changing another key's mode keeps a live recording; changing the active key's discards it") + func rebindModesMidRecording() { + let command = TriggerKey.rightCommand.keyCode + let left = TriggerKey.leftCommand.keyCode + var router = DictationKeyRouter(activations: [command: .tap, left: .tap]) + _ = router.handle(.flagsChanged(keyCode: command, triggerFlagIsOn: true), at: .zero) + _ = router.handle(.flagsChanged(keyCode: command, triggerFlagIsOn: false), at: .milliseconds(100)) + let other = router.rebind(activations: [command: .tap, left: .hold]) + #expect(!other) + let active = router.rebind(activations: [command: .hold, left: .hold]) + #expect(active) } /// Raw `CGEventFlags` as macOS reports a lone press of each key: the generic @@ -123,6 +170,9 @@ struct TriggerKeyStyleTests { .rightCommand: 0x10_0000 | 0x10 | 0x100, .rightOption: 0x8_0000 | 0x40 | 0x100, .function: 0x80_0000 | 0x100, + .leftCommand: 0x10_0000 | 0x08 | 0x100, + .leftOption: 0x8_0000 | 0x20 | 0x100, + .leftControl: 0x4_0000 | 0x01 | 0x100, ] @Test("each trigger key's own flag bit reads as down, and another key's doesn't") @@ -141,11 +191,12 @@ struct TriggerKeyStyleTests { #expect(optionUp == .flagsChanged(keyCode: option, triggerFlagIsOn: false)) } - @Test("a router built from the store dictates with every key given a style") - func routerFromStoreFiresForStyledKeys() throws { - _ = keyStyles.setStyle(casual.id.uuidString, for: .rightOption) - _ = keyStyles.setStyle(TriggerKeyStyleStore.currentStyleID, for: .function) - var router = DictationKeyRouter(triggerKeyCodes: Set(keyStyles.enabledKeys.map(\.keyCode))) + @Test("a router built from the bindings fires for every key that does something") + func routerFromBindingsFires() throws { + choices.setChoice(.dictate(style: casual.id.uuidString), for: .rightOption) + choices.setChoice(.dictate, for: .function) + var router = DictationKeyRouter( + activations: Dictionary(uniqueKeysWithValues: bindings.activations.map { ($0.key.keyCode, $0.value) })) for (index, key) in [TriggerKey.rightOption, .function, .rightCommand].enumerated() { let start = Duration.seconds(index * 10) let flags = try #require(Self.pressedFlags[key]) @@ -160,9 +211,8 @@ struct TriggerKeyStyleTests { @Test("a key's style shapes one dictation and never moves the selection") func pressStyleOverridesWithoutPersisting() { let store = formalSelected() - _ = keyStyles.setStyle(casual.id.uuidString, for: .rightOption) - _ = keyStyles.setStyle(StyleProfileStore.defaultStyleID, for: .function) - _ = keyStyles.setStyle(TriggerKeyStyleStore.currentStyleID, for: .rightCommand) + choices.setChoice(.dictate(style: casual.id.uuidString), for: .rightOption) + choices.setChoice(.dictate(style: StyleProfileStore.defaultStyleID), for: .function) let formalStyle = PressStyle(instructions: "no contractions", name: "Formal") #expect(PressStyle(key: nil, defaults: defaults) == formalStyle) #expect(PressStyle(key: .rightCommand, defaults: defaults) == formalStyle) @@ -186,6 +236,6 @@ struct TriggerKeyStyleTests { func deletedStyleFallsBackToSelection() { let store = formalSelected() #expect(store.profile(for: UUID().uuidString) == formal) - #expect(store.profile(for: TriggerKeyStyleStore.currentStyleID) == formal) + #expect(store.profile(for: TriggerKeyChoice.mainWindowStyle) == formal) } } diff --git a/scripts/check-invariants.sh b/scripts/check-invariants.sh index 8ae97e69..c646e59c 100755 --- a/scripts/check-invariants.sh +++ b/scripts/check-invariants.sh @@ -85,15 +85,10 @@ TESTS="Tests/*.swift App/Blurt/BlurtUITests/*.swift App/BlurtiOS/BlurtiOSTests/* # design keeps every audio type out of it, so an import there is a decision # being reversed by accident. KEYBOARD="App/BlurtiOS/BlurtKeyboard/*.swift App/BlurtiOS/BlurtDesign/*.swift App/BlurtiOS/BlurtiOSCore/*.swift" -# The streaming-STT rule's one carve-out: the experimental read-selection-aloud -# feature speaks through AssemblyAI's streaming TTS, which is WebSocket-only. -# The rule is about *dictation* staying one request/one response, so the TTS -# client — that one file — is excluded; a socket anywhere else still fails. -NOT_TTS=":!Sources/BlurtEngine/TTS/AssemblyAISpeechSynthesizer.swift" # The client-side-LLM rule's one carve-out, on the same reasoning: read-aloud # shapes what the voice says through the LLM Gateway (the skip-jargon listening # rewrite, and the answer to a spoken question about a selection), because the -# TTS socket has no server-side rewrite or reply to ask for. The rule is about +# TTS API has no server-side rewrite or reply to ask for. The rule is about # *dictation* cleanup staying on the dictation request, so that client — that # one file — is excluded; a gateway call anywhere else still fails. NOT_READ_ALOUD_LLM=":!Sources/BlurtEngine/TTS/ReadAloudLLM.swift" @@ -125,7 +120,7 @@ PATTERNS=( ) SCOPES=( "$ENGINE $APP" - "$ENGINE $APP $NOT_TTS" + "$ENGINE $APP" "$ENGINE $APP $NOT_READ_ALOUD_LLM" "$ENGINE $APP" "$ENGINE" @@ -141,7 +136,7 @@ SCOPES=( ) ADVICE=( "MicCapture builds a fresh AVCaptureSession recorder per capture — a long-lived engine goes stale on a device switch" - "the dictation API returns the full text in one response; there is no streaming path (the one socket is the TTS client, AssemblyAISpeechSynthesizer.swift)" + "the dictation API returns the full text in one response; there is no streaming path, and no socket anywhere (read-aloud's TTS is a streaming HTTP response)" "cleanup is the API's server-side rewrite via config.llm_instruction on the same /v1/transcribe/live call (the one gateway client is read-aloud's, ReadAloudLLM.swift)" "transcription is a remote AssemblyAI call — no on-device ASR/LLM, no model cache" "leave language to the model's own detection; setting the field takes that away"