Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ aai.settings.api_key = "your-key"
- `aai.AsyncSyncTranscriber` — Asyncio counterpart of `SyncTranscriber`. Same input types, config, result, and errors; `transcribe()`, `transcribe_live()` and `warm()` are coroutines; `open_live()` returns an `AsyncLiveSession`. Owns an HTTP pool: use `async with` or `await aclose()`, or pass an `aai.AsyncClient` to share one
- `aai.SyncTranscriptionConfig` — Sync options: `model` (default `universal-3-5-pro`), `prompt`, `keyterms_prompt`, `conversation_context`, `language_codes`, `timestamps`, `sample_rate`, `channels`
- `aai.SyncTranscriptResponse` — Sync result: `.text`, `.words` (`SyncWord` with `confidence` always, `start`/`end` only when `timestamps=True`), `.confidence`, `.audio_duration_ms`, `.session_id`, `.request_time_ms`
- `aai.DictationTranscriber` — Dictation API: audio uploaded as it is spoken over one live request, transcript (and optional LLM pass) out. Methods: `open_live()` (push-style `DictationLiveSession`: `write()`/`close()`/`result()`/`abort()`), `transcribe_live()` (iterable, file object, bytes or path), `warm()`. There is no buffered `transcribe()`: every entry point posts to `/v1/transcribe/live`
- `aai.DictationTranscriber` — Dictation API: audio uploaded as it is spoken over one live request, transcript (and optional LLM pass) out. Methods: `open_live()` (push-style `DictationLiveSession`: `write()`/`close()`/`result()`/`abort()`), `transcribe_live()` (iterable, file object, bytes or path), `warm()`. There is no buffered `transcribe()`: every entry point starts on `/v1/transcribe/live`, which caps audio at 120 s. Past the cap (a 413 `audio_too_large`), `fallback_to_buffered=True` (the default) reads the source to its end and posts the whole recording once to `/v1/transcribe` (up to 20 min / 256 MB): slower result, and an in-memory copy of the audio (~32 KB/s at 16 kHz mono). `fallback_to_buffered=False` keeps no copy and raises the 413
- `aai.AsyncDictationTranscriber` — Asyncio counterpart of `DictationTranscriber`; `open_live()` returns an `AsyncDictationLiveSession`. Owns an HTTP pool: use `async with` or `await aclose()`
- `aai.DictationConfig` — Dictation options: `sample_rate`, `channels`, `language_codes`, `keyterms_prompt`, `llm_instruction`, `rewrite_mode` (`aai.DictationRewriteMode.fast` default | `smart`). Rejects unknown fields
- `aai.DictationResponse` — Dictation result: `.text`, `.words` (`DictationWord` with `text` + `confidence`), `.confidence`, `.llm_response`, `.llm_error`, `.audio_duration_ms`, `.session_id`, `.request_time_ms`, `.sync_time_ms`, and `.final_text` (the LLM rewrite, falling back to `.text`)
Expand Down
27 changes: 25 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -682,7 +682,7 @@ except aai.SyncTranscriptError as error:

### **Dictation Examples**

`aai.DictationTranscriber` is for dictation: a person speaks a short note and gets it back as text, optionally cleaned up or reformatted by an LLM. It has one connection, the live upload — audio is sent as it is spoken, the service transcribes each speech segment as it lands, and when the speaker stops what remains is the final segment and the LLM pass. There is no job id and no polling. Audio must be WAV or raw 16-bit PCM, up to 120 s per request.
`aai.DictationTranscriber` is for dictation: a person speaks a short note and gets it back as text, optionally cleaned up or reformatted by an LLM. Every dictation starts on the live upload — audio is sent as it is spoken, the service transcribes each speech segment as it lands, and when the speaker stops what remains is the final segment and the LLM pass. There is no job id and no polling. Audio must be WAV or raw 16-bit PCM. The live upload takes up to 120 s; a longer dictation, up to 20 minutes, falls back to the buffered endpoint on its own (see below).

The result is a `DictationResponse`: `text` is the raw transcript, `llm_response` the rewritten text when the LLM pass ran, and `final_text` is the one to show the user — the rewrite when there is one, the raw transcript otherwise.

Expand Down Expand Up @@ -716,7 +716,7 @@ Leaving the `with` block calls `close()`. If the block raises, the session is ab

This is **not** the streaming API. A session returns one finished transcript when the audio ends; use `aai.RealTimeTranscriber` when you need words back while the speaker is still talking.

The server aborts an upload that goes silent for long, so keep writing until the speaker is done rather than pausing. Errors that would normally arrive at the end (bad key, rate limit, capacity, audio too large) can surface part-way through the upload; `result()` raises them as `DictationError`.
The server aborts an upload that goes silent for long, so keep writing until the speaker is done rather than pausing. Errors that would normally arrive at the end (bad key, rate limit, capacity) can surface part-way through the upload; `result()` raises them as `DictationError`. Audio past the 120 s live cap is not an error by default: see "Dictate for longer than 2 minutes" below.

</details>

Expand Down Expand Up @@ -869,6 +869,29 @@ The transcriber owns an HTTP connection pool: use `async with`, or call `await t

</details>

<details>
<summary>Dictate for longer than 2 minutes</summary>

The live upload is capped at 120 s of audio. Past it the server answers `413` with `error_code="audio_too_large"`, and by default the transcriber falls back to the buffered endpoint (`/v1/transcribe`, up to 20 minutes / 256 MB) for you: it keeps reading your audio source until it ends — the iterator is exhausted, or the session is closed — then posts the whole recording once, with the same config, and returns that result like a live one. Your code does not change.

The trade-offs:

- **Slower result.** After a fallback the result waits for the whole recording to upload and be transcribed, so a long dictation returns more slowly than a live one, where only the final segment is left once the speaker stops.
- **Memory.** To have the recording to resend, the transcriber keeps an in-memory copy of all the audio it uploads: about 32 KB per second at 16 kHz mono, so about 38 MB for 20 minutes. Past 256 MB the copy is dropped and a `413` `DictationError` is raised.

Only a `413` `audio_too_large` from the live endpoint falls back; every other error is raised as before. If the buffered request fails too (past 20 minutes, say), its own error is raised. `abort()`, or an exception from your producer, drops the dictation with no fallback request.

To keep dictations live-only and hold no copy of the audio, turn it off:

```python
transcriber = aai.DictationTranscriber(fallback_to_buffered=False)
# a dictation past 120 s now raises DictationError(status_code=413, error_code="audio_too_large")
```

`aai.AsyncDictationTranscriber` takes the same option.

</details>

<details>
<summary>Handle errors</summary>

Expand Down
2 changes: 1 addition & 1 deletion assemblyai/__version__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "1.9.0"
__version__ = "1.10.0"
26 changes: 20 additions & 6 deletions assemblyai/dictation/v1/_base.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,16 @@
from __future__ import annotations

import os
from typing import AsyncIterable, BinaryIO, Iterable, Mapping, Optional, Tuple, Union
from typing import (
AsyncIterable,
BinaryIO,
Callable,
Iterable,
Mapping,
Optional,
Tuple,
Union,
)
from urllib.parse import urlparse

from ... import _audio
Expand All @@ -19,11 +28,11 @@
"resolve_source",
]

# Audio for a dictation request. The live endpoint is the only one this
# client opens, so it takes every shape audio comes in: a producer that is
# still recording (an iterable of chunks, or a file object read as it fills)
# and audio that is already complete (bytes, or a local path), which goes out
# over the same connection as a single chunk.
# Audio for a dictation request. Every request starts on the live endpoint, so
# it takes every shape audio comes in: a producer that is still recording (an
# iterable of chunks, or a file object read as it fills) and audio that is
# already complete (bytes, or a local path), which goes out over the same
# connection as a single chunk.
AudioSource = Union[
str, bytes, bytearray, "os.PathLike[str]", BinaryIO, Iterable[bytes]
]
Expand Down Expand Up @@ -162,16 +171,19 @@ def __init__(
client: _client.Client,
config: DictationConfig,
owner: str,
fallback_to_buffered: bool,
) -> None:
self._client = client
self.config = config
self._owner = owner
self.fallback_to_buffered = fallback_to_buffered

def transcribe_live(
self,
*,
data: AudioSource,
config: Optional[DictationConfig],
cancelled: Optional[Callable[[], bool]] = None,
) -> DictationResponse:
config = config or self.config
chunks, filename, content_type = resolve_source(data, config, self._owner)
Expand All @@ -184,4 +196,6 @@ def transcribe_live(
audio_content_type=content_type,
config=_config_to_json(config),
timeout=self._client.settings.dictation_http_timeout,
fallback_to_buffered=self.fallback_to_buffered,
cancelled=cancelled,
)
Loading
Loading