[dan-ince-aai]

What and why

A customer uses DictationTranscriber live dictation for audio under 2 minutes, and wants it to keep working up to 20 minutes without stitching anything together themselves. Today every entry point posts to POST /v1/transcribe/live, which caps a request at 120 s. Past it, production answers 413 {"error":"audio exceeds 120000 ms","error_code":"audio_too_large"}, so any live dictation over 2 minutes fails.

The Dictation API also serves POST /v1/transcribe, the buffered endpoint: the whole recording in one body, up to 20 minutes / 256 MB, with the same config fields and the same response shape. This PR makes the SDK fall back to it on its own when a live dictation runs past the cap.

The same behaviour is being built in the Node SDK and in the Blurt macOS app from one shared spec, so the three should match.

Behaviour

Applies to transcribe_live() and open_live() on both DictationTranscriber and AsyncDictationTranscriber:

1. Copy. While the fallback is on, the transcriber keeps an in-memory copy of every audio byte handed to the live upload, from the first byte. Only audio is copied, not the multipart framing.
2. Trigger. The fallback runs only when the live request fails with HTTP 413 and error_code == "audio_too_large". Every other failure (another 413 code, 401/404, 429, 5xx, transport errors) raises exactly as before.
3. Drain to key-up. On the trigger, the transcriber keeps reading the caller's audio source until it ends: the iterator is exhausted, or close()/result() is called on a live session. It appends to the copy as it reads. It wraps the source in its own iterator (_RecordedChunks / _AsyncRecordedChunks), so the producer can still be read after httpx stops pulling the request body. That covers a 413 that arrives mid-upload as well as one that arrives after the whole body was sent.
4. One buffered POST. The complete copy goes once to {dictation_base_url}/v1/transcribe as multipart, with the same config JSON (including sample_rate/channels for PCM) and the same audio filename and Content-Type. The body is sent with a Content-Length in 1 MB pieces, so the recording is never copied whole a second time. Its response is parsed and returned like a live result. If it fails, the transcriber raises its error (for example its own 413 past 20 minutes), not the live 413.
5. Bounded memory. Once the copy passes 256 MB (api.BUFFERED_MAX_BYTES, 256 MiB), the copy is dropped, the producer is not read further, and a DictationError(status_code=413, error_code="audio_too_large") is raised. See the reviewer note below on when this check runs.
6. Abort means abort. abort() or an exception from the producer still drops everything, with no fallback request. That includes an abort() after close() while the live 413 is still in flight: the session passes an "aborted?" check that runs just before the buffered POST.
7. Timeouts. Draining the producer is recording time and no timeout bounds it. The buffered POST gets the normal settings.dictation_http_timeout.

Opt-out

aai.DictationTranscriber(fallback_to_buffered=False)        # default True
aai.AsyncDictationTranscriber(fallback_to_buffered=False)

With False, behaviour is today's: no copy is kept, the request body is built exactly as before, and a dictation past 120 s raises the live 413. Both transcribers also expose a read-only fallback_to_buffered property.

Trade-offs

• Latency. After a fallback, the result waits for the whole recording to upload again and be transcribed. A long dictation therefore returns more slowly than a live one, where only the final segment is left at key-up. The real-API numbers below show about 1.7 to 1.9 s after key-up for 130 s of audio, against about 0.95 s for a 30 s live clip.
• Memory. The copy costs about 32 KB per second at 16 kHz mono, or about 38 MB at 20 minutes. It is capped at 256 MB.

The constructor, transcribe_live() and open_live() docstrings, the README Dictation section (new "Dictate for longer than 2 minutes" block) and CLAUDE.md all state both trade-offs plainly. Version bumped to 1.10.0.

How the live 413 actually surfaces (worth knowing)

Two probes against production with plain httpx/httpcore over HTTP/1.1 showed that the server keeps reading the live body after it has decided on the 413. httpx sent the whole body first and only then read the response:

• 130 s sent as 121 s fast plus a slow tail: httpx pulled all 130 s and got the 413 at the end.
• 121 s plus a 10-minute tail (720 s, 23 MB) with a 20 s timeout: httpx pulled all 720 s in about 1.9 s, with no stall and no write timeout, then got the 413.

So in production the 413 reaches the client at key-up, and the drain step is usually a no-op. The drain path still matters: if a server answers early and closes the connection, httpcore suppresses the write error and returns the response mid-upload. The unit tests cover that case with a transport that stops reading the body part-way.

The same finding is why the 256 MB guard runs on every append, not only while draining. Most of a long recording arrives during the live upload, so a drain-only check would leave memory unbounded until key-up. Any copy over 256 MB is already far past the live cap (120 s of 16-bit audio is at most about 23 MB), so the visible result is the same 413 audio_too_large, just raised sooner. Please reconcile this with the Node SDK and Blurt, since the shared spec says "while draining".

Test evidence

Unit tests (tests/unit/test_dictation.py, tests/unit/test_dictation_async.py) use a scripted httpx transport. It reads the live body for N pieces (or all of it) and then answers, which mimics httpcore stopping the body pull once a response is in. They cover:

• the success path is unchanged, with the fallback on and off (same chunked body, live route only)
• a 413 audio_too_large mid-upload: the producer keeps being read after the 413, and the buffered request carries all the audio, including chunks produced after the 413. It also has the same config, filename and Content-Type, a Content-Length, and no chunked encoding
• a 413 after the full body was sent, and a file-object source read only partly by the live request
• fallback_to_buffered=False raises the live 413 and doesn't read the producer further
• other 413 codes (payload_too_large, problem-details title, no code), 401, 429, 500 (even with error_code=audio_too_large) and 503 don't fall back
• a failed fallback raises its own error (a buffered 413 past 20 min, a 500)
• the 256 MB guard, both while draining and during the live upload; no guard when the fallback is off
• a producer exception while draining propagates with no fallback
• open_live(): audio written after the 413 reaches the buffered request; abort() while draining, or after close() with the 413 still pending, makes no fallback request
• async: the same cases for AsyncDictationTranscriber.transcribe_live() and AsyncDictationLiveSession

Two existing tests encoded "the buffered endpoint is never requested". They now say "not requested while the live request succeeds", and the envelope-parsing test for a 413 audio_too_large turns the fallback off.

Results:

• pytest tests/unit: 998 passed (958 on master)
• dictation tests on the httpx floors, emulating the httpx0.22 and httpx0.24 tox factors: 148 passed on each (httpx 0.22.0 / httpcore 0.14.7; httpx 0.24.1 / httpcore 0.17.3). That includes the explicit Content-Length on a streamed body.
• ruff format --check and ruff check on the changed files: clean
• mypy==1.5.1 --follow-imports=silent --ignore-missing-imports --python-version 3.9 on the changed assemblyai/ files, as the lint job runs it: clean

Real API (production, dictation.assemblyai.com, 16 kHz mono S16LE, config={"sample_rate":16000,"channels":1}). The producer sent the first 121 s in 0.25 s chunks, then 1 s chunks with 0.3 s sleeps, so it was still recording past the cap. An httpx event hook logged each endpoint:

Entry point	Audio	Requests (status)	Result audio_duration_ms	Words	Result after key-up
transcribe_live()	30 s	/v1/transcribe/live (200) only	30000	73	0.95 s
transcribe_live()	130 s	/v1/transcribe/live (413) → /v1/transcribe (200)	130000	300	1.73 s
open_live()	130 s	/v1/transcribe/live (413) → /v1/transcribe (200)	130000	300	1.77 s
AsyncDictationTranscriber.transcribe_live()	130 s	/v1/transcribe/live (413) → /v1/transcribe (200)	130000	300	1.88 s

In each 130 s run the live 413 arrived at key-up (within 10 ms), and the fallback posted the full 4,160,000 bytes. The buffered request_time_ms was about 1.5 to 1.65 s.

Notes for reviewers

1. The fallback depends on POST /v1/transcribe staying available. It is publicly documented today, but AssemblyAI/assemblyai-docs#490 proposes removing it from the docs. The two need reconciling before this ships: either the endpoint stays supported, or this fallback needs another target.
2. Raising the live endpoint's 120 s cap server-side would make this fallback unnecessary. Clients would then keep live latency for long dictations, with no second upload and no in-memory copy.
3. BUFFERED_MAX_BYTES is 256 MiB (268,435,456 bytes). If the server's 256 MB limit is decimal, it should be lowered to match, here and in the sibling clients.

🤖 Generated with Claude Code