[feat] Configure the context window for custom OpenAI-compatible models
What problem are you trying to solve?
At 4585c8e455ac5bdeb76b618343ce748ef08134f4, a custom OpenAI-compatible
endpoint can be selected with api_provider, base_url, and api_key_helper,
but crates/cli/src/agent.rs constructs every SessionConfig with
context_window: 200_000. A custom endpoint with a 32,768-token window therefore
cannot declare its actual context budget through the same settings.
The existing provider works: an external Rust test runner used
crab_api::create_backend with the ordinary configuration and streamed requests
through the native OpenAI serializer and SSE parser for tsubasa-fast and
tsubasa-pro. Four controlled requests covered text and HTTP 401, with a
4,096-token output request. Each outgoing body passed the real Tsubasa strict
schema and input/output budget check before a fixture response. The existing
38 OpenAI tests also pass. These checks do not establish a full CLI session or
live model qualification; the fixed session window is a separate source finding.
Proposed solution
Add an optional context-window setting for a selected custom model and pass it
to SessionConfig. Preserve the current default when no override is present.
The same value should govern compaction/admission, including output headroom.
Tests could load the override through normal settings and verify that session
construction uses 32,768, while an unconfigured model preserves existing behavior.
Alternatives considered
Reducing --max-tokens bounds output but does not correct a 200,000-token
session context assumption. A new provider or custom transport would duplicate
the already-working OpenAI path.
Prior art
The existing custom-provider settings establish the endpoint-selection route.
Related settings issue #6 and provider proposal #128 were reviewed; this request
specifically concerns the session context limit.
Scope
LLM provider, Settings, Session, CLI.
Are you willing to contribute?
Prepared with AI assistance on macOS arm64 with Rust 1.98.1. No credentials or
private prompts are included.
[feat] Configure the context window for custom OpenAI-compatible models
What problem are you trying to solve?
At
4585c8e455ac5bdeb76b618343ce748ef08134f4, a custom OpenAI-compatibleendpoint can be selected with
api_provider,base_url, andapi_key_helper,but
crates/cli/src/agent.rsconstructs everySessionConfigwithcontext_window: 200_000. A custom endpoint with a 32,768-token window thereforecannot declare its actual context budget through the same settings.
The existing provider works: an external Rust test runner used
crab_api::create_backendwith the ordinary configuration and streamed requeststhrough the native OpenAI serializer and SSE parser for
tsubasa-fastandtsubasa-pro. Four controlled requests covered text and HTTP 401, with a4,096-token output request. Each outgoing body passed the real Tsubasa strict
schema and input/output budget check before a fixture response. The existing
38 OpenAI tests also pass. These checks do not establish a full CLI session or
live model qualification; the fixed session window is a separate source finding.
Proposed solution
Add an optional context-window setting for a selected custom model and pass it
to
SessionConfig. Preserve the current default when no override is present.The same value should govern compaction/admission, including output headroom.
Tests could load the override through normal settings and verify that session
construction uses 32,768, while an unconfigured model preserves existing behavior.
Alternatives considered
Reducing
--max-tokensbounds output but does not correct a 200,000-tokensession context assumption. A new provider or custom transport would duplicate
the already-working OpenAI path.
Prior art
The existing custom-provider settings establish the endpoint-selection route.
Related settings issue #6 and provider proposal #128 were reviewed; this request
specifically concerns the session context limit.
Scope
LLM provider, Settings, Session, CLI.
Are you willing to contribute?
Prepared with AI assistance on macOS arm64 with Rust 1.98.1. No credentials or
private prompts are included.