Repository navigation
feat(ai): add OpenAI-compatible endpoint - #138
jgilman1337 wants to merge 2 commits into
Conversation
Allow a custom OpenAI-compatible base URL and optional API key so local servers such as LM Studio, llama.cpp, and vLLM can be used as the AI provider. Co-authored-by: Cursor <cursoragent@cursor.com>
Local servers like llama-swap do not emit text-start on the Responses API stream, which broke agent chat. Co-authored-by: Cursor <cursoragent@cursor.com>
|
An additional fix is available in this PR. Calling The OpenAI-compatible provider was calling the Responses API (
|
Summary
Adds an OpenAI-compatible AI provider so JobSync can use a custom base URL and optional API key (LM Studio, llama.cpp, vLLM, and similar local servers).
This continues #79. The original contributor went MIA, so this rebases the work onto
devand includes the review requests from that PR.Changes
{baseURL}/v1/modelsand run inference through the OpenAI SDK client{ baseURL, apiKey }vs a plain string) without widening other providers toanyRelated Issues
Follows up #79
Testing
npx eslinton the changed files — passnpx tsc --noEmit— passopenai-compatible-keyresolution,getModel, and the models routeMade with Cursor