Skip to content

feat(ai): add OpenAI-compatible endpoint - #138

Open
jgilman1337 wants to merge 2 commits into
Gsync:devfrom
jgilman1337:feat/openai-compatible-endpoints
Open

jgilman1337 wants to merge 2 commits into
Gsync:devfrom
jgilman1337:feat/openai-compatible-endpoints

Conversation

@jgilman1337

Copy link
Copy Markdown

Summary

Adds an OpenAI-compatible AI provider so JobSync can use a custom base URL and optional API key (LM Studio, llama.cpp, vLLM, and similar local servers).

This continues #79. The original contributor went MIA, so this rebases the work onto dev and includes the review requests from that PR.

Changes

  • Register an OpenAI-compatible provider with a configurable base URL and optional API key
  • List models from {baseURL}/v1/models and run inference through the OpenAI SDK client
  • Verify dual credentials ({ baseURL, apiKey } vs a plain string) without widening other providers to any
  • Require an explicit model selection instead of calling the API with an empty model name
  • Cover credential resolution, verification, and model listing in unit tests

Related Issues

Follows up #79

Testing

  • npx eslint on the changed files — pass
  • npx tsc --noEmit — pass
  • Added tests for dual-credential verification, openai-compatible-key resolution, getModel, and the models route
  • Configure the provider in Settings with a local server URL, optional key, then select a model under AI Provider

Made with Cursor

jgilman1337 and others added 2 commits October 6, 2026 11:50
Allow a custom OpenAI-compatible base URL and optional API key so local
servers such as LM Studio, llama.cpp, and vLLM can be used as the AI
provider.

Co-authored-by: Cursor <cursoragent@cursor.com>
Local servers like llama-swap do not emit text-start on the Responses API stream, which broke agent chat.

Co-authored-by: Cursor <cursoragent@cursor.com>
@jgilman1337

jgilman1337 commented Oct 6, 2026 •

Copy link
Copy Markdown
Author

An additional fix is available in this PR. Calling gemma4-e4b-it using this new provider was failing. OpenAI-compatible chat used the Responses API. Local servers only implement Chat Completions, so Gemma’s stream skipped text-start and the UI aborted.

The OpenAI-compatible provider was calling the Responses API (/v1/responses). Local servers like llama-swap speak Chat Completions (/v1/chat/completions).

createOpenAI()(model) in the AI SDK defaults to Responses. Gemma’s stream then came back as text-delta chunks with msg_… ids and no text-start. The UI stream requires text-start first, so chat failed with:

Received text-delta for missing text part…

1cdbb02 switches that provider to .chat(model) so it uses Chat Completions, which those local servers actually implement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant