Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,48 @@ recompiling. A breaking change waits for 2.0. (Before RC.1, `1.0.0-beta.N` and e
beta.4 → RC.1 changes that can break a build are listed under *Changed — breaking (pre-GA)* in the
RC.1 entry, and each entry's own "Beta caveats" apply to the betas.)

## 2.3.0 — GA — 2026-10-27

OpenAI's Decisions API as a fourth decider, and refusals in `Decision`. Additive only: code that
builds against 2.2.0 builds unchanged, and `LocalLMLabSDKInference` and `LocalLMLabSDKClaude` have
no source changes.

### Added

- **OpenAI Decisions in `LocalLMLabSDKRemote`.** `JevDecisionProvider` now speaks OpenAI's
`/v1/decisions` as well as Jev's wire format, chosen by `JevProviderConfig`'s new `wire`
(`DecisionWireFormat`: `.jev`, `.openAIDecisions`). The `.openAI(key:)` preset registers
`openai:gpt-6-luna` (fidelity `.native`). `.openAICompatibleDecisions(baseURL:auth:models:…)` is for
any other server that speaks the same API, such as Fireworks, Vercel AI Gateway or LocalAI. For a
server that token-scores a general model, pass `fidelity: .tokenScored(calibrated: false)`.
`probe(for:timeout:)` checks the key and model with `GET /v1/models`.
- **`OpenAIDecisionWire` in `LocalLMLabSDKCore`**, the codec for both directions, with OpenAI's
input limits (`DecisionLimits.openAIDecisions`) checked before anything is sent.
- **Refusals.** A backend can now decline a question: `Decision.refusals` lists the refused ids, and
`isRefused(_:)` checks one. A refused id has no entry in `answers`. Only OpenAI's Decisions API
refuses today; every other backend leaves `refusals` empty. `lab.decide` accepts a decision in
which each question is either answered or refused. `lab.evaluate` counts a refusal as a miss
(`QuestionScore.refused`) that adds no calibration sample. The Jev wire carries refusals as
`{"type":"refusal"}`.

### Changed

- **`Decision`'s JSON** gains `refusals`. JSON written by 2.2 and earlier still decodes, and a
decision without refusals encodes exactly as it did before. The new `Decision` and
`JevProviderConfig` initializers are overloads, not added defaulted parameters, so the existing
initializers are unchanged.

### Checksums (SHA-256)

```
LocalLMLabSDKCore-2.3.0-GA.xcframework.zip 7821197d0cf5b4c92e36af5c537295e594afbb55d60d2f98b0773a3625403575
LocalLMLabSDKClaude-2.3.0-GA.xcframework.zip 80e143412d0bd0538091b59ea050015d28828855809203ea51c1cd49675e8449
LocalLMLabSDKInference-2.3.0-GA.xcframework.zip 63dc8bcba5e91e1324809ca412a23ebb0b9e04386361436eacc0be9fe6d4a8e0
LocalLMLabSDKRemote-2.3.0-GA.xcframework.zip dae65d45bea196f6cd2538328fad1a0e33eceae5fac5ee965183706ac942daaf
```

---

## 2.2.0 — GA — 2026-10-22

Nemotron 3.5 support in `LocalLMLabSDKInference`; no API change (the public surface is identical to
Expand Down
10 changes: 5 additions & 5 deletions Components/Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -24,17 +24,17 @@ struct SDKRelease {
// or edit `defaultSDKVersion`. For a release not listed, add its entry (URL follows the pattern
// below; checksum is the `.sha256` next to the zip on that GitHub release). Keep this in step
// with `examples/model-switch/Package.swift`, which consumes Core through this package.
let defaultSDKVersion = "2.2.0-GA"
let defaultSDKVersion = "2.3.0-GA"

let knownSDKReleases: [String: SDKRelease] = [
"2.1.0-GA": SDKRelease(
url: "https://github.com/ancientcomputing/locallm/releases/download/v2.1.0-GA/LocalLMLabSDKCore-2.1.0-GA.xcframework.zip",
checksum: "8d541e06f81d1586c959d6159a259c759853c88b2c68385ab28de493af6c5776"
),
"2.2.0-GA": SDKRelease(
url: "https://github.com/ancientcomputing/locallm/releases/download/v2.2.0-GA/LocalLMLabSDKCore-2.2.0-GA.xcframework.zip",
checksum: "5efc40cae9f8a1bf53bfe03195c5365398bc9501b3e6b0feee3e5c67385537f4"
),
"2.3.0-GA": SDKRelease(
url: "https://github.com/ancientcomputing/locallm/releases/download/v2.3.0-GA/LocalLMLabSDKCore-2.3.0-GA.xcframework.zip",
checksum: "7821197d0cf5b4c92e36af5c537295e594afbb55d60d2f98b0773a3625403575"
),
]

func failManifest(_ message: String) -> Never {
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Notes:
1. The 2.2.0-GA release branch is at [https://github.com/ancientcomputing/locallm/tree/2.2.0-GA](https://github.com/ancientcomputing/locallm/tree/2.2.0-GA)
1. The 2.3.0-GA release branch is at [https://github.com/ancientcomputing/locallm/tree/2.3.0-GA](https://github.com/ancientcomputing/locallm/tree/2.3.0-GA)
2. The project website is at [https://thisbrain.ai/locallm/](https://thisbrain.ai/locallm/)

# LocalLM Lab
Expand Down Expand Up @@ -40,7 +40,7 @@ doesn't carry.

**Product page:** [thisbrain.ai/locallm](https://thisbrain.ai/locallm) — or
get the app directly:
[LocalLM Lab 2.2.0-GA (arm64 DMG)](https://github.com/ancientcomputing/locallm/releases/download/v2.2.0-GA/LocalLM.Lab-2.2.0-GA-arm64.dmg).
[LocalLM Lab 2.3.0-GA (arm64 DMG)](https://github.com/ancientcomputing/locallm/releases/download/v2.3.0-GA/LocalLM.Lab-2.3.0-GA-arm64.dmg).

## LocalLM Lab SDK

Expand Down
33 changes: 30 additions & 3 deletions docs/api-surface.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> Machine-generated by `scripts/dump-public-api.sh` from the built `.swiftinterface`s.
> Do not hand-edit. `sdk-guide.md` §12 is the *narrative* reference; this is the check.
> SDK version: `"2.2.0"` (the published `v2.2.0-GA` xcframeworks, built from dd96469) · generated 2026-10-22T20:25Z
> SDK version: `"2.3.0"` (the published `v2.3.0-GA` xcframeworks, built from 9b45072) · generated 2026-10-27T19:05Z

## LocalLMLabSDKCore

Expand Down Expand Up @@ -801,15 +801,18 @@ public var modelID: ModelID
public var fidelity: DecisionFidelity
public var latency: Duration
public var usage: DecisionUsage?
public var refusals: Set<String>
public init(answers: [String : DecisionAnswer], modelID: ModelID, fidelity: DecisionFidelity, latency: Duration, usage: DecisionUsage? = nil)
public init(answers: [String : DecisionAnswer], modelID: ModelID, fidelity: DecisionFidelity, latency: Duration, usage: DecisionUsage? = nil, refusals: Set<String>)
public func isRefused(_ id: String) -> Bool
public init(from decoder: any Decoder) throws
public func encode(to encoder: any Encoder) throws
public func noul(_ id: String) -> Double?
public func choice(_ id: String) -> ChoiceAnswer?
public func score(_ id: String) -> ScoreAnswer?
public static func == (a: Decision, b: Decision) -> Bool
public func encode(to encoder: any Encoder) throws
public func hash(into hasher: inout Hasher)
public var hashValue: Int
public init(from decoder: any Decoder) throws
public enum DecisionError : Error, LocalizedError, Sendable, Equatable
case invalidRequest(String)
case noRoute(RouteName)
Expand Down Expand Up @@ -936,6 +939,7 @@ public var answers: [String : DecisionAnswer]
public var usage: DecisionUsage?
public var fidelity: DecisionFidelity?
public var errorMessage: String?
public var refusals: Set<String>
public static func encodeRequest(_ request: DecisionRequest, model: String) throws -> Data
public static func decodeRequest(_ data: Data) throws -> JevWire.Request
public static func encodeResponse(_ decision: Decision, model: String, request: DecisionRequest) throws -> Data
Expand All @@ -949,6 +953,7 @@ public var id: String
public var kind: DecisionCalibration.Sample.Kind
public var marked: Int
public var correct: Int
public var refused: Int
public var accuracy: Double
public var meanConfidenceWhenRight: Double?
public var meanConfidenceWhenWrong: Double?
Expand Down Expand Up @@ -1613,6 +1618,22 @@ case keepWarm([RouteName])
public static func == (a: ModelResidency, b: ModelResidency) -> Bool
public func encode(to encoder: any Encoder) throws
public init(from decoder: any Decoder) throws
public enum DecisionWireFormat : String, Sendable, Hashable, Codable
case jev
case openAIDecisions
public init?(rawValue: String)
public var rawValue: String
public static let openAIDecisions: DecisionLimits
public enum OpenAIDecisionWire
public static let maxTextCharacters: Int
public static let maxJSONDepth: Int
public static let maxJSONValues: Int
public static let maxJSONBytes: Int
public static func encodeRequest(_ request: DecisionRequest, model: String) throws -> Data
public static func decodeRequest(_ data: Data) throws -> JevWire.Request
public static func encodeResponse(_ decision: Decision, model: String, request: DecisionRequest) throws -> Data
public static func decodeResponse(_ data: Data, request: DecisionRequest) throws -> JevWire.Response
public static func errorMessage(in data: Data) -> String?
public struct PCCModelProvider : ModelProvider
public static var scheme: String
public init()
Expand Down Expand Up @@ -2553,10 +2574,14 @@ public var fidelity: DecisionFidelity
public var limits: DecisionLimits
public var responseLimits: RemoteResponseLimits
public var timeout: TimeInterval
public var wire: DecisionWireFormat
public init(scheme: String, displayName: String, endpoint: URL, auth: RemoteProviderConfig.Auth, models: [String] = [], allowArbitraryModelIDs: Bool = true, fidelity: DecisionFidelity, limits: DecisionLimits = .jev, responseLimits: RemoteResponseLimits = .default, timeout: TimeInterval = 60)
public init(scheme: String, displayName: String, endpoint: URL, auth: RemoteProviderConfig.Auth, models: [String] = [], allowArbitraryModelIDs: Bool = true, fidelity: DecisionFidelity, limits: DecisionLimits = .jev, responseLimits: RemoteResponseLimits = .default, timeout: TimeInterval = 60, wire: DecisionWireFormat)
public static func openRouter(key: String) -> JevProviderConfig
public static func featherless(key: String) -> JevProviderConfig
public static func featherlessDemo() -> JevProviderConfig
public static func openAI(key: String) -> JevProviderConfig
public static func openAICompatibleDecisions(baseURL: URL, auth: RemoteProviderConfig.Auth, models: [String], limits: DecisionLimits = .openAIDecisions, scheme: String = "openaicompat", fidelity: DecisionFidelity = .native) -> JevProviderConfig
public func validated() throws -> JevProviderConfig
public static func == (a: JevProviderConfig, b: JevProviderConfig) -> Bool
public struct JevDecisionProvider : DecisionProvider
Expand All @@ -2567,7 +2592,9 @@ public var limits: DecisionLimits
public func fidelity(for id: ModelID) -> DecisionFidelity
public func availability(for id: ModelID) -> ModelAvailability
public func decide(_ request: DecisionRequest, using id: ModelID) async throws -> Decision
public func probe(for id: ModelID, timeout: Duration = .seconds(8)) async -> ModelAvailability
public static func requestBody(model: String, request: DecisionRequest) throws -> Data
public static func requestBody(model: String, request: DecisionRequest, wire: DecisionWireFormat) throws -> Data
public static func openAI(apiKey: String, models: [RemoteModel] = []) -> RemoteProviderConfig
public static func anthropic(apiKey: String, models: [RemoteModel] = []) -> RemoteProviderConfig
public static func openAIResponses(apiKey: String, models: [RemoteModel] = []) -> RemoteProviderConfig
Expand Down
30 changes: 28 additions & 2 deletions docs/sdk-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ below for the working reference apps this guide is drawn from.

Requires macOS 26+ on Apple Silicon, Swift 6 tools.

**Status note**: this guide describes the API as of `2.2.0`. `2.0.0` was a one-time breaking release;
**Status note**: this guide describes the API as of `2.3.0`. `2.0.0` was a one-time breaking release;
every 2.x release after it is source compatible with it (see [§9](#9-scope-and-boundaries-what-core-doesnt-do)
for the compatibility policy). Coming from 1.x: [`migrating-to-2.0.md`](migrating-to-2.0.md); from
`0.8.x`: [`migrating-to-1.0.md`](migrating-to-1.0.md) first. New in 2.0: per-server trust and tool
Expand Down Expand Up @@ -1895,13 +1895,14 @@ is [`examples/model-switch`](../examples/model-switch/).*
A **decision model** (a *decider*) takes one input and a few typed questions, and answers each
with one of the options you allowed plus a probability for every option. It writes no text.
TypeSafe's **Jev** is the original; the SDK uses Jev's vocabulary and asks the same questions of
three backends:
four backends:

| Decider | Runs | Module | OS | `Decision.fidelity` |
|---|---|---|---|---|
| **OpenJev** (`OpenJevDecisionProvider`) | On the Mac, on an MLX model | Inference | macOS 27 | `.tokenScored(calibrated:)` |
| **TypeSafe Jev** via OpenRouter (`JevDecisionProvider(.openRouter(key:))`) | Hosted | Remote | macOS 26+ | `.native` |
| **Featherless Simple Jev** (`.featherless(key:)`, `.featherlessDemo()`) | Hosted | Remote | macOS 26+ | `.tokenScored(calibrated: false)` |
| **OpenAI Decisions** (`JevDecisionProvider(.openAI(key:))`, `openai:gpt-6-luna`; SDK 2.3) | Hosted | Remote | macOS 26+ | `.native` |

`lab.decide` and everything else in this section except OpenJev is in Core and works on macOS 26.

Expand Down Expand Up @@ -1945,8 +1946,13 @@ d.noul("refund") // Double: probability of yes
d.score("urgency") // ScoreAnswer: .score 1.6 (expected level, 0-based), .level, .probabilities, .legend
d.fidelity // .native or .tokenScored(calibrated:)
d.modelID, d.latency, d.usage // usage: tokens and cost, from hosted backends
d.isRefused("refund") // true if the backend declined that question (OpenAI Decisions only)
```

- **A backend can decline a question** (SDK 2.3). Only OpenAI's Decisions API does this today. A
refused id is in `d.refusals` and has no answer, so check `d.isRefused(id)` before reading it
and take the cautious path. `lab.evaluate` counts a refusal as a miss.

- **Question kinds** use Jev's names: `noul` (yes / no; the answer is P(yes)), `choice` (one of
several, each with an optional description), `score` (an ordered scale described in words).
Ids are what your code reads; `instructions` is what the model reads.
Expand Down Expand Up @@ -2031,6 +2037,26 @@ choice options keep your order on the wire, JSON `state` is sent as JSON, and `d
the tokens and cost. Like other remote providers, it's off by default and your UI should say
where inputs are sent.

**OpenAI's Decisions API (SDK 2.3)** uses the same provider with a different wire format,
`JevProviderConfig`'s `wire: .openAIDecisions` (the other presets use `.jev`):

```swift
try lab.models.register(decision: JevDecisionProvider(.openAI(key: key)))
lab.models.route(decision: "support", to: ModelID("openai:gpt-6-luna")!)
```

- `.openAI(key:)` offers `openai:gpt-6-luna` only, with `.native` fidelity: the probabilities are
OpenAI's own. Its limits are `DecisionLimits.openAIDecisions` (400 questions, 255 choice
options, 10 score levels), checked before anything is sent.
- `.openAICompatibleDecisions(baseURL:auth:models:)` is for any other server that speaks the same
API (Fireworks, Vercel AI Gateway, LocalAI): requests go to `baseURL` + `/decisions`. For a
server that token-scores a general model, pass `fidelity: .tokenScored(calibrated: false)`.
- `probe(for:timeout:)` checks the key and model with one `GET …/models`, never a decision:
`.needsCredential` for a rejected key, `.unsupportedModel` for a model the server doesn't list,
`.providerError` when it's unreachable or slow (8 seconds by default). On the Jev wire it
returns the local check only.
- It's the one backend that can decline a question: check `d.isRefused(id)` (above).

### A decider and your chat model: one model or two

The pattern the SDK is built for is two models per feature: a decider for the typed choices (does
Expand Down
Loading
Loading